On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularization
{{output}}
Accurately aligning large language models (LLMs) with human preferences is crucial for informing fair, economically sound, and statistically efficient decision-making processes. However, we argue that the predominant approach for aligning LLMs with human prefe... ...
