ARES: Adaptive Red-Teaming and End-to-End Repair of Policy-Reward System
arXiv:2604.18789v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback RLHF is central to aligning Large Language Models LLMs, yet it introduces a critical vulnerability: an imperfect Reward Model RM can become a single point of failure when it fails to penalize unsafe...
arXiv:2604.18789v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback RLHF is central to aligning Large Language Models LLMs, yet it introduces a critical vulnerability: an imperfect Reward Model RM can become a single point of failure when it fails to penalize unsafe behaviors. While existing redteaming approaches primarily target policylevel weaknesses, they overlook what we term systemic weaknesses cases where both the core LLM and the RM fail in tandem.
이 콘텐츠는 ArXiv AI 원본 기사의 요약입니다. 전문은 원본 사이트에서 확인해주세요.
원문 기사 보기 →