Bridging the Gap Between Harmfulness Belief and Refusal Behavior for Safety Alignment

Lu Zhang, Chen Feng, Qingzhuo Wang, Wen Shen, Zhihua Wei

Safety-aligned LLMs can recognize harmful requests internally yet still produce compliant responses. We find that harmfulness information is not reliably preserved between the final instruction token and the response-start token. Bridging Harmfulness and Refusal (BHR) trains a LoRA adapter to carry this information to the response-start representation, establishing an additional pathway to refusal.

A belief loss maintains accurate harmfulness judgments at the instruction position. A belief-gated refusal loss uses these judgments to regulate refusal learning and reduce over-refusal on benign inputs. Experiments across multiple LLMs and safety benchmarks show improved robustness to diverse jailbreak attacks while preserving general capabilities.

Recommended citation: Lu Zhang, Chen Feng, Qingzhuo Wang, Wen Shen, Zhihua Wei. (2026). "Bridging the Gap Between Harmfulness Belief and Refusal Behavior for Safety Alignment." NeurIPS 2026.