ML//alignment//jailbreak

A jailbreak is an attack on a language model that bypasses its safety training to make it produce restricted content. It began with prompts like `Ignore previous instructions`, and now uses multi-turn attacks, encoding tricks and persona manipulation.

A jailbreak is an attack on a language model that bypasses its safety training to make it produce restricted content. It began with prompts like Ignore previous instructions, and now uses multi-turn attacks, encoding tricks and persona manipulation.

Jailbreaks show that RLHF alignment is a thin behavioral veneer rather than deep understanding. Red teaming searches for them before deployment, so that offense informs defense.