ML//alignment//AI safety

AI safety is the work of ensuring that AI systems do what we actually want, beyond what we measured. Its three strands are alignment (values), robustness (edge cases) and interpretability (understanding).

AI safety is the work of ensuring that AI systems do what we actually want, beyond what we measured. Its three strands are alignment (values), robustness (edge cases) and interpretability (understanding).

The gap between optimizing the reward function and doing the right thing is the entire problem, and its central failure mode is Goodhart's curse: optimize a proxy hard enough and it diverges from the real goal.