Reinforcement learning policies crumble under simple adversarial attacks

The same adversarial example techniques that famously trick image classifiers into seeing a gibberish pattern as a panda have now proven devastatingly effective against reinforcement learning agents. In a new paper, researchers demonstrate that introducing tiny, often imperceptible perturbations to a policy’s raw input can cause performance to crater—regardless of the task or training algorithm used.

This isn’t just an academic curiosity. The threat model here envisions an adversary who can inject these small distortions at test time, a scenario that’s frighteningly plausible for real-world systems like autonomous navigation or robotic manipulation. The team tested a subclass of well-known crafting methods across both white-box settings, where the attacker has full knowledge of the model, and black-box settings, where they don’t.

The really sobering part? The attacks don’t need to be visually obvious to wreck performance. Perturbations that a human would shrug off—or not even notice—were enough to send a trained policy into a tailspin. The researchers characterized just how deep this vulnerability runs, showing it’s not a quirk of one particular approach to reinforcement learning. It’s systemic.

Videos accompanying the paper make the case more vividly than any chart could. Watching an agent that was perfectly competent moments earlier flail helplessly after a near-invisible tweak to its input underscores a structural weakness we haven’t adequately addressed. If we’re serious about deploying RL in safety-critical environments, the community needs to treat adversarial robustness not as an optional add-on, but as a first-class design constraint.