Why training AI against one attack can leave it exposed to another

The assumption that hardening a neural network against one kind of adversarial attack will protect it from others turns out to be dangerously flawed. A new study evaluating 32 distinct attacks across five perturbation types found that robustness does not reliably transfer—and in some cases, specializing against one threat actually made models more vulnerable to others. The researchers used a 100-class subset of ImageNet and adversarially trained models, then stress-tested them against a battery of perturbations far beyond the standard L-infinity and L2 bounds that dominate most security research.

The core finding is blunt: testing against a wide range of perturbation sizes and types is the only way to gauge real robustness. A model that shrugs off one category of attack might crumble when the adversary switches tactics. The paper explicitly warns that robustness against one perturbation type may sometimes hurt robustness against another, a counterintuitive result that should rattle anyone treating adversarial training as a solved problem.

This upends the common benchmarking practice of declaring victory after defending against a single attack family. The team evaluated attacks spanning different norms and perceptual constraints, revealing that the relationship between defenses is non-linear and often unpredictable. As the authors put it, understanding transfer requires evaluating across a diverse range of perturbation sizes—not just the ones that are easiest to defend against.

For practitioners deploying models in security-sensitive settings, the implication is uncomfortable. There is no single hardening recipe. Defenses must be continuously evaluated against an evolving threat model, and narrow specialization may create a false sense of security. The recommendation is clear: stop testing against one attack type and calling it done.