The DRY principle—Don’t Repeat Yourself—is practically scripture in software engineering. Hugging Face’s Transformers library violates it constantly, copying attention mechanism code across more than 50 model files. Sometimes entire BERT implementations get duplicated with only a small tweak. That’s not sloppiness. It’s a deliberate design choice the team calls the single model file policy: everything needed for a model’s forward pass lives in exactly one file, and abstraction across models is routinely rejected.
Patrick von Platen, one of the library’s maintainers, explains the reasoning in a recent post. The first factor is community. Transformers is built for open-source contribution, and when a bug fix in one model can’t break a hundred others, contributors actually fix things. Reviewers can verify a PR without tracing a centralized attention layer through every architecture that depends on it. The second factor is that modeling code is the product. The library has been forked over 10,000 times and the Transformers paper cited more than a thousand times—clear evidence that users aren’t just calling APIs, they’re reading and modifying the source. A single file that contains all logic in order makes that dramatically easier.
The third reason is pace. Machine learning research moves so quickly that standardizing any component is a bet the field will punish. Two years ago, BERT’s self-attention looked like a safe abstraction. Then came T5’s relative positional embeddings, Reformer and Longformer’s chunked attention variants, and DeBERTa’s split attention mechanisms. Every new architecture would force a painful decision: retrofit the existing abstraction or spawn another awkwardly named function. The single model file policy sidesteps that entirely. Each model owns its implementation, for better or worse.
What this means in practice is that Transformers optimizes for readers over writers. Variable names are chosen for expressiveness, not brevity. New contributors copy existing code and tweak it rather than compose from shared modules. It’s an approach that would horrify most seasoned software architects—and it’s hard to argue with 10,000 forks. The bet is that in a field where models go stale in months, duplication is cheaper than premature abstraction. So far, the community seems to agree.
