SetFit Cut Witty Works' Training Data Needs to 20 Sentences Per Word

Witty Works, a Swiss company building inclusive-language tools since 2018, hit a wall with their writing assistant. Their rule-based approach, built on spaCy and a knowledge base of roughly 2,300 flagged words in German and English, handled about 85% of cases cleanly. The remaining 15% required understanding context — like distinguishing between “fossil fuels” and calling someone “an old fossil.” That’s a fundamentally different problem than pattern matching.

The initial plan was vanilla transformers to extract token embeddings, but Hugging Face experts pushed the team toward sentence embeddings instead. The reasoning was straightforward: a word’s meaning shifts based on its surrounding sentence, so representing the whole sentence captures that nuance. Witty Works then fed those embeddings into a classical classifier — KNN or logistic regression — to determine whether a flagged word was actually being used problematically in context.

The real breakthrough came from SetFit, the few-shot fine-tuning framework. Fine-tuning a standard BERT classifier would have demanded hundreds of annotated examples per word category, a labeling burden Witty Works couldn’t afford as a small team. SetFit’s contrastive learning approach changed the math entirely. Lead Data Scientist Elena Nazarenko reported the model performed adequately with just 15 to 20 labeled sentences per specific word. Hugging Face Chief Evangelist Julien Simon called SetFit “a great tool to add to the ML toolbox.”

This tracks with the broader shift toward parameter-efficient fine-tuning across the industry. Companies with modest annotation budgets have been stuck between brittle rule-based systems and data-hungry deep learning. SetFit and similar few-shot methods are collapsing that gap. What Witty Works demonstrated here isn’t exotic research — it’s the practical economics of shipping NLP features when labeled data is scarce. The team saved time and money by not building a large dataset, and that’s the kind of tradeoff that determines whether a product actually ships.