Google's BERT solved 11+ NLP tasks at once — here's how it works

Before BERT arrived in 2018 from Google AI Language, natural language processing was a patchwork of single-purpose models. You needed one model for sentiment analysis, another for named entity recognition, a third for question answering. BERT, short for Bidirectional Encoder Representations from Transformers, changed that calculus entirely by handling 11+ common language tasks with a single architecture — and outperforming the specialized models on most of them.

The trick was masked language modeling. Previous approaches processed text left-to-right or right-to-left, which meant they only ever saw half the context. BERT hides a word in a sentence and forces itself to predict it using words on both sides simultaneously. It’s the same thing humans do when a phone call cuts out mid-sentence — you fill in the blank from surrounding context. That bidirectional capability, combined with a training corpus of 3.3 billion words from Wikipedia (~2.5B) and Google’s BooksCorpus (~800M), gave BERT a depth of language understanding that earlier models couldn’t touch. Training took 64 TPUs running for four days.

Google put BERT into production search in November 2020, and the difference shows up in everyday queries. Search for ‘can you get medicine for someone pharmacy’ and pre-BERT Google would surface results about getting your own prescription filled. Post-BERT, the system understands the phrase ‘for someone’ changes the entire intent — you’re picking up medication on behalf of another person. That’s not a small tweak. It’s the difference between answering the question asked and answering a question that sounds similar.

For developers who don’t have racks of TPUs lying around, smaller variants like DistilBERT run 60% faster while retaining over 95% of BERT’s performance. Twenty-three lighter BERT models were released in March 2020 alone, reflecting real demand for running this kind of language intelligence on phones and laptops rather than data centers. The environmental cost of training large models remains a genuine concern, but the open-source release of BERT’s weights means most practitioners never pay that cost themselves — they fine-tune what Google already built.