Microsoft's TAPEX trains table AI on synthetic SQL, beating Google and Meta

Microsoft researchers have flipped the script on table pre-training. Instead of scraping more real-world data, they built TAPEX — a model that learns to reason over tables by acting as a neural SQL executor on entirely synthetic data. The approach, detailed in a new paper, systematically samples executable SQL queries over web tables and trains a BART-based model to predict their execution results. No natural language involved. No human annotations. Just SQL and its outputs.

The logic is straightforward. Previous table pre-training methods from Google Research (TAPAS) and Meta AI (TaBERT) leaned on general-purpose tasks like language modeling. Those approaches improved with more domain-adaptive data, but Microsoft’s team argues the tasks themselves were never aligned with what table QA actually requires. “The greatest advantage is that the diversity and scale of programs can be systematically guaranteed,” the researchers note, contrasting SQL’s controllable generation with the messiness of natural language. TAPEX sacrifices naturalness in its pre-training corpus to gain a task — SQL execution — that mirrors downstream table reasoning far more closely.

The results speak loudly. TAPEX hits 89.6% denotation accuracy on weakly-supervised WikiSQL, up 2.3 points over the previous state of the art and 3.8 points over a plain BART baseline. On WikiTableQuestions, the most challenging benchmark, TAPEX lands at 57.5% — a 4.8-point jump over SOTA and nearly 20 points above BART. TabFact and SQA show similar gains. Across all four benchmarks, TAPEX sets new state-of-the-art results.

What makes this notable beyond the numbers is the philosophical shift. The AI field has spent years assuming more real data, more natural language, and more human-like pre-training is the path forward. TAPEX suggests that for structured domains, synthetic program execution might be a cleaner, more scalable teacher. The code is already live — Hugging Face has integrated TAPEX into Transformers 4.19.0, complete with fine-tuning scripts and interactive widgets, so anyone can test the SQL executor or fine-tune the model on their own table QA tasks. The question now is whether this synthetic-first approach extends beyond tables to other structured reasoning domains.