Meet Talkie: The 13B-parameter LLM that doesn't know WWII happened

If your AI starts spouting the rhetorical stylings of a 1920s flapper, don’t panic. You’ve just stumbled onto Talkie, a new 13-billion-parameter language model from a trio of researchers that has a hard knowledge cutoff of December 31, 1930. Trained exclusively on pre-1931 books, newspapers, patents, and case law, Talkie is blissfully unaware of the Great Depression’s depths, the rise of the Nazis, and microwave ovens. It’s a deliberately anachronistic AI, and it’s the largest of its kind, according to its creators.

The point of this digital time capsule isn’t just novelty, though chatting with an AI that doesn’t know what an LLM is presents a genuinely weird loop. The team, which includes University of Toronto associate professor David Duvenaud, sees it as a research tool for testing AI’s ability to predict the future and forge new scientific ground. They’re eyeing a specific benchmark proposed by Google DeepMind’s Demis Hassabis: give an AI the same information Einstein had before 1915 and see if it can independently derive general relativity. That’s a towering intellectual bar, and Talkie is nowhere near clearing it. When benchmarked on Python programming tasks against a modern model with identical architecture, its successes were limited to trivial, one-line solutions.

Duvenaud outlined other, perhaps more grounded, uses. The model offers a window into cultural and legal interpretation as it existed at the time. “We can use these models to try to understand how a law would have been interpreted at the time it was written, based on the implicit assumptions and meaning of language at the time,” he told The Register. There’s also a philosophical angle: observing how an AI forms a self-conception when it has no concept of what an AI is. It’s a self-fulfilling prophecy without the modern context.

Performance-wise, the vintage model significantly underperforms its modern counterpart across standard evaluations, even when questions with anachronistic content are removed. The team openly acknowledges a “big capability gap” in both data and compute, calling it an “amateur research effort” they never expect to fully close. The project is less about building a useful assistant and more about creating a controlled artifact for studying how models think, predict, and reflect the world embedded in their training data.