OpenAI and Georgetown Warn Language Models Will Supercharge Propaganda

A sobering new report from OpenAI, Georgetown’s Center for Security and Emerging Technology, and the Stanford Internet Observatory maps out exactly how advanced language models will become a propagandist’s dream tool. The research, which included an October 2021 workshop with 30 experts, marks one of the first systematic attempts to forecast AI-enabled disinformation before it explodes at scale.

The bottom-line judgment is blunt: language models will be useful for propagandists and will likely transform online influence operations. The technology threatens to disrupt three pillars of any disinformation campaign. First, it democratizes access by slashing the cost of running an operation, letting new, less-resourced bad actors set up shop. Second, it changes behavior by making expensive tactics like personalized messaging suddenly cheap, and it enables new ones like real-time chatbot propaganda. Third, the content itself becomes harder to trace and potentially more persuasive, especially for operators who lack the cultural or linguistic fluency to craft believable text on their own.

The researchers built a pipeline framework to pinpoint where defenses could work. For a language model to be weaponized, a propagandist needs the model to exist, needs reliable access to it, needs the ability to disseminate its output, and needs an end user to actually see it. Interrupting any link in that chain offers a mitigation opportunity—whether it’s controlling API access, developing better detection tools, or changing how platforms distribute content.

But there’s a grim practicality baked into the analysis. Even if companies lock down their most powerful models, the report acknowledges that propagandists will simply gravitate toward open-source alternatives. Nation states, with deeper pockets and fewer scruples, will likely invest in building their own. The report doesn’t peddle easy solutions, instead laying out a series of uncomfortable unknowns: When will user-friendly text generation tools become so widespread that anyone can spin up a propaganda mill? Will norms against AI-enabled ops even develop, and who would enforce them? It’s a call for more research to reduce the uncertainty, but the underlying message is that the window for getting ahead of this is closing fast.