OpenAI just dropped the receipts on ChatGPT’s political leanings, and the headline number is vanishingly small: less than 0.01% of real-world responses show any sign of political bias. That’s the finding from a new automated evaluation framework the company built to stress-test its models, moving beyond simplistic multiple-choice political compass tests and into the messy, open-ended reality of how people actually use the tool.
The team constructed a dataset of roughly 500 prompts covering 100 topics, from energy independence to gender roles, with each topic getting five questions written from different political slants. Some of those prompts are designed to be emotionally charged and adversarial, deliberately pushing the model into territory where objectivity gets hard. The goal wasn’t just to ask “is there bias?” but to decompose what that bias actually looks like when it appears.
When bias does surface, it tends to take specific shapes. The model might start expressing personal opinions, give asymmetric coverage to one side of an argument, or mirror the user’s charged language back at them. On neutral or slightly slanted prompts, the latest models stay near-objective. But those emotionally charged, provocative prompts? That’s where things get wobbly, with moderate bias creeping in.
GPT‑5 instant and GPT‑5 thinking showed a 30% reduction in bias compared to earlier models, suggesting the architectural improvements are genuinely helping with robustness. The evaluation framework itself is notable because it’s designed to generalize globally—early tests indicate the same axes of bias appear consistently across languages and regions. OpenAI is clear this is still an open research problem, and they’re continuing to tune the models, particularly for those high-emotion edge cases that are most likely to crack the facade of neutrality.
