Bocconi's 1,000-student test: ChatGPT boosts grades, causal training boosts originality

A randomized experiment with more than 1,000 first-year Bocconi University students found that ChatGPT access and a critical-thinking exercise improved student work in different, complementary ways. Students with access to GPT-4o scored nearly a full point higher on a five-point grading rubric for a real-world marketing assignment involving the university’s merchandise store. Their submissions contained more ideas, followed clearer logic, and looked more similar to expert recommendations.

But the critical-thinking training—which taught causal reasoning through a game, examples, and feedback—produced something the rubric didn’t capture. Those students explained more clearly why their ideas might succeed or fail, yet their rubric scores didn’t budge. The grading criteria only measured whether recommendations addressed two marketing goals: boosting awareness and store use. Text analysis told a different story. Students who completed the causal exercise generated a wider variety of ideas that were more distinct from their peers’ submissions.

Students who received both ChatGPT access and the training showed the strongest overall results. Their idea variety matched the training-only group, while their rubric scores and idea counts tracked with the ChatGPT-only group. They also demonstrated stronger logical coherence and more evidence of questioning assumptions.

This experiment, conducted with OpenAI Economic Research, stands out because its randomized design isolates what each intervention actually contributes. That’s rare in the fast-growing literature on AI in education. The findings suggest an uncomfortable truth for educators: if AI can help novices produce polished, expert-like answers, then grading only the final output misses what may matter more—whether a student generated an idea nobody else had. Schools may need to redesign assessments to capture originality, not just correctness. The question is whether rubrics built for a pre-AI classroom can survive contact with tools that make polished mediocrity effortless.