A randomized experiment at Bocconi University just exposed something uncomfortable about how we grade student work. Researchers split 1,053 freshmen into four groups — some got a causal reasoning lesson, some got GPT-4o access, some got both, and some got neither. The students with ChatGPT scored nearly a full point higher on a 1-to-5 scale for a marketing assignment. Their answers had about two more ideas on average, were more logically coherent, and matched what subject-matter experts recommended more closely than the control group’s work.
But here’s the twist: the causal reasoning lesson didn’t improve traditional scores at all. In fact, those students scored slightly worse. What they did produce was more diverse thinking — more explanations of why a proposal would work, more attention to the conditions under which it might fail, and more divergence from what their peers wrote. The grading rubric simply didn’t reward any of that. It rewarded polish, structure, and staying inside the expected solution space. The authors are blunt about the implication: if you want originality to count, you have to build it into the grading criteria explicitly.
The study has real limitations worth naming. There was no follow-up test where students had to demonstrate understanding without AI. The authors acknowledge they can’t tell whether the GPT advantage reflected actual learning or just better output. Randomization happened across 13 class sections, not individual students. And several authors work at OpenAI, the company whose technology is being studied. That’s not disqualifying — but it’s the kind of detail that should make a careful reader pause.
What makes this more than an academic curiosity is how it fits with what we already know from larger datasets. Research covering over 500,000 US college grades found that top grades after ChatGPT’s launch increased most in writing- and programming-heavy courses with big homework components. Controlled experiments showed that when AI was taken away, students who had leaned on it performed worse than control groups. The drop was steepest among those who used GPT mainly to get direct answers. The Bocconi study can’t answer whether these freshmen will face the same cliff — because nobody asked them to prove what they retained. That’s the question that actually matters, and it’s still sitting there unanswered.
