GPT-4 meets K-12 data: Zelma turns test scores into plain talk

A Brown University economist has finally dragged state test data out of dusty PDFs and into a chat window. Dr. Emily Oster and her team of student researchers spent a year wrangling ELA and math scores for every U.S. student in grades 3-8 into one uniform format, then handed it to GPT-4. The result is Zelma, a research assistant that answers plain-language questions with charts, not spreadsheets.

“Zelma makes it possible for parents and policymakers alike to use plain language to get instant, tailored educational insights,” Oster said. That’s the pitch. In practice, you can ask for the top five non-charter districts in Minnesota for math proficiency and learn they all hover around 77%. Or you can surface racial disparities in California ELA scores over time and get a flag that a 2015 assessment change makes pre- and post-2015 results apples-to-oranges comparisons.

Novy, the development partner, handled the tricky bits. They used function calling so GPT-4 picks the right visuals and fields. They fine-tuned a model for a type-ahead that only suggests questions with available data. And they stashed known-good example graphs in a vector database to catch edge cases where the model might hallucinate a chart out of thin air. The design choices are just as important as the tech—all queries are public, which discourages junk prompts and lets users learn from each other. Zelma also shows its SQL code so you can verify its logic, or read the plain-language context if SQL isn’t your thing.

Who’s this actually for? A school board member can generate a visualization mid-meeting instead of waiting days for a district analyst. A parent comparing school districts before a move gets historical trends instantly. A governor’s chief of staff can pull up the best-performing schools in the state without a data team on standby. Oster’s team is betting that when adults in the system can have concrete conversations about student outcomes without first wrestling with data portals, those conversations actually happen. That’s the theory, anyway. The real test is whether anyone changes a decision because of what they asked Zelma.