Healthify tapped OpenAI to fix what custom ML couldn't

Healthify’s decade-long AI journey tells a story you don’t hear often enough: what happens when a company that built genuinely impressive custom models decides to rip them out and plug in someone else’s API. The Indian health platform, which counts over 40 million users and has helped shed more than 25 million pounds collectively, had already done the hard work. Their virtual nutritionist Ria used hierarchical LSTMs and custom NLU to handle the majority of user messages by 2020. Their Snap feature leveraged CNNs to recognize single Indian food items with roughly 80% accuracy. But those numbers masked real friction. CEO Tushar Vashisht admitted Snap was only used 10 to 20 percent of the time because it choked on photos with multiple foods. Ria was rules-based, so it couldn’t answer genuinely complex questions like how yesterday’s meal affected a user’s sleep. And scaling to new countries was a slog — it took them two years just to expand into Southeast Asia because every market required localizing models for its language, foods, and exercise norms.

So Healthify did what pragmatic engineering teams do. They tested everything. The team evaluated open-source models and integrated with “everybody who’s out there,” as Vashisht put it, before concluding OpenAI was the best fit. The turning point came when GPT-4 Vision instantly outperformed the entire Snap pipeline — not just on single items but on multi-food photos, and across global cuisines with zero localization effort. GPT-4 replaced Ria’s rules-based architecture for handling complex nutritional queries, and Whisper slotted into the Coach Co-pilot to transcribe calls. The integration wasn’t particularly dramatic from an engineering standpoint, which is exactly the point. Vashisht described building proofs-of-concept and production systems as remarkably straightforward.

What made the difference wasn’t just raw accuracy. Healthify needed to solve a deceptively mundane problem: matching food names returned by GPT-4 to the food names in Healthify’s own database. GPT has its own dictionary, and Healthify has theirs, and those two vocabularies don’t line up automatically. OpenAI’s Embeddings model handled that semantic matching in a way that went beyond simple text comparison. Fine-tuning also played a role — Healthify could ground the models to their specific use case without wrestling with infrastructure. The out-of-the-box fine-tuning capability meant they weren’t just using a generic model; they were adapting it to their domain with minimal configuration overhead.

There’s a quiet lesson here about the economics of AI build-vs-buy decisions. Healthify didn’t lack the talent to build custom models — they’d been doing it for years with demonstrable results. But maintaining and improving those systems, especially across markets, created a drag that a general-purpose API simply erased. The question now is whether this pattern holds for other health-tech companies with proprietary data moats, or if Healthify’s case is specific to the messiness of food recognition and nutrition coaching. Either way, 25 million pounds of aggregate weight loss suggests users aren’t complaining about what’s under the hood.