Community developer Maxence Dominici wanted a simple microservice: something that could read a Discord customer review and automatically flag it as positive or negative, so his team could respond quickly to unhappy customers. He didn’t need massive scale — around 2,000 requests a month — so he prioritized simplicity over raw performance. What followed was a trial-and-error journey through three different Google Cloud services before landing on a workable solution.
Dominici started with the Hugging Face transformers library, using the sentiment-analysis pipeline with DistilBERT base uncased finetuned on SST-2. He quickly discovered that the .h5 file he’d downloaded wasn’t a standard Keras model, but rather a weights checkpoint. That confusion pushed him toward the pipeline API, which worked well enough that he decided to serve it himself.
His first stop was Google’s AI Platform Prediction, which seemed promising for TensorFlow models. But the service is still in beta, and Dominici hit an issue that prevented him from using it. He then moved to App Engine, where TensorFlow wouldn’t install due to a missing system dependency. PyTorch worked, but an F4_1G instance couldn’t handle more than two concurrent requests — not exactly production-ready.
Cloud Run ended up being the winner. By containerizing the pipeline with Docker, Dominici got the memory and vCPU flexibility he needed, and PyTorch loaded the model faster than TensorFlow. His final setup is straightforward: a main.py file handling GET requests with the review text and an optional API key, a Dockerfile, the model folder with pytorch_model.bin, config.json, and vocab.txt, plus a requirements.txt. The whole thing is lean enough to serve predictions without overthinking infrastructure. It’s a useful reminder that serverless ML deployment often comes down to matching your framework to the platform’s quirks — and that beta services can burn more time than they save.
