Building a text classification pipeline usually means choosing between coding everything yourself or handing the keys to an automated system. A new approach pairs Kili’s annotation platform with HuggingFace AutoTrain to create an active learning loop that handles both. The project uses a real dataset of roughly 40,000 Medium app reviews from Google Play to demonstrate the workflow.
AutoTrain handles the unglamorous parts of machine learning. Data cleaning, model selection, and hyperparameter optimization all run automatically. The framework supports binary and multi-label text classification, token classification, question answering, summarization, and text scoring across a wide language list including English, German, French, Spanish, Finnish, Swedish, Hindi, and Dutch. For languages outside that list, custom models and tokenizers remain an option.
Kili brings the human element. The platform supports annotation across images, video, text, PDF, and voice data with quality management tools built in. For this project, the team defined four review categories: Subscription, Content, Interface, and User Experience. The distinction between Interface and User Experience is worth noting — Interface captures UI complaints and payment issues, while User Experience holds the abstract, general sentiment that doesn’t fit elsewhere.
The workflow is iterative by design. Labeled data feeds into AutoTrain, the model trains, and the cycle repeats with new annotations. The team also plans to compare AutoTrain’s output against a manually built model using the same data. This is where things get interesting: AutoTrain’s convenience often comes at the cost of fine-grained control, so seeing how the two approaches stack up on the same dataset will tell us whether the automation tradeoff is worth it for text classification tasks. The full code and dataset are available on GitHub.
