Text embeddings from large language models have become the default starting point for modern NLP classifiers, but they arrive with a nagging problem: nobody really knows what’s inside those dense vectors. A new hands-on tutorial from Scikit-LLM walks through exactly how to crack that open, using a combination of probing classifiers, UMAP projections, and SHAP values to inspect what an LLM actually learns about text.
The setup is refreshingly practical. Instead of paying for API calls, the workflow runs entirely on a local Ollama server, generating embeddings for 1,000 IMDB movie reviews — 500 positive, 500 negative — using the all-minilm model. A simple logistic regression classifier trained on those embeddings hit respectable precision and recall scores, which is notable because a shallow linear model performing well against 384-dimensional vectors suggests the embeddings are doing real semantic work, not just memorizing surface patterns.
UMAP visualization adds another layer of evidence. When projected down to two dimensions using cosine similarity, the scatterplot doesn’t show perfect separation between positive and negative reviews, but a subtle geographic pattern emerges: the southern region of the plot skews negative, while positive reviews cluster elsewhere. That’s actually a healthy sign — it means the semantic structure survived aggressive dimensionality reduction, which rarely happens if the embeddings are noise.
The final piece is SHAP, which identifies which latent dimensions in the embedding space exert the most influence on the classifier’s decisions. This is the step that transforms the exercise from academic curiosity into a debugging tool: instead of treating the LLM as an opaque oracle, you can see which axes of meaning are pulling the prediction one way or another. It’s the same principle data scientists apply to tabular models, finally extended to dense text representations.
For teams building production classifiers on top of LLM embeddings, this workflow offers a template for trust. You can now answer questions like “is my model actually learning sentiment, or just sentence length?” and “which embedding dimensions matter most?” — questions that were unanswerable when the embedding step was treated as a black box. The tutorial’s approach isn’t flashy, but it’s the kind of interpretability work that separates reliable systems from impressive demos.
