Embeddings have quietly become the workhorse of modern NLP, powering everything from recommendation engines to semantic search. But the math gets ugly fast. A single embedding with 1024 dimensions in float32 eats 4KB. Need to search across 250 million vectors? That’s roughly a terabyte of memory, and at AWS’s estimated $3.80 per GB/month on x2gd instances, you’re looking at a bill that makes finance teams wince. The dominant fix so far has been dimensionality reduction — PCA and similar tricks — but those tend to underperform when applied to embeddings. Matryoshka Representation Learning from OpenAI changed the game by letting you just truncate to the first n dimensions. Nomic’s embed-text-v1.5 holds onto 95.8% of performance at 3x compression and 90% at 6x, while OpenAI’s text-embedding-3-large manages 93.1% retention at a 12x squeeze.
Binary quantization takes a different route. Instead of throwing away dimensions, it converts every float32 value to a single bit — 0 if the normalized value is negative, 1 if positive. That’s a 32x reduction with a single thresholding operation. The knock-on effect is that similarity search can use Hamming distance, which counts the number of bit positions where two vectors differ. Hamming distance tears through CPUs in just two cycles, which is why retrieval speeds jump by up to 32x. But raw binary search has a real cost: you drop to roughly 92.5% of full retrieval performance.
The fix comes from a 2021 paper by Yamada et al., who proposed a rescoring step. First, retrieve a larger candidate set using binary-to-binary Hamming distance. Then, re-rank those candidates by comparing the original float32 query embedding against the binary document embeddings using dot product. The float32 query retains the fine-grained signal that pure binary matching loses, while the document store stays at 1/32nd the size. With this hybrid approach, performance climbs back to roughly 96% of the full float32 baseline — a 3.5 percentage point recovery for a modest increase in compute.
Sentence Transformers makes the whole thing straightforward. You load a model like mixedbread-ai/mxbai-embed-large-v1, encode your corpus, and pack the binary values into bytes with np.packbits. A 1024-dimensional float32 embedding becomes a 128-dimensional uint8 array. The tradeoff is stark: you give up a few points of retrieval quality to slash your infrastructure bill by 32x. For production systems already paying five figures a month on vector databases, that’s not a marginal optimization — it’s the difference between scaling up and staying flat. The rescoring trick is the key insight here, and it’s worth noting how neatly it mirrors the two-stage retrieval architectures that have dominated search for decades.
