AI Pulse by Inblix

Topic: model-serving

4 articles

Explore our coverage of model-serving — 4 curated articles, summaries, and related resources from the Inblix archive.

Stop benchmarking LLMs wrong: why time.perf_counter() alone is a trap — Inblix summary
Research

Stop benchmarking LLMs wrong: why time.perf_counter() alone is a trap

Machine Learning Mastery · Aug 4, 2026 · 2 min read

Measuring the performance of a large language model service isn't a single-number game. Anyone who's spent time optimiz...

Why your LLM's GPU is mostly idle—and how continuous batching fixes it — Inblix summary
Research

Why your LLM's GPU is mostly idle—and how continuous batching fixes it

Machine Learning Mastery · Aug 4, 2026 · 2 min read

Most GPUs serving AI models are sitting idle. The fix isn't a faster chip—it's smarter scheduling. Batching groups requ...

Hugging Face transformers now runs at native vLLM speed with zero porting — Inblix summary
Research

Hugging Face transformers now runs at native vLLM speed with zero porting

Hugging Face Blog · Jul 8, 2026 · 2 min read

The wall between using a model in Hugging Face transformers and deploying it at top speed in vLLM just crumbled. A new...

Hugging Face's TGI now serves 30 LoRA models from a single GPU deployment — Inblix summary
Research

Hugging Face's TGI now serves 30 LoRA models from a single GPU deployment

Hugging Face Blog · Jul 18, 2024 · 2 min read

Hugging Face just solved one of the most annoying problems in LLM deployment: paying for multiple GPUs when you need mu...