Tagged: text-generation-inference

5 articles

Latest

Hugging Face ships SageMaker container to run 12B LLMs on AWS GPUs

Llama 2 benchmark: GPTQ lets you run 13B on a single GPU at half the cost

Hugging Face ends its Gaudi fork, baking Intel AI chip support directly into TGI

Hugging Face TGI 1.4 brings OpenAI API drop-in replacement for open LLMs

Hugging Face TGI hits AWS Inferentia2, cutting GPU dependence for LLM deployments