Stand Up a Private vLLM Server on HF Jobs With a Single Docker-Style Command

Hugging Face just made spinning up a private, GPU-backed vLLM server so simple it feels like a cheat code. If you’ve ever wanted to test a model, run evals, or batch-generate outputs without wrestling infrastructure, the new hf jobs run command is essentially docker run for HF’s cloud. You point it at the official vllm/vllm-openai image, pick a hardware flavor, and hit enter. Two minutes later you’ve got an OpenAI-compatible API endpoint gated behind your personal HF token.

The magic command sandwiches a few key flags between the job definition and the vLLM arguments. --flavor a10g-large grabs a single GPU, --expose 8000 punches the server’s port through HF’s proxy as a public (but authenticated) URL, and --timeout 2h acts as a billing safety net. After the image name, you pass the standard vLLM launch string: vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000. The terminal spits back a URL like https://<job_id>--8000.hf.jobs and you’re in business.

Querying it is dead simple since vLLM speaks the OpenAI API natively. A curl request needs only your HF token as a bearer token, and the Python snippet is a two-line change to the OpenAI client’s base_url and api_key. Crucially, the endpoint isn’t open to the world—every request requires a token with read access to the job’s namespace, making the jobs proxy your de facto API gateway without extra configuration.

The economics are transparent and forgiving. An a10g-large runs at $1.50 per hour, billed per second, and you can kill the server explicitly with hf jobs cancel to stop the meter. For larger models like Qwen3.5’s 122B mixture-of-experts beast, you just step up to a multi-GPU flavor like h200x2 and add --tensor-parallel-size 2 to shard the model. Hugh Hou detailed the entire workflow in a blog post that should have infrastructure engineers rethinking whether they need a dedicated serving layer for development work at all.