NVIDIA has released Switchyard, an Apache 2.0 Rust proxy that solves a persistent headache for teams running coding agents: the API mismatch between what agents speak and what their backends expect. Claude Code talks Anthropic Messages, Codex CLI talks OpenAI, and plenty of teams want to serve models through vLLM, NIM, or Ollama without rewriting their agent stack. Switchyard sits in the middle, decoding inbound requests into provider-neutral types, routing them to a backend, and translating both requests and streaming responses back into the client’s native format.
The proxy accepts three inbound formats — OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages — and each can address any configured route. That decoupling is the whole point. The agent’s API and the backend’s API no longer need to match, which means teams can swap models behind the scenes without touching client code. NVIDIA ships four routing algorithms out of the box: passthrough for simple forwarding, random with optional weights and a seed for A/B testing, llm_classifier that calls a judge model to decide between weak and strong targets, and stage_router that scores recent tool-result signals to pick a capable or efficient tier without an extra classifier call on every turn.
Deployment comes in three flavors. The launcher path targets coding agents directly with switchyard launch claude, codex, or openclaw. A standalone server installs via cargo and validates configs with --dry-run. And switchyard-libsy embeds the routing logic in Rust applications without owning an HTTP stack — the algorithm picks a target but hands every model call back to the caller. Observability is built in, with Prometheus metrics on /metrics covering requests, errors, latency, token usage, and a routing overhead metric in milliseconds that strips out the model call time itself. A JSON routing log and per-session stats endpoint round out the picture.
The catch: this is pre-alpha software. NVIDIA labels it experimental, warns it’s not for production, and expects the API and algorithms to change significantly before v1.0. That’s honest framing, and it matters. Teams burned by breaking changes in fast-moving AI tooling will want to treat Switchyard as an evaluation target, not infrastructure. But the design is sound — translation layers like this have been the missing glue for multi-provider agent deployments, and a Rust implementation with typed routing algorithms suggests NVIDIA is serious about the problem, even if the version number says otherwise.
