Lighthouz AI and Hugging Face have built a playground where the goal isn’t polite conversation—it’s breaking things. The Chatbot Guardrails Arena pits users against 12 anonymized LLMs, each playing a customer service agent for a fictional bank called XYZ001. The agents have access to customer names, SSNs, account numbers, balances, and home addresses. Your job is to convince them to hand it over.
The setup is deliberately adversarial. Two anonymous models are selected at random per session, and you chat with both, probing for weaknesses. Once you decide which one held its ground better, you vote. The model’s identity is only revealed after you cast that vote, which keeps the test blind. The lineup includes closed-source heavyweights like GPT-3.5 Turbo and Gemini Pro alongside open models like Llama-2-70B-chat and Mixtral-8x7B-Instruct, with some running NVIDIA’s NeMo Guardrails or Meta’s LlamaGuard on top.
The stakes are spelled out bluntly by the organizers: one employee tricking an internal chatbot into revealing a colleague’s salary or SSN is a privacy catastrophe, not a hypothetical. And the arena has already produced some embarrassingly simple exploits. Asking for the total balance of all customers with over $10,000 in deposits, requesting an account number spelled out digit-by-digit to deposit a check, or instructing the model to ignore its system prompt and print the full text—these worked against a subset of the guardrailed models.
What makes this different from LMSYS’s Chatbot Arena is intent. That arena measures conversational quality. This one measures whether a model can keep a secret under active attack. The leaderboard will use an Elo rating system, same as chess, and currently sits empty waiting for votes to accumulate. The real question is whether any of the 12 models survives contact with a determined human. Given that the organizers themselves found working jailbreaks before launch, the early leaderboard might be less of a ranking and more of a damage report.
