LLM-as-a-judge has baked-in bias: How to fix your RAG evals

Most teams ship an LLM feature after eyeballing a few outputs. Then a silent prompt tweak breaks something, and nobody knows until a user complains. That’s the default failure mode, and it’s exactly what application evaluation frameworks are designed to prevent. But here’s the catch: the “LLM-as-a-judge” mechanism almost all of them rely on has measurable, published biases you can’t afford to ignore.

The three dominant open-source tools in 2026 aren’t competing for the same job. RAGAS is your pick for research-backed retrieval scoring, with rigorous methodology around faithfulness and context precision. DeepEval expands that into a CI/CD-native testing suite designed to block bad deploys. Promptfoo is the lightweight, security-focused red-teaming tool you run before users find your prompt injection vulnerabilities. Mature GenAI programs don’t pick one; they run Promptfoo plus one of the others in parallel.

Before you even pick a framework, you need to get clear on what you’re actually measuring. Teams constantly conflate model benchmarking (MMLU, HumanEval) with application evaluation (does my RAG pipeline hallucinate on my data?) and production monitoring (LangSmith, Braintrust). If you’re asking “which eval framework should I use,” you almost certainly need the second category, probably paired with the third.

But the part most comparisons skip is the most uncomfortable: the judge is biased. Position bias means a model prefers responses based on their order in the prompt. Verbosity bias means longer, more confident-sounding wrong answers score higher than terse correct ones. Self-preference bias means models favor their own outputs. Using an LLM to grade another LLM without actively designing around these blind spots is just automating your confirmation bias. You need an audit harness that deliberately rotates response order, normalizes for length, and compares scores across different judge models to detect these failure modes before they silently corrupt your quality gates.