ByteDance's EdgeBench reveals GPT-5.5's 22% edge, but time budgets flip the leaderboard

ByteDance has quietly dropped a benchmark that does something most others don’t: it measures agent performance not just by task, but by how much time you give the model to think. EdgeBench, released through the company’s Seed division on Hugging Face, packs 51 tasks across wildly different environments—some tasks need internet access, others run in isolated Docker containers, and a handful are straight-up game modes where agents navigate interactive worlds.

The setup is more intricate than your average eval. Each task comes with its own judge configuration and what the team calls “rescale functions” (using something labeled SForge internally) that transform raw agent outputs into normalized scores. We pulled the task specs directly from the repo and found a mix of runtime images, with no single environment dominating. That diversity matters—it means a model can’t just memorize one type of interaction pattern.

Here’s where it gets interesting. We scraped the leaderboard data from the README (Claude Opus 4.8, GPT-5.5, GPT-5.4, GLM-5.1, and DS-V4-Pro are the contenders) and mapped performance across six time budgets ranging from 2 to 12 seconds. The ranking isn’t static. Models swap positions as the clock ticks up, and we fit log-sigmoid scaling curves to quantify exactly how much each model gains from extra compute. GPT-5.5 grabs the top slot in raw scores, but the curves suggest some competitors get more marginal benefit per additional second.

The real story isn’t who’s winning—it’s that interaction-time budgets are an under-explored axis in agent evaluation. Most benchmarks freeze everything except the model. EdgeBench lets the clock run, and the results show why that matters: faster isn’t always better, and “best” depends heavily on what latency you can tolerate in production.