BigCode, the open scientific collaboration run jointly by Hugging Face and ServiceNow, has released StarCoder — a 15-billion-parameter code generation model that outperforms OpenAI’s code-cushman-001, the original Codex model that powered early GitHub Copilot. The model was trained on one trillion tokens drawn from The Stack 1.2, a dataset of permissively licensed GitHub code spanning more than 80 programming languages, plus commits, issues, and Jupyter notebooks.
On HumanEval, the standard Python benchmark that tests whether models can complete functions from signatures and docstrings, StarCoder hit 34% initially. But the team discovered a quirk: the model would sometimes spit out placeholder text like ”# Solution here” — a habit picked up from training on coding exercises. Adding a specific prompt referencing a solution file pushed the score past 40%, which BigCode claims is a new state-of-the-art for open models. For context, that puts a 15B open model ahead of far larger systems like PaLM, LaMDA, and LLaMA on this test, despite StarCoder being a fraction of their size.
The model’s 8,000-token context window is also the longest of any open LLM at release, which opens up practical applications beyond simple autocomplete. BigCode demonstrated that with a carefully constructed prompt inspired by Anthropic’s HHH framework, StarCoder can function as a technical assistant — answering programming questions, explaining code in natural language, and making modifications based on instructions. The multilingual evaluation on MultiPL-E showed StarCoder matching or beating code-cushman-001 across many languages, and it clearly won on DS-1000, a data science benchmark.
What sets this release apart isn’t just the benchmark numbers. BigCode took real steps on the responsible AI front: they built an improved PII redaction pipeline with Toloka to strip names, passwords, and emails from training data, created an attribution tracing tool so users can find generated code in the original dataset, and released everything under an updated OpenRAIL license that simplifies commercial integration. The timing matters too — this lands as enterprises are actively looking for alternatives to closed APIs, and having a genuinely competitive open model with a clean licensing story removes one of the biggest blockers to adoption. Whether StarCoder holds its edge as Meta, Google, and others push their own open code models remains to be seen, but for now the bar just moved.
