OpenAI's Codex Can Now Train Its Own Open-Source Models, But Should It?

OpenAI is handing its coding agent, Codex, the keys to the Hugging Face kingdom, and the implications are a bit dizzying. This isn’t just about writing nicer code snippets. The new integration, built on the ‘HF-skills’ repository, lets Codex autonomously run an entire machine learning experiment from a single prompt. You can tell it to fine-tune a Qwen3-0.6B model on a dataset like open-r1/codeforces-cots, and it will validate the data, pick the right GPU (a t4-small in that case), submit the job to Hugging Face, and even monitor the training metrics via Trackio. It’s a production-ready pipeline that supports supervised fine-tuning, DPO, and RL with verifiable rewards for models up to 7B parameters.

This goes significantly further than a previous Claude Code tutorial on the same theme. The real ambition here is to move from a helpful assistant to a semi-autonomous research partner. The goal, as the team puts it, is for Codex to “make more decisions on its own based on the training report and evaluation results.” The setup is straightforward if you have a paid Hugging Face plan: clone the skills repo, authenticate, and Codex automatically detects the AGENTS.md file to load its new capabilities. You can then order it to run an experiment, evaluate models with the openai_humaneval benchmark, and maintain a continuously updated report, effectively delegating the grunt work of model refinement.

The sheer utility of this is undeniable for individual developers or small teams drowning in experimentation overhead. Codex handles the context-switching nightmare of managing cloud jobs. Yet, I can’t shake the feeling that we’re watching a snake eat its own tail with a bit too much enthusiasm. OpenAI’s premier coding agent is now being positioned to fine-tune and publish open-source models from competitors like Qwen. It’s a brilliant strategic move that puts OpenAI’s tooling at the center of the open-source ecosystem, but it also completely blurs the line between the platform and the models it’s supposed to be helping you build. The next logical step—letting the agent decide which model architecture to use based on a report it wrote—isn’t science fiction; it’s likely just a few prompts away.