Doodle Dash is a web game that runs a Vision Transformer entirely in your browser, making over 60 predictions per second without a server round-trip. The game gives you one minute to draw as many items as you can, one prompt at a time. When the model predicts the correct label, the canvas clears and a new word appears. Inspired by Google’s Quick, Draw!, the project was built by finetuning apple/mobilevit-small — a lightweight Vision Transformer with just 5.6M parameters and a ~20 MB file size — on a subset of Google’s Quick, Draw! dataset containing over 5 million drawings across 345 categories.
The entire pipeline is surprisingly accessible. The team used a Colab notebook to finetune the MobileViT model, then converted it to ONNX using Hugging Face’s Optimum library. From there, Transformers.js — a JavaScript library that runs Hugging Face models directly in the browser via ONNX Runtime — handles the inference. The finetuned model is hosted on the Hugging Face Hub as Xenova/quickdraw-mobilevit-small.
One of the more interesting architectural choices is running inference inside a Web Worker. Machine learning inference is computationally intensive, and doing it on the main thread would cause the UI to stutter and drop drawing gestures. Offloading it to a separate thread keeps the canvas responsive while the model churns through predictions in near real-time.
This approach signals something bigger for the web gaming space. We’ve seen browser-based ML demos before, but most leaned on tiny models or heavy quantization. Using a full Vision Transformer at 60+ inferences per second — without quantization, per the code snippet — shows how much headroom modern browsers and ONNX Runtime actually have. The project is part of an Open Source AI Game Jam, which suggests the goal isn’t just a one-off demo but a template for other developers to build on. If in-browser models can handle this kind of workload today, the gap between native and web-based AI experiences keeps shrinking faster than most people assume.
