DroidDoodle
On-device agentic AI driving a drawing canvas on Android.
- Services
- Product DesignEngineeringApp Development
- Industries
- AIMobile
- Tools
- KotlinC++on-device LLM
- Highlight
- A phone runs the model that drives the canvas.
- Live
- Archived
How much agency fits in a phone
The interesting question about small local models is not what they can write. It is how much agency they can be given — how far a 1B model can be trusted to act, if the world it acts in is small enough and the tools it has are good enough.
DroidDoodle is that experiment with a drawing canvas as the world: an offline Android app where an on-device model manipulates a structured board through tool calls, rather than generating an image.
The world model is the actual work
A model asked to draw produces pixels nobody can review. A model asked to call tools against a structured board produces a plan you can read, refuse, and undo — which is the same argument Tessera makes about pixel art, arrived at from the other direction.
So the board is immutable, grid-snapped, with undo by reference: every operation produces a new board rather than mutating one, so undo is holding on to the previous reference rather than reversing anything. That is a boring data-structure decision and it is why a wrong plan costs nothing.
A learning laboratory for agentic AI — how much agency can be created from small local models with well-defined worlds and excellent tools.
- Agent core
- Pure KotlinTool calls, not pixelsJVM unit tests
- World
- Immutable boardGrid-snappedUndo by reference
- Planned
- llama.cpp via JNIJetpack Compose UIGBNF grammar
- Status
- Packages P7–P10 unstartedNo device run yetPlans hand-written
What it cost
Building the agent core against hand-written plans was the right order and it has a sharp edge. The board, the tools, the validation and the undo model are all testable on a JVM without a model anywhere near them, which is why they are the parts that are finished.
The edge is that everything downstream of "a real model produces a plan" is therefore untested by construction — and that is not a small remainder.
Where it does not work yet
This one is early, and its own README is the bluntest thing in this repository, so it gets quoted rather than softened: "nothing has run on a device, no real model has produced a single plan."
The GBNF grammar that is supposed to constrain the model's output has never been run against llama.cpp. Packages P7 through P10 — the Compose UI, the JNI bridge, the trace UI and on-device measurement — are unstarted. Every plan the agent core has been tested with was written by hand.
It is on this site because the design work is real and the honest status is more useful than a demo would be. The thesis at the top of this page — a phone runs the model that drives the canvas — is the intent. It is not yet a fact, and the day it becomes one this paragraph gets rewritten.