Open-Source Whiteboard AI Tutor Benchmarks Six LLMs
molly554/whiteboard-ai-tutor splits whiteboard tutoring into a JSON drawing protocol any model can drive, then measures how well six of them teach.
A GitHub project called molly554/whiteboard-ai-tutor was created on 2026-11-11 and held 303 stars and 18 forks as of 2026-11-19. It is two things in one MIT-licensed repository, mostly written in HTML: a working open-source whiteboard AI tutor, and a whiteboard AI tutor benchmark that pushes six models through one frozen prompt. Every generated board is stored in the repo, so anyone can replay a lesson stroke by stroke in a browser.
The author's argument is that a tutor which writes, draws and erases while it explains looks like a model capability but is really a small protocol. Once the app owns the drawing, any LLM can play the tutor.
What the repository actually contains
Two pieces. A reference implementation of the loop: the protocol, a whiteboard player, and a runner for Claude, Gemini and DeepSeek. And a benchmark: the same prompt, eight scenarios across science, maths and economics, six models, two thinking settings and three repeats, for 288 generation runs. Sixty-four boards went to blind review.
The source is explicit that this is not a hosted tutoring app. The player replays recorded lessons. Generating new ones means running the benchmark runner with your own API key. The quickstart needs no keys at all: clone, run python serve.py, open localhost:8811. The full benchmark matrix costs about $4 according to the README.
How the protocol works
The model never touches a canvas. It streams JSON steps, each with a line of narration and a list of operations. There are six operations (write, line, rect, ellipse, path, erase) on a 1600 by 900 board. Because the first operation is usable the moment it arrives, a student sees the first stroke after two to four seconds on the fastest models, long before the lesson finishes generating.
When a student interrupts, the app stops the stream and sends the question back together with the board state: each object's id, type, bounding box and text. The model replies with new steps only, and may erase or annotate existing objects by id. The tested prompt is frozen and written in Chinese, with an English translation in the repository.
What the benchmark found
Blind-reviewed boards passed at the low thinking setting, with median time to first stroke and cost per lesson:
- Gemini 4 Argon: 11 of 11, 8.0 s, $0.0277. Slowest to start; the source notes its price doubles on 2026-12-01.
- Claude Haiku 5.5: 9 of 11, 12.9 s, $0.0019.
- Gemini 3.8 Flash: 8 of 11, 3.4 s, $0.0080.
- Claude Sonnet 5.5: 7 of 11, 2.8 s, $0.0158.
- DeepSeek V4 Pro: 5 of 10, 3.8 s, $0.0064.
- DeepSeek V4.1 Flash: 4 of 10, 2.5 s, $0.0017.
The same question, why the Pythagorean theorem works, was drawn by all six. Gemini 4 Argon produced two arrangements with exact geometry; DeepSeek V4.1 Flash put no actual argument on the board.
Five findings stand out. Of the 20 failed boards, 12 failed because the figure contradicted the text, while written statements were almost never wrong. There is no single winner: accuracy, speed and price split across different models. A cheap, fast model with thinking off can fail the protocol outright, as DeepSeek V4.1 Flash did by omitting the op field in 7 of 24 calls, which parses as JSON and draws nothing. The project's own linter was a poor judge, flagging 41 of 64 boards for hard errors when blind review passed 26 of those. And more thinking mostly bought waiting: with thinking on, first stroke took 30 to 120 seconds on Gemini 3.8 Flash and both DeepSeek models, and 23 calls hit the 8,000-token output cap.
Interruptions were handled but messily. All 24 interruption boards answered the student, and 11 wrote over the existing drawing or wiped the board to start again.
Limits and open questions
The README lists its own constraints. Eleven boards per model means 9/11, 8/11 and 7/11 are not distinguishable from each other. Only final boards were reviewed, not intermediate states or pacing. One frozen prompt, in Chinese, so different wording could reorder the results. Eight scenarios, with nothing about real student questions at scale. Latency came from one desktop, and Claude and Gemini went through OpenRouter, which adds a hop, so comparing models to each other is safer than comparing them to your own numbers. Medium-thinking parse rates are distorted by the token cap. Handwriting depends on the Ink Free font, which ships with Windows, so replays elsewhere can look different from the reviewed images. Costs are list prices on 2026-11-11.
The author suggests a next step worth watching: higher-level operations such as mark_intersection(a, b) or label(id, text), so that code rather than the model computes positions. That targets the most common failure directly, but the source does not report results from testing it.
What this means for building apps with AI
The transferable idea is the split. A model that emits structured operations, plus a player that renders them, keeps generation and presentation separate and lets the app resume from stored state instead of a fresh prompt. That pattern fits any app where a model should change something on screen step by step, not just tutoring. If you are wiring a model into something you are building, Draftship's guide to adding AI to your app covers the practical side of that setup.
Primary sources
This article was drafted with AI assistance from the primary sources listed below and checked automatically against them. An editor hasn't reviewed it yet.