6 min read
Handwriting without a neural network
This is the second half of a pair. In the first post I trained a neural network to draw like a hand, and it is honestly still bad at it. Inkside makes handwritten pages that look far better, and there is no neural network in it at all. Same goal, the opposite approach.
A language for handwritten pages
Inkside turns a text document into handwritten study sheets. The document is Markdown with LaTeX for formulas, plus small languages for diagrams: plots, flowcharts, timelines, mind maps, molecules, and a draw block for everything else. A drawing is a list of statements, and things can point at each other by name:
client = computer at (14, 32) size 20 x 18
lb = rect at (52, 32) size 26 x 12 {r: 2, wash: sky} "Load Balancer"
s1 = rect at (110, 10) size 24 x 10 {r: 2, wash: green} "Server 1"
arrow client -> lb
a1 = arrow lb -> s1 bend -4
banana at a1.mid + (-1, -3.2) size 8 x 4 rotate -22The banana sits on the middle of the first arrow, the arrow runs from the edge of one box to the edge of the other, and the text wraps itself to fit its box. Nobody picks pixel positions for the details. The language is meant to be written by a language model, so it reports every mistake with its line and a suggestion, and a broken diagram never takes the page down with it.
Where the hand comes from
Letters come from handwriting fonts that I turned into pen strokes along the middle of each letter, so the result is real ink, not filled outlines. Every letter is varied a little, so two “a”s are never identical, and the baselines sit exactly on the lines of the paper.
Shapes get the same treatment. A circle starts at a random point, wobbles a bit more the bigger it is, and runs a little past where it closes, like a pen that doesn't stop in time. Colour doesn't quite keep to the lines. Hatching has uneven gaps. All of that randomness is seeded, so the same document always gives the same strokes, and editing one label leaves every other stroke where it was.
Over 400 drawings, drawn by agents
The diagrams are where Inkside gets its quality, and it's not from the renderer. It has over 400 components: a heart with its chambers and valves, a plant cell, a distillation set-up, a volcano, an eye, a circuit lamp. Each one is a small program in the same drawing language, with named parts that a label can point at.
I didn't draw them myself. I had 25 Opus 5.5 agents work in parallel, each on its own component. An agent wrote the component, rendered it, looked at the picture, and fixed what was wrong, in a loop of about 30 minutes per component until it looked right. Then the next one. That's a lot of compute, but it's spent once. Using a component afterwards costs nothing:
h = heart at (50, 58)
label h.aorta "Aorta" right
label h.ra "Right atrium" left
label h.mitral "Mitral valve" rightWhat it can't do
Inkside can only draw what someone has drawn before. There is no banana in the library, so the one above is four lines I wrote into the document. Ask for something that isn't in the library and nobody writes it, and you get boxes, circles and arrows. It doesn't look at a sketch and copy it, and it can't invent a drawing it has never seen.
It also doesn't know how people draw. The pages look handwritten, but the order of the strokes is just the order of the document, and the pen moves at an even speed. The little imperfections are rules I wrote, not something learned from real hands. Every new component costs agent hours, and the library only grows as fast as someone pays for it.
Two ways to the same page
Put the two projects side by side and they fail in opposite places.
- Inkside places pre-drawn elements in a human way. Its pages are always readable, exact and instant, and they open in Goodnotes as editable ink. But everything it draws, someone drew first.
- Stylus learns the act of drawing from recordings of real people: the order, the rhythm, where the label goes. It can in principle draw anything it is shown. Today its letters are hard to read and its shapes are rough, because finishing it would take weeks of training on expensive hardware.
For a study sheet tomorrow, Inkside wins easily, and that's what it's for. The interesting future is probably the two together: a library of good components and a clean layout from Inkside, drawn in the order and rhythm that a model like Stylus has learned from people.