Making Mad Tea Synth

For 20 years, I’ve used a js-based interactive programming environment to prototype animation, sound dsp, utility functions and approximations etc. But often, js is not actually the target language which means I have to port the code. LLMs have made porting step much easier, but it still feels like a terrible workflow…

So what would be the ideal feature set?

For portability/simplicity, an implementation in plain C without standard C libraries would be nice. But C is tedious for anything large, so I felt this might be a good test case to try out these new LLMs. To see if the LLMs could do the tedious parts, and I could do mostly design, architecture and specification?

There was also other random things I wanted to do, like a demo for the new Pebble Watch and a procedural version of a font. I also had some ideas about a minimalist timeline player, and an abandoned/unfinished vst synth created during the RE-FORM project.

For at least half of the project, there was no clear vision how these parts would fit together.

LLMs

I feel conflicted about LLMs. It might not be a coincidence that what I am most excited of building in this moment, is a joyful programming environment without AI integration.

Just like with industrial meat consumption and aviation, the ethical choice in regards to LLMs is probably to limit your usage. But as someone who is interested in software and the inner working of the human mind, LLMs are very fascinating. For decades we’ve tried to make algorithms that understand basic English, and mostly failing even in limited domains.

Now there is a method that does all major domains in all major languages. And it’s an amalgam of compute and human culture (well, at least the parts that made it to the internet or print). A technology based on the commons, that should belong to us all.

LLM assisted software development

Obviously I don’t feel the same kind of authorship as I would when implementing myself. But I still feel some kind of authorship, maybe because developing still takes months of labour. It has also been more familiar and similar to traditional software development than I expected. 95% of prompts feels the same as would go into your typical bug/feature-tracker.

It’s the same endless iteration, back and forth, as you traverse the possibility space, but taking larger steps. I don’t think I will go back to manually setting up environments and build scripts, refactoring, spending days debugging trivial crashes/leaks, updating tests etc.

After avoiding anything similar to node/npm for 15 years, the internet dependence of decent llms feels like a sad defeat. But the worst change is that this kind of development rarely brings me into a deeper flow state in the way implementation did. At the same time, it’s great to be able to pick the most fun parts to implement myself. I think working this way will make me worse at coding and debugging, but also better at architecture and product.

Parser

In January I started porting the parser from spine. It’s probably not a good parser, but it’s the only one I know well. I also knew I would probably want to do a reactive/spine-like backend eventually.

Parsing flow:

tokenize  ->  interning -> join tokens  ->  token iterator -> parse

3 categories:            '=','='  -> '=='     parses  
  numbers                '.','.'  -> '..'     comments &
  alphanumeric           '..','.' -> '...'    quoted strings
  single chars                 etc            on demand

The AST node tree uses a single struct, and also holds whitespace data etc for lossless conversion back to text.

I chose to add interning after tokenization to match my mental model of the js implementation. Also added type categorization for literals and similar things to align more with C.

Tea++ (Compiler / VM)

The following are excerpts from the prompt to create the initial vm architecture plan:

As you can tell, I had no clue how to best implement this. As I reviewed/iterated the plan, I postponed js/lua backends and went for a C-aligned internal representation and no virtual stack. Starting with three types: int, float and double.

The LLM quickly implemented a starting point complete with unit tests and bugs.

C Macro solution (Meld)

My first attempt to integrate this into a workflow was a precompiler-makro-based thing, trying to make C code itself reactive. In the source code you’d do something like:

live_func(int, pixelcol, int x, int y, int t) 
/*
    ((x) ^ y) & 4091 & (t | 15)
*/
live_func_end(x, y, t)

Then when the pixelcol function runs (in the dev build), it would access the C source code file on disk, extract the text/parse/compile-for-vm/cache and run it in the vm. It would also start polling the date of the C source code for changes (ugly file watcher). And then when the file is changed, it can emit the release/C version of the code into the same file:

// -------------------------- MELD GENERATED

static int pixelcol_gen(int x, int y, int t)
{
    return (((x ^ y) & 4091) & (t | 15));
}

// -------------------------- END MELD GENERATED

This workflow felt too slow to me, and I was also missing sliders or similar to quickly explore value ranges. It’s also strange with code that is highlighted as comments in the editor…

I briefly looked into making vscode plugins, but complexity and existing keyboard shortcuts made me feel I would not be able to create the kind of experience I had in mind. And even if I got it to work, it would easily break when using other plugins or in a future version…

A more appealing option was a simple portable UI…

Textmode UI

But, why textmode in 2026??

Textmode UI sketches from 2016, using the neat ascii-to-svg conversion in markdeep

Font Editor

Fonts need very few points, and a bitfield per point determines corner roundness, relative thickness, linecap types. A feature that reduced complexity was the axis-snapped linecaps (exist in both square and butt varieties)

Axis-snapped linecap on the leg of R

I spent a lot of time iterating on the glyph stroke and rendering code, however the font editor app itself was vibecoded with almost no code review.

Using the editor I was able to reproduced something close to Idealist Hacker Mono. But I felt that as a thinner font, it did not work that well. So I made a new variant, which is rounder and more conventional, Assembly Line.

Font comparison: Assembly Line vs Idealist Hacker

System supports only ascii (+custom glyphs above 127). It’s also missing good features for optical thinning of bowls etc (this is needed to improve bold versions of glyphs like R,W,m,w,a,b,c,d,e,g,h,p,w,y and r).

UI System

I’ve made UI systems in the past but this was the first time I went for an immidiate-mode api. It always draws the character buffer (textmode screen) as it’s processing a frame. If buffer is same as last frame, no need to draw pixels at all.

To make it even simpler, it’s writing/drawing to the character buffer in the same order as processing (front-to-back, using a simple drawn-flag per character).

This design made the LLMs confused and caused many errors (getting the order wrong) until it was documented in multiple places. Similar issues occurred with the unconventional parser design. If you care about a specific design, it means more problems and having to guard against drift towards the common.

Try UI Demo

Interactive Coding Editor

The basic textarea behaviour demanded a lot of iteration, and then the undo/redo was reworked two times so it would not consume huge amounts of memory.

The code editor is designed to have your typical editing behaviour, including selecting text and cut n paste. The experimental stuff uses right mouse button:

The machanical-wheel-scrubber was also an idea I’ve been wanting to do for many years

MicroW8 Live Coding

Since before the RE-FORM project, I’ve had a crude microw8 backend for win32 (to be able to run C code made for microw8 in a windows app).

Combining everything mentioned so far gives the MicroW8 Live Coding app:

Screenshot from early june

This screenshot was captured some days after my Pebble Time 2 arrived (about 4 months late). Since the pebble SDK did not support hardware floats, I had decided to add fixed point support to the compiler (see the results pane).

For good precision of fixed point mul/div, 64-bit int support was also added to the compiler. The vm + coding editor was also ported to WASM / madtealab to graph and debug the fixed point maths.

Live coding on the watch?

To the detriment of both code quality and my own sanity, I tried to get the vm to run on the watch…

This also required a rework of Meld, splitting it into 2 parts, a server with ui that would compile, test-run and serialize, and the embedded part that would run the stripped down vm on the watch. Communication and code would be piped through the pebble debugging api. However, in the end I was hitting all kinds of limits, so this was not a feasible workflow for a growing project.

Except for some early debugging, sound testing, and speaker filter tweaking, Meld was not really used.

Mad Tea Synth VST

I had started on a VST (virtual instrument plugin) when working on the synth for RE-FORM, but given it up. I used that as a starting point. First step was replacing the expression evaluator in that prototype with with Tea++, and integrating the interactive coding editor. Then make a small api to access the pixel/line/text rendering from Tea++.

Events are triggered by midi, and every event can trigger sound, visuals or both. I settled on a split view layout with sound on the left and visuals on the right. I don’t have any screenshot from early on, but this is how it ended up later:

What time is it?

I was aiming to make a watch demo for Evoke, and I really had to get started (even though there was no synth api etc). So from now on Mad Tea Synth development was lower prio and only driven by the needs for the demo.

Overall I was quite happy with the experience of using the vst, except a few days where it felt more like wading through compiler bugs to do the simplest things.

The demo has its own write-up

Levels of debugging

Broken font rendering on pebble

Debugging manually on the the hardware is very time consuming, so it’s important to have multiple levels:

A memorable experience during development was using the open source harness pi.dev together with deepseek - giving it access to the toolchain and watch hardware. Then as I was cooking and doing household tasks, I could see new builds being pushed and debugged wirelessly on the watch.

Playback

Through Tea++, we can output C code for instruments and visuals. But how is the sequence played back in the demo on the watch?

Short answer: - Code generation again. In memory it’s a bit like a tracker format but instead of indexing patterns, we index pattern columns.

Long answer: - Read more about a generalized minimal sequencer.

After demo completion

Driving tool development with an actual project (and under time pressure) is great. It makes sure any added features are useful.

There was some rough edges to clean up afterwards, when doing the write-ups I found additional bugs. I had a list of improvements in mind as well:

Future

At some point I want to add shader support, first software based, and later GPU based with shadertoy export.

The reactive implementation of Tea++ would also be interesting to try to integrate.

Download

I don’t recommend using this yet. But for anyone who want to try it out or know the details of the implementation:

Mad Tea Synth VST (source code of tool + demo included)