I’ve pondered on a minimal sequencer / music player… It’s still somewhat of an inital draft, but I tried it out in What time is it / Mad Tea Synth

Ideals:

• Ability to reuse player for both music & visuals
• Support optional stateless (without seek) playback (for gpu / parallelism)
• Channel-based data, but without having to optimize the channel structure manually like in a tracker

The experiment I came up with is inferior to any normal tracker format from a musical-features perspective, but I think it’s interesting as a starting point.

Conversion process

(assume input data is quantized and midi-like)

1. Convert to pure events, different combinations of vel+note -> unique events
2. Convert to 2D matrix (large matrix of booleans : events x 16ths)
3. Compact into channels (matrix of integer id : channels x 16ths)
4. Select pattern length, and slice into patterns
5. Decimate each pattern, if possible (AABBAABB -> ABAB)
6. Detect repetitions (ABAB -> AB)
7. append to column_data (or just reference if already inside)

We then repeat steps 3-7, using different pattern length in step 4 (and different methods in step 3 and 7) to find smallest size.

Clarification: To the user and in the Mad Tea Synth UI, an event is code triggered by midi (single events can be assigned to multiple midi notes and trigger different velocities). An event in the context of sequencer playback is an integer, where the specifics/details are assumed to be handled/stored elsewhere.

Playback

The data format in memory is similar to a tracker, but indexed in a different way; The song/sequence is list of patterns (not a list of pattern references). However each pattern column uses references instead. A song is represented as an array uint16[pattern_count * channel_count].

Pattern column layout (uint16)

// song data
const uint16 sequence[PATTERN_COUNT * CHANNEL_COUNT] = {...};
const uint8  column_data[SEQUENCE_DATA_COUNT] = {...};

// state
int   channel_timestamp[CHANNEL_COUNT];
uint8 channel_event    [CHANNEL_COUNT];

// sequence tick
if((sample_id % SAMPLES_PER_SONGTICK) == 0) {
    song_tick   = sample_id / SAMPLES_PER_SONGTICK;
    pattern_pos = song_tick &  PATTERN_LEN_MASK;
    pattern_id  = song_tick >> PATTERN_LEN_SHIFT;
    for (int c = 0; c < CHANNEL_COUNT; c++) {
        column       = sequence[pattern_id * CHANNEL_COUNT + c];
        res_shift    = (column & RES_MASK) >> RES_SHIFT;
        repeat_shift = (column & REPEAT_MASK) >> REPEAT_SHIFT;
        base         = (column & COLUMN_ID_MASK) << COLUMN_SHIFT;
        looped_pos   = (pattern_pos >> res_shift) &
                                 ((1 << repeat_shift) - 1);
        event_id     = column_data[base + looped_pos];   
        if (event_id != channel_event[c]) {
            channel_timestamp[c] = sample_id;
            channel_event    [c] = event_id;
        }
    }
}

// render / draw
for (int c = 0; c < CHANNEL_COUNT; c++) {
    int ev = state->channel_event[c];
    if (ev) {
        int t = sample_id - state->channel_timestamp[c];
        switch (ev) { 
            case 1: ...
            case 2: ...
            case 3: ...
        }
    }
}

The above is not stateless, but it needs very little change (but we’ll pay in data size). What we do is slice all longer events by the grid, so an 8th note in a 16th note grid would be represented as 2 separate events, and note-slice-events will carry the offsets in 16ths (except the inital slice). So already at step 2 we’d have a larger matrix.

// combined stateless tick/render

song_tick   = sample_id / SAMPLES_PER_SONGTICK;
pattern_pos = song_tick &  PATTERN_LEN_MASK;
pattern_id  = song_tick >> PATTERN_LEN_SHIFT;
for (int c = 0; c < CHANNEL_COUNT; c++) {
    column       = sequence[pattern_id * CHANNEL_COUNT + c];
    res_shift    = (column & RES_MASK) >> RES_SHIFT;
    repeat_shift = (column & REPEAT_MASK) >> REPEAT_SHIFT;
    base         = (column & COLUMN_ID_MASK) << COLUMN_SHIFT;
    looped_pos   = (pattern_pos >> res_shift) &
                                ((1 << repeat_shift) - 1);
    event_id     = column_data[base + looped_pos];   

    // above code is same as stateful

    switch (event_id) { 
        case 1:  // first slice of 8th note
            t = sample_id + (0 - song_tick) * SAMPLES_PER_SONGTICK;
            ...       
        case 2:  // second slice of same note
            t = sample_id + (1 - song_tick) * SAMPLES_PER_SONGTICK;
            ...      
    }
}

Note that a wierd quirk of the stateful impl is that there is no way to separate two 16th notes from one 8th note. So if two identical notes are played in succession, we need to allocate an additional event so a repeated baseline on the same note (and the same velocity) will resolve to 2 alternating events (at process step 2). Initially, I tried to include note-length or retrig into the playback implementation, but none of my attempts felt clean enough.

There is also an annoying ‘cliff’ if the total event count go above 255, cos then we can no longer reference them using bytes (exporter just switches to 16bit). What time is it - which is a simple and quite repetitive arrangement used 192 events…

(I use 16ths here for simplicity, we could of course divide into smaller pieces, but 3 or 12 per beat probably won’t go well as loop sizes are always power of 2 in this impl)

Actual Output

Example pattern:

snare       ....OO......OO......OO......O.O.|
kick        OO......OO......OO......OO......|
bg          OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO|
(grid)      |   |   |   |   |   |   |   |   |

(Kick/snare have both audio and visual code, bg has only visuals)

The exporter cuts it onto 4 patterns and 2 channels:

kick/snare  KK..SS..  KK..SS..  KK..SS..  KK..S.S.
bg          BBBBBBBB  BBBBBBBB  BBBBBBBB  BBBBBBBB

Exported Data:

static const unsigned char song_column_data[SONG_COLUMN_DATA_COUNT] = {
    1,1,0,0,3,0,3,0, 1,0,3,0, 2, // KK..S.S.  K.S.  B
};

static const unsigned short song_sequence[SONG_SEQUENCE_COUNT] = {
    0x4808,0x000C,0x4808,0x000C,0x4808,0x000C,0x6000,0x000C,
};

Render funcs:

Simplest code-gen variants (there are multiple ways to export audio batch depending on code size vs speed, do batch looping per event etc)

int song_render_audio_batch(int sample_id, song_SongState* state, short int* out_samples)
{
    song_sequence_tick(sample_id, state);
    for (int i = 0; i < BATCHSIZE; i++) {
        int sample = 0; // fx16, summed across channels
        for (int channel_id = 0; channel_id < SONG_CHANNEL_COUNT; channel_id++) {
            int samp = sample_id + i - state->channel_timestamp[channel_id];
            switch (state->channel_event[channel_id]) {
                case 1: sample += mul16(kick(((int)((long long)samp * (long long)SONG_FX22_PER_SAMPLE(4194304LL)))), kick_36_v96_VELMUL); break;
                case 3: sample += mul16(snare(((int)((long long)samp * (long long)SONG_FX22_PER_SAMPLE(4194304LL)))), snare_37_v96_VELMUL); break;
            }
        }
        out_samples[i] = song_fx16_to_pcm16(sample);
    }
    return sample_id + BATCHSIZE;
}

int song_visual_render(int sample_id, song_SongState* state)
{
    song_sequence_tick(sample_id, state);
    for (int channel_id = 0; channel_id < SONG_CHANNEL_COUNT; channel_id++) {
        int ev = state->channel_event[channel_id];
        if (ev) {
            vscreen_images_reset();
            switch (ev) {
                case 1: kick_visual(); break;
                case 2: bg_visual(((int)(((long long)sample_id * SONG_BEAT_FX42_PER_SAMPLE) >> 20))); break;
                case 3: snare_visual(); break;
            }
        }
    }
    return sample_id;
}

Example C code

• sequencer-export.h
• sequencer-export.c