A MIDI Mixer as a Cockpit for Parallel Coding Agents

Joint work with Javier Ron.

TLDR: We turn a $60 Korg nanoKONTROL2 — a USB mixer built for audio production — into a physical control surface for eight parallel coding agents, one per fader/column.

Here’s nanokontrol_agent.py in action: spawning agents, moving voice focus between columns, and sweeping the effort fader through its six reasoning levels live.

The controller itself, eight channel strips ready to be repurposed:

The Vision

Every coding agent interface today is the same: a chat window, a text box, a keyboard. That’s fine for one agent. It falls apart once you’re running several in parallel, because a keyboard has no notion of “column 3” or “the agent I’m currently talking to” — you tab, you click, you scroll, you lose track of which pane is listening.

A mixing console solves exactly this problem for audio engineers: eight physical channels, each with its own fader, solo button, and mute button, laid out spatially so your hand finds the right one without looking. nanokontrol_agent.py borrows that solved problem wholesale. Each of the nanoKONTROL2’s eight channel strips becomes one coding agent. Muscle memory replaces window management.

The Hardware Mapping

The repurposing is literal:

• S (spawn) — spawn or kill the agent in that column. LED on = agent alive.
• M (move focus) — move voice focus to that column. LED on = currently focused.
• R (reset) — clear that agent’s conversation history.
• Fader — reasoning effort budget for that column, quantized into six levels (off, minimal, low, medium, high, xhigh), 512 to 32000 tokens.
• PLAY — hold to record voice, release to transcribe and send to the focused agent.
• STOP — abort the focused agent’s current turn, or kill an in-progress recording.

There’s no on-screen equivalent for any of this. You never touch the keyboard except to quit.

Under the Hood

Architecturally it’s three threads talking through one queue:

• a MIDI read thread parsing raw Control Change bytes off /dev/snd/midiC1D0,
• a per-turn agent thread running a manual tool-call loop, with a single run_shell tool,
• the urwid main loop, which drains the queue on a 50ms alarm and is the only thing allowed to touch widgets.

Voice capture is push-to-talk: arecord starts on PLAY down, SIGINT-stops on release, and speech_recognition hands the WAV to Google’s STT. No wake word, no VAD — the fader-and-button vocabulary already tells the system exactly when you’re talking and to whom.

Use Case

Sit at the console, hit S on four channels to spawn four agents against four different problems in the same repo. Set channel 1’s fader low (quick, cheap answers) and channel 3’s fader to xhigh (a gnarly bug that needs real reasoning). Hold PLAY, say “grep for where the retry logic lives,” release — the transcript lands in whichever channel’s M LED is lit. Watch the shell commands and output scroll in that column’s pane. Hit STOP if it’s going down the wrong path. Hit M on channel 3 to swing your voice over there without breaking stride.

It’s the closest thing to conducting an orchestra of coding agents.