Air DJ

The idea

Physical DJ controllers feel great, but the price keeps the hobby out of reach for most people. Even an entry level board runs around $400, while almost everyone already owns a laptop with a webcam. Air DJ closes that gap by turning the camera into the controller. A gesture model trained on my own hands classifies motions in real time and routes them out as MIDI, so any DJ software can read them like a regular controller.

How it works

A MediaPipe hand tracker feeds 21 landmarks per hand into a gesture classifier I trained on data I collected myself. The classifier output maps to virtual MIDI messages routed into VirtualDJ, which handles decks, beatgrids, and the actual audio.

The debug overlay with the hand skeleton exists to show an audience how the system sees them. During an actual set, it is closed. The interface is the gesture, the music, and the room.

Takeaways

1. Creativity is the real leverage with Claude Code.

   The agent is a strong executor but a mediocre inventor. It happily builds whatever you describe, so the value is in deciding what to build. Claude did the coding. The framing, the building, and the execution were mine.

2. Bridge existing tools, do not replace them.

   Air DJ does not rebuild a DJ engine. It outputs virtual MIDI so VirtualDJ handles decks, beatgrids, and mixing. One small app plugging into a mature one beats reinventing the stack.

3. Gesture design was the hard part.

   The ML was not the bottleneck. The real work was inventing motions that felt obvious on the first try. The crossfader became crossed fingers sliding side to side because the word and the motion match. Every gesture had to carry that kind of immediate meaning or it got cut.

Status

Working prototype. Runs as a standalone macOS app. Next up is hardening the gesture vocabulary and tightening the mappings into VirtualDJ.