0xreversed/with.ai
File formatsResearch
Target: GarageBand project format

Decoded the GarageBand format so Claude could compose with local instruments, inside a 24 hour music video

GarageBand project format: Composed on your own instrument sounds, format learned by Claude.
Image: Anshu (video frame on X)

What was done

Claude reverse engineered GarageBand project data to compose and run the music locally, part of a music video with Blender, Qwen-Image and a local singer model.

Claude Opus 5.5 reversed the GarageBand file format so that music could be composed and played with the local GarageBand instruments on a Mac, as one step inside a fully local production. The result is a music video made 100 percent locally in 24 hours: 3D modelling and editing with Blender driven by Python scripts, images with Qwen-Image 2.1, and vocals from an open model called SoulX-Singer which Claude got running on Apple MLX.

How it was done

The composition pipeline writes GarageBand project data that GarageBand instruments play, so the music keeps the local instrument sounds rather than going through a synth. The rest of the video pipeline is scripted the same way: Blender for the 3D animation of the characters, including fretting and strumming that follows physical accuracy per the author, and image generation for the scenes.

Results

A finished music video with its own band and songs, generated entirely on a MacBook, with the GarageBand format decoding as one of the working parts. The author reports the characters fret and strum physically correctly, and the tweet thread carries the video itself.

From the source