Transformer Based Audio Synthesis

via “streaming transformer inference for long-form audio”

Meta's library for music and audio generation.

Unique: Implements rolling key-value cache for transformer attention, enabling efficient incremental generation of audio chunks without reprocessing previous context. Maintains generation coherence across chunk boundaries through overlapping context windows.

vs others: Enables generation of arbitrarily long audio without memory explosion; practical for streaming applications. More efficient than regenerating full sequences for each chunk.