Existing text-to-speech (TTS) systems achieve high quality, yet streaming and long-form generation remain challenging. We propose simple adaptations to a standard encoder–decoder Transformer TTS architecture that enable both capabilities without limits on total duration. We first observe that attention patterns in these models are well organized, following a structure that motivates a specific windowing strategy: a prefixed sliding window for autoregressive decoding and a sliding cross-attention for text conditioning. Together, we show that these adaptations address length generalization and error accumulation while being fully streamable in text and audio. Trained exclusively on segments shorter than 30 seconds, our system produces seamless, consistent speech at practically unbounded lengths, outperforming baselines that rely on full context training. We validate the approach through continuous synthesis over hours long synthesis and demonstrate competitive results against state-of-the-art open-source baselines on both short- and long-form benchmarks.
Cross-attention weights are visualised in real time as the model decodes. Sink tokens attract a large share of attention mass throughout generation; while they are not strictly necessary for synthesis quality, they prove valuable for analysis — giving rise to a distinctive and interpretable signature that makes the decoding dynamics easy to follow.
The first 8 seconds are the speaker prefix used to clone the target voice.
Samples are capped at 20 minutes here for efficiency. The next section presents 10-second excerpts taken at various positions from hour-long runs for cross-model comparison.
10-second excerpts extracted at various positions from runs up to one hour long. All models receive the same text and speaker prompt. Baselines suffer from speaker drift, error accumulation, or a hard architectural limit on context length.
Zero-shot TTS evaluation on the SEED-TTS benchmark. Each row corresponds to one test utterance; columns show outputs from each model. The target transcript is shown for convenience.