s2pro-native
You type a sentence and a voice reads it back — and it starts talking before you've lifted your finger off the key: about a seventh of a second to the first sound, and from there it produces speech faster than you can listen to it. Any voice, many languages, and you can clone a new one from a few seconds of a recording.
The unusual part is what's not in it. Almost every AI speech model is really a small model wrapped in a giant tower of Python that breaks every few months. I threw the tower away and rewrote the model itself — every mathematical step — in C. What ships is a single program, the model's weights, and nothing else.
Why bother
Every open speech model ships with a Python serving stack, and that stack is most of what you actually deploy. It drags in a framework, a runtime, a package manager and a set of versions that will break in eight months. The model itself is a few gigabytes of weights and about a dozen operations. So I wrote the dozen operations — the prompt building, the transformer stack, the vocoder, the scheduler, the HTTP server, all of it in C and CUDA against Fish Audio S2-Pro. Python, PyTorch, SGLang and vLLM are not part of the runtime.
What ships is a binary, the weights, and a CUDA driver. The interesting consequence isn't the size — it's that there is no layer left to blame. When a frame takes 22 milliseconds, the reason is in code I wrote.
What happens to a sentence
Text goes through a hand-written byte-level BPE tokenizer, which is bit-exact against the reference implementation on a 4,000-case fuzz — the boring part that quietly ruins everything downstream if it's wrong.
Then two transformers in sequence, per frame. A 36-layer backbone picks one semantic token. A 4-layer decoder expands that token into ten residual codes, nine greedy steps deep. Those codes go into a DAC vocoder that produces 2,048 audio samples — 46.4 milliseconds of sound at 44.1 kHz. Repeat until the sentence is finished.
Audio streams out while it's still being generated, with overlapping vocoder windows crossfaded so the seams are inaudible. A client can start playing the first word before the last one exists.
Where it stands
Measured on one DGX Spark (GB10, sm_121), streamed over HTTP:
| Serving path | Wall RTF | First audio |
|---|---|---|
| Zero-shot | 0.51 | 0.14 s |
| Named voice, warm cache | 0.52 | 0.20 s |
| Long-form (~143 s), chunked | 0.50 | 0.20 s |
RTF is compute divided by audio; below 1.0 means synthesis outruns playback. Every serving path is below it now. The same stack started at 2.05 — the pre-release version of this page said realtime was unreachable at 16-bit weights by construction, and it was. The way out was making the weights smaller without making them wrong: BF16 to INT8 to a packed group-wise INT4 backbone, then quantization-aware self-distillation of the decoder and LM head so the entire weight stream is INT4. Each frame now reads 4.37 GB of weights instead of 15.5, at 4.5 bits per weight, and a decode step takes 21.7 ms against the 46.4 ms frame budget. The engine holds up to five concurrent streams before any of them falls behind realtime.
The gate that says no
Speech quality degrades gracefully and dishonestly. A path can be measurably wrong and still sound fine on the one sentence you tested. So numerical fidelity is checked against the PyTorch reference layer by layer, on fixed inputs, with a pass mark set before the run.
FP8 was twice as fast and failed that gate — error compounds across 36 layers until cosine similarity collapses to 0.33 — so it never shipped. INT8 passed everywhere and became the workhorse. The all-INT4 stack that serves today went through the same gate plus an end-of-speech probe over 24 utterance pairs and critical listening before release. The rejected ideas are documented with the same rigor as the shipped ones: speculative multi-frame decoding was closed by measurement, INT4 KV cache failed outright. Negative results are results.
Voices
The server ships a registry of 33 named voices across 7 languages — drop a .wavand a matching transcript into a folder and that voice is available by name, encoded once at startup, its prompt prefix kept warm in a KV cache. That cache is why a named voice answers in 0.20 s instead of the 1.58 s it used to take. Clients can also send a reference clip with the request and get that voice cloned on the fly. Mixed-language text stays one generation; the voice doesn't change when the language does.
Status
v1.0.1, stable. It ships as a container — two commands and the server is up, fetching the checkpoint into a volume on first start. The HTTP API is a frozen OpenAPI 3.1 contract that evolves backward-compatibly within 1.x. Streaming, cloning, the voice registry and long-form chunking are exercised end to end on real hardware.
The source is Apache-2.0. The weights are not mine to give: S2-Pro is published by Fish Audio under their own research license, and the repository fetches rather than redistributes them.