s2pro-native

Pre-release · GitHub ↗

s2pro-native turns text into speech using Fish Audio S2-Pro, and does the whole thing in C and CUDA. Prompt building, the transformer stack, the vocoder, the scheduler, the HTTP server — all of it native. Python, PyTorch, SGLang and vLLM are not part of the runtime.

Why bother

Every open speech model ships with a Python serving stack, and that stack is most of what you actually deploy. It drags in a framework, a runtime, a package manager and a set of versions that will break in eight months. The model itself is a few gigabytes of weights and about a dozen operations.

So I wrote the dozen operations. What ships is a binary, the weights, and a CUDA driver. The interesting consequence isn't the size — it's that there is no layer left to blame. When a frame takes 92 milliseconds, the reason is in code I wrote.

What happens to a sentence

Text goes through a hand-written byte-level BPE tokenizer, which is bit-exact against the reference implementation on a 4,000-case fuzz — the boring part that quietly ruins everything downstream if it's wrong.

Then two transformers in sequence, per frame. A 36-layer backbone picks one semantic token. A 4-layer decoder expands that token into ten residual codes, nine greedy steps deep. Those codes go into a DAC vocoder that produces 2,048 audio samples — 46.4 milliseconds of sound at 44.1 kHz. Repeat until the sentence is finished.

Audio streams out while it's still being generated, with overlapping vocoder windows crossfaded so the seams are inaudible. A client can start playing the first word before the last one exists.

Where it stands

Measured on one DGX Spark (GB10, sm_121), single stream, 41-token prompt, 60 frames, greedy sampling:

RTF is compute divided by audio. Below 1.0 means synthesis outruns playback. It is not there yet, and the reason is bandwidth, not arithmetic: at batch 1 every single frame reads about 7.75 billion weight parameters out of memory. With 16-bit weights that is above realtime on this memory system by construction — no amount of kernel tuning fixes it. Eight-bit weights move the floor to roughly 35 ms per frame.

The gate that says no

Speech quality degrades gracefully and dishonestly. A path can be measurably wrong and still sound fine on the one sentence you tested. So numerical fidelity is checked against the PyTorch reference layer by layer, on fixed inputs, with a pass mark set before the run.

BF16 passes everywhere — cosine similarity ≥ 0.99996 through the backbone, the same first-frame token, the vocoder at 65.9 dB SNR. The FP8 path fails: error compounds across 36 layers until cosine similarity collapses to 0.33. It is twice as fast and it is unusable, so its numbers above stand as a measurement of the memory system and nothing more. INT8 with per-channel weight scales is the current candidate for the 8-bit path, and it goes through the same gate before it goes anywhere near a release.

Voices

Drop a .wavand a matching transcript into a folder and that voice is available by name — encoded once at startup, multilingual by construction, no per-language setup. Clients can also send a reference clip with the request and get that voice back on the fly. Mixed-language text stays one generation; the voice doesn't change when the language does.

The reference voices aren't committed to the repository. One command generates any or all thirty of them at deploy time, which keeps a few hundred megabytes of audio out of Git history.

Status

Functional pre-release. Voice cloning, the voice registry and the streaming HTTP server are exercised end to end on real hardware — around 1.1 s to first audio, 2.5 s with a long reference clip. Single-stream synthesis is still above realtime. There is no release line yet.

The source is Apache-2.0. The weights are not mine to give: S2-Pro is published by Fish Audio under their own research license, and the repository fetches rather than redistributes them.