s2pro-native
You type a sentence and a voice reads it out. The first sound comes about a seventh of a second later, and from there the speech is made faster than you can listen to it. Any voice, many languages, and you can clone a new voice from a few seconds of a recording.
Most speech models come wrapped in a huge stack of Python that breaks every few months. I removed that stack and rewrote the model itself in C and CUDA, every step of the maths. What ships is one program and the model's weights.

Why I did it
With an open speech model, most of what you deploy is the serving stack: a framework, a runtime, a package manager, and versions that will clash in a year. The model itself is a few gigabytes of weights and about a dozen operations. So I wrote those operations myself against Fish Audio S2-Pro: the prompt building, the transformers, the vocoder, the scheduler and the HTTP server. Python, PyTorch, SGLang and vLLM aren't part of it.
The best part is that nothing is in between anymore. When a frame takes 22 milliseconds, the reason is in code I wrote, and I can go and fix it.
What happens to a sentence
text tokenizer byte-level BPE, written by handbackbone 36 layers, picks one token per frame decoder 4 layers, turns it into ten codes vocoder DAC, 2,048 samples = 46.4 ms of soundagain, until the sentence is done → audio
The text goes through a tokenizer I wrote by hand. On 4,000 random test cases it gives exactly the same output as the original. That sounds boring, but if it's wrong, everything after it is wrong too.
Then two transformers run for every frame. A 36-layer backbone picks one token, and a 4-layer decoder turns it into ten codes. A DAC vocoder turns those codes into 2,048 audio samples, which is 46.4 milliseconds of sound. This repeats until the sentence is finished.
The audio streams out while it is still being made. The vocoder windows overlap and are crossfaded, so you can't hear the seams. A client can play the first word before the last one exists.
How fast it is
Measured on one DGX Spark (GB10, sm_121), streamed over HTTP:
request wall RTF first audio
new voice from a clip 0.51 0.14 s
named voice, warm cache 0.52 0.20 s
long text (~143 s), in chunks 0.50 0.20 sRTF is compute time divided by audio length. Below 1.0, speech is made faster than it plays. It started at 2.05. The first version of this page said realtime was impossible with 16-bit weights, and that was true. The way out was smaller weights that are still correct: 8-bit, then a packed 4-bit backbone, and then I trained the decoder and the output layer to work with 4-bit weights too. A frame now reads 4.37 GB of weights instead of 15.5 GB, and a step takes 21.7 ms of its 46.4 ms budget. Up to five streams can run at once before one falls behind.
I wrote about the way there, and the fast version I threw away, in FP8 was twice as fast, so I didn't ship it.
Checking that it's right
A speech model can be wrong and still sound fine on the one sentence you test. So every change is compared against the original PyTorch model layer by layer, on fixed inputs, with a pass mark I set before the run.
FP8 was twice as fast and failed that check: the error grew over 36 layers until the output only matched with a cosine similarity of 0.33. So it never shipped. INT8 passed everywhere. The 4-bit version passed the same check, an end-of-speech test over 24 pairs of sentences, and careful listening. The ideas that didn't work are written down in the repository too, like speculative multi-frame decoding and a 4-bit KV cache.
Voices
The server comes with 33 named voices in 7 languages. You put a .wav and its transcript into a folder, and that voice can be used by name. It is encoded once at startup and kept warm in a cache, which is why a named voice answers in 0.20 s instead of the 1.58 s it used to take. You can also send a clip with the request and the voice is cloned on the spot. If a text switches language, the voice stays the same.
Status
v1.0.1, stable. It runs as a container: two commands and the server is up, and it downloads the model on the first start. The HTTP API is a fixed OpenAPI 3.1 contract that only changes in compatible ways within 1.x.
The code is Apache-2.0. The weights aren't mine to give away: Fish Audio publishes S2-Pro under their own research license, so the repository downloads them instead of including them.