How word-level subtitle timing actually works
Most tools guess subtitle timings from character counts. Speech synthesis can report exactly when each word was spoken — here is what that data looks like and why it changes the result.
There are two ways to produce a subtitle file from synthesised speech, and the difference between them is the difference between subtitles that line up and subtitles that nearly line up.
The guess
The cheap approach: synthesise the audio, measure how long it came out, then divide that duration across the text proportionally to character count. It requires nothing from the synthesis engine and it is what a lot of free tools do.
It is also wrong in a specific, compounding way. Speech is not evenly paced. A comma buys 80 milliseconds of silence. A full stop buys more. “Through” takes longer to say than “the” despite being closer in length than in duration. Every one of those mismatches pushes the remaining cues further out of alignment, so the file drifts worse the longer it runs — subtitles that look fine in the first sentence and are visibly late by the end.
The measurement
The synthesis service can simply tell you. Alongside the audio stream, it emits a word boundary event for every word it speaks:
{ "text": "sentence", "offset": 1100, "duration": 620 }That is the word, when it starts, and how long it lasts. No estimation is involved — it is the engine reporting what it did.
Build subtitles from those and the timings are not approximately right. They are the timings.
The part nobody warns you about
Word boundary events give you bare tokens. The source text reads
If you can hear this sentence, the protocol works. and the events come back
as sentence and works — no comma, no full stop. Concatenate the tokens and
your subtitles are stripped of every punctuation mark in the original:
If you can hear this sentence the protocol works
Which is unusable for anything anyone would actually read.
The fix is to stop treating the tokens as the text. Walk a cursor through the original source, find where each token came from, and slice the cue out of the source between one token’s position and the next. Punctuation, capitalisation and spacing all survive, because you never rebuilt the text in the first place — you only located it.
One guard matters here: the service normalises some input, so “2026” may be spoken as several words that appear nowhere in your text. When fewer than 80% of tokens can be located, the match is not trustworthy and it is better to fall back to joined tokens than to slice the source at the wrong offsets.
Why one render instead of streaming
There is a tempting optimisation: stream the audio so playback starts sooner, and fetch the timings separately.
It does not work, for a reason that only shows up later. This synthesis is not deterministic — the same text, the same voice, the same settings produce slightly different audio on each call. Timings fetched in a second request describe a different rendering than the audio you streamed. The subtitles would be perfectly accurate for audio nobody has.
So one call returns both, and they are guaranteed to describe each other. The cost is measurable — roughly 400 milliseconds to first audio instead of near instant — and for the tool’s actual purpose that is the right trade. Exact sync is the entire point; a few hundred milliseconds of latency is not.
What you can see
The chronogram on the front page draws this data directly. Every block is one word, positioned at its true offset and drawn to its true duration. The 80 millisecond gap where the comma sits is visible, and so is the 612 milliseconds of silence after the final word. Nothing in that drawing is illustrative — it is the timing data, rendered.