Offline narration timing recovery

These tools estimate word timestamps from existing recordings for reviewed paragraph-level synchronization. They do not regenerate narration or change source recordings. They are ASR tools, not ground-truth forced alignment: text matching, omissions, equations, citations, headings, tables and changed article content require review before publication.

Setup

Use Python 3.11+, FFmpeg on PATH, and a separate virtual environment with pip install -r requirements.txt. The requirements pin the tested dependency versions. Obtain an English CTranslate2 model separately from its official publisher, then supply its existing local directory. No model is downloaded by these scripts; Hugging Face offline mode is enabled. No recording is uploaded.

The tested model was SYSTRAN faster-whisper-small.en, revision d1d751a5f8271d482d14ca55d9e2deeebbae577f, model.bin SHA256 62b2a45b05ee59acb4a5341b33ee35e041395d378d418a18acfe4c9e768ee37a. Model files occupy approximately 0.5 GB. The script hashes the actual supplied model files and records installed engine versions; it does not claim this provenance for other models.

Run

Choose output directories outside recording/model directories. Existing output files are never overwritten.

python transcribe.py --model-dir /path/to/local/model --output-dir /path/to/asr-output /path/to/article/kokoro-heart.mp3
python refine_silence.py --asr /path/to/asr-output/article.asr.json --recording /path/to/article/kokoro-heart.mp3 --output-dir /path/to/refined-output

Multiple recordings can be passed to transcribe.py to reuse the loaded model. Use --threads to set CPU concurrency (default 4). kokoro-heart.mp3 outputs are named by their parent folder; other recordings use their filename stem. Duplicate names are rejected.

Both tools decode with FFmpeg to an in-memory 16 kHz mono float array and verify source SHA256 before and after processing. Refinement also verifies its recording matches ASR provenance and that the input JSON has not changed. They only write new JSON output and never write to recording/model files. Raw JSON retains original segment scores and every word’s start/end/probability. Refinement preserves those values and adds optional speech_start_estimate plus a flattened word-index change list.

The refinement finds measured quiet gaps at -45 dBFS using 10 ms frames inside unusually long word spans. It requires at least 250 ms quiet and 40 ms audible material afterward. It can remove preceding silence incorrectly attached to a first word, but is not an independent speech recognizer. Apply suggestions only at reviewed, confidently matched source boundaries. Never distribute timings uniformly within a paragraph or section.

Validation and review

On the 464-second RLHF calibration recording, CPU int8 inference took about 52 seconds. Literal token matching to the original synthesis transcript was 94.3%; acronym/number/citation variations remain. Against known synthesis chunk starts, matched first-word median error was 0.09 seconds, maximum 1.515 seconds before refinement; measured silence refinement reduced these to 0.074 and 0.491 seconds. These are calibration observations, not guaranteed accuracy on other recordings. Chunk boundaries may themselves include leading silence.

The 1,207-second harness recording took about 132 seconds; 98.8% of reference tokens matched the ASR transcript. Typical observed inference throughput was about 10 times real time with four CPU threads. Runtime varies with hardware and content. There are no paid API/service calls, though local compute/storage resources are used.

Before publishing: match transcript spans monotonically to current rendered source; inspect low-confidence words, missing/extra content, start/end drift and all unmatched boundaries. Group ambiguous adjacent blocks under a real, reviewed interval or omit unsafe synchronization. Do not invent internal timestamps. Keep original ASR output, refined output, source/model hashes and review records. Verify original MP3 bytes remain unchanged and run browser regressions for seeking, following, resizing and mobile layout.

Official engine documentation: https://github.com/SYSTRAN/faster-whisper

Run the dependency-free safety regression tests with python test_safety.py. They check mismatched-recording rejection and preservation of an existing reviewed output. A separate offline smoke run of transcription and refinement was also tested against the cached model.