On-device speech-to-text: what we gained

HardwareOne P4 + G2 glasses, Oct 3-11 2026. Every number below was measured in this project unless marked as an estimate. Version numbers are labels for the test builds, in the order they were made; the firmware's own version did not change.

1.1Oct 10deployedThe firmware on the P4 (gen2-25397). The model is rebuilt for every phrase.
1.2Oct 10, afternoonc3xDesign version: early final, short-reply rule, model kept for the session with its bookkeeping in PSRAM.
1.3Oct 10, eveningc4aFaster depthwise layers on both cores; head start for short replies.
1.4Oct 10, 18:30c5Faster language-model decoder (identical text).
1.5Oct 11, 00:45c7Each layer's setup prepared once per session; 200 ms tail confirmed.
1.6Oct 11, 03:15c9Head start grabs the audio at once, even while a preview runs.
1.7Oct 11, 05:30c11Continuous phrases capped at 10 s (half the session audio buffers).
1.8Oct 11, 08:20c12 + card LM1.6 MB language model with 9.2k tech terms (43k words) instead of 2.5 MB.
1.9Oct 11, 08:30c13, firstSpeaker labels ("Person N"), computed in the speech worker.
2.0Oct 11, 09:20-10:45c13, finalSpeaker labels in a background task, ESP-IDF fix, trimmed kernels, labelled transcript lines.
Caption delay after you stop talking
3.6 s → ~1.0-1.1 s
1.1 → 1.3-1.8, P4 mic, quiet room
Short replies ("yeah", "okay")
3.4-17 s → ~0.9 s
no longer wait for the next sentence
Free PSRAM at its lowest, long speech
1.6 → 5.4 MB
1.6 → 1.7, the 10 s phrase cap
Speaker labels (2.0)
91% right
no caption delay added
Invented words on pure noise
30 → 9.5 /min
deployed model vs old model
500-hour training run on the Mac
33 h → ~10 h
same results, 3x faster steps

1. The model: accuracy and noise

Broad error rate on the full test sets (lower is better)

NVIDIA original 26.83 % Old P4 model (general) 23.11 % general2, LR 2e-5 23.92 % general + noise clips 23.18 % ON THE P4: general2, LR 1e-5 22.80 % best

Words invented on pure background noise (lower is better)

NVIDIA original 18.8 words/min Old P4 model (general) 30.1 words/min general2, LR 2e-5 8.0 words/min general + noise clips 8.8 words/min ON THE P4: general2, LR 1e-5 9.5 words/min -68% vs old

The noise-only training clips cut invented words by about 70% at no accuracy cost (general + noise clips: 23.18 vs 23.11). The new data and the 500-hour run gave the deployed model its accuracy edge. The 2e-5 learning rate was the mistake to avoid.

2. Training speed on the Mac

Training throughput (audio seconds processed per second)

Stock trainer 18.5 audio-s/s Depthwise-conv fix + masks + fused AdamW 55.1 audio-s/s 3.0x

One checkpoint scoring pass

Checkpoint scoring on the CPU 1300 s 16 per run On the graphics chip 109 s same scores

Where a 500-hour run's time goes

Before training steps: 26.6 h checkpoint scoring: 5.8 h test sets: 0.6 h 33.0 h After (projected) training steps: 8.6 h checkpoint scoring: 0.5 h test sets: 0.6 h 9.7 h training steps checkpoint scoring test sets

The cause: one operation (the backward pass of the depthwise convolutions) was very slow on the Mac's GPU. A replacement written in this project gives the same gradients to within a millionth.

3. Caption delay on the P4

End of speech to caption ready (median, P4 mic)

1.1 3.61 s all phrases 1.1 + cache, -O2, model reuse 2.90 s step toward 1.2 1.2 1.69 s 1.3 1.08 s 3 sessions, 55 phrases 1.4 1.21 s 1 session, 18 phrases 1.5 1.07 s 11 sessions, 203 phrases 1.6 1.08 s 6 sessions, 112 phrases 1.7 1.07 s 3 sessions, 50 phrases 1.8 1.00 s 1 session, 14 phrases

From 1.3 on, each change saved tens of milliseconds, which is smaller than the spread between sessions (about ±0.1 s, more for one-session versions). So the medians from 1.4 on move up and down by noise; what each change did is in the per-change chart below, measured clip by clip. 1.4 and 1.8 were run once, so their medians rest on 14-18 phrases.

Short replies

1.1 3.40 s and up to 17 s when merged 1.1 + cache, -O2, model reuse 3.60 s step toward 1.2 1.2 1.45 s 1-s close for short replies 1.3 0.89 s head start in the pause 1.4 0.55 s 1 session 1.5 0.88 s 1.6 0.88 s 1.7 0.72 s 1.8 0.86 s 1 session

Longer phrases

1.1 3.39 s 1.2 1.98 s 1.3 1.76 s faster model 1.4 1.70 s 1 session 1.5 1.63 s 1.6 1.67 s 1.7 1.60 s 1.8 1.73 s 1 session

Speech model time per second of audio (median phrase)

1.1 412 ms per audio-s 1.2 304 ms per audio-s 1.3 266 ms per audio-s 1.4 262 ms per audio-s 1.5 251 ms per audio-s 1.6 253 ms per audio-s 1.7 258 ms per audio-s 1.8 261 ms per audio-s

Language-model step per phrase (median)

1.1 157 ms 1.2 102 ms 1.3 104 ms 1.4 48 ms faster decoder 1.5 43 ms 1.6 46 ms 1.7 48 ms 1.8 47 ms

Model time fell about 40% from 1.1 to 1.5 and then stayed flat: 1.6-1.8 changed when the model starts and how much memory it needs, not its speed. The language-model step halved with the new decoder in 1.4; its size (2.5 or 1.6 MB) does not change its speed.

Long unbroken speech: the 10 s phrase cap (12-17 s utterances)

1.6 (20 s phrase cap) 1.93 s 1.7 (10 s phrase cap) 1.43 s per clip: median -34 ms, mean -369 ms

Medians over the same 14 clips, 3 sessions each. With a 20 s cap a long utterance is transcribed in one piece after you stop; with 10 s the first part is cut and transcribed while you are still talking. For a typical clip that changes little (-34 ms), but some finish much sooner (95% range -416 to +19 ms). The main gain is memory (section 4).

Where the time goes, step by step: all phrases

1.1 waiting for the phrase to end: 0.94 s waiting for the processor: 0.03 s model rebuild: 0.16 s audio features: 0.15 s model: pointwise layers: 1.52 s model: depthwise layers: 0.83 s model: adds and others: 0.08 s language model: 0.15 s hand-off to the caption: 0.02 s 3.88 s 1.2 waiting for the phrase to end: 0.49 s head start: 200 ms of quiet: 0.03 s waiting for the processor: 0.01 s audio features: 0.01 s model: pointwise layers: 0.69 s model: depthwise layers: 0.36 s model: adds and others: 0.03 s language model: 0.09 s hand-off to the caption: 0.01 s 1.71 s 1.3 waiting for the phrase to end: 0.09 s head start: 200 ms of quiet: 0.08 s audio features: 0.01 s model: pointwise layers: 0.70 s model: depthwise layers: 0.24 s model: adds and others: 0.03 s language model: 0.08 s hand-off to the caption: 0.01 s 1.25 s 1.4 waiting for the phrase to end: 0.02 s head start: 200 ms of quiet: 0.03 s audio features: 0.01 s model: pointwise layers: 0.80 s model: depthwise layers: 0.28 s model: adds and others: 0.04 s language model: 0.05 s hand-off to the caption: 0.01 s 1.25 s 1.5 waiting for the phrase to end: 0.11 s head start: 200 ms of quiet: 0.08 s audio features: 0.01 s model: pointwise layers: 0.66 s model: depthwise layers: 0.22 s model: adds and others: 0.04 s language model: 0.04 s hand-off to the caption: 0.01 s 1.16 s 1.6 waiting for the phrase to end: 0.13 s head start: 200 ms of quiet: 0.08 s audio features: 0.01 s model: pointwise layers: 0.64 s model: depthwise layers: 0.21 s model: adds and others: 0.04 s language model: 0.04 s hand-off to the caption: 0.01 s 1.15 s 1.7 waiting for the phrase to end: 0.10 s head start: 200 ms of quiet: 0.05 s audio features: 0.01 s model: pointwise layers: 0.66 s model: depthwise layers: 0.21 s model: adds and others: 0.04 s language model: 0.04 s hand-off to the caption: 0.01 s 1.13 s 1.8 waiting for the phrase to end: 0.11 s head start: 200 ms of quiet: 0.07 s audio features: 0.01 s model: pointwise layers: 0.60 s model: depthwise layers: 0.19 s model: adds and others: 0.03 s language model: 0.03 s hand-off to the caption: 0.01 s 1.06 s waiting for the phrase to end head start: 200 ms of quiet waiting for the processor model rebuild audio features model: pointwise layers model: depthwise layers model: adds and others language model hand-off to the caption

Short replies (same colours)

1.1 waiting for the phrase to end: 2.17 s waiting for the processor: 0.03 s model rebuild: 0.16 s audio features: 0.12 s model: pointwise layers: 1.27 s model: depthwise layers: 0.70 s model: adds and others: 0.07 s language model: 0.12 s hand-off to the caption: 0.02 s 4.66 s 1.2 waiting for the phrase to end: 0.75 s waiting for the processor: 0.01 s audio features: 0.01 s model: pointwise layers: 0.40 s model: depthwise layers: 0.19 s model: adds and others: 0.02 s language model: 0.05 s hand-off to the caption: 0.01 s 1.43 s 1.3 waiting for the phrase to end: 0.15 s audio features: 0.01 s model: pointwise layers: 0.42 s model: depthwise layers: 0.14 s model: adds and others: 0.02 s language model: 0.06 s hand-off to the caption: 0.01 s 0.81 s 1.4 waiting for the phrase to end: 0.01 s audio features: 0.01 s model: pointwise layers: 0.41 s model: depthwise layers: 0.14 s model: adds and others: 0.02 s language model: 0.03 s hand-off to the caption: 0.01 s 0.63 s 1.5 waiting for the phrase to end: 0.16 s audio features: 0.01 s model: pointwise layers: 0.38 s model: depthwise layers: 0.11 s model: adds and others: 0.02 s language model: 0.03 s hand-off to the caption: 0.01 s 0.72 s 1.6 waiting for the phrase to end: 0.17 s audio features: 0.01 s model: pointwise layers: 0.39 s model: depthwise layers: 0.12 s model: adds and others: 0.02 s language model: 0.03 s hand-off to the caption: 0.01 s 0.74 s 1.7 waiting for the phrase to end: 0.15 s audio features: 0.01 s model: pointwise layers: 0.38 s model: depthwise layers: 0.11 s model: adds and others: 0.03 s language model: 0.02 s hand-off to the caption: 0.01 s 0.71 s 1.8 waiting for the phrase to end: 0.18 s audio features: 0.01 s model: pointwise layers: 0.44 s model: depthwise layers: 0.13 s model: adds and others: 0.03 s language model: 0.03 s hand-off to the caption: 0.01 s 0.82 s

Longer phrases (same colours)

1.1 waiting for the phrase to end: 0.42 s waiting for the processor: 0.03 s model rebuild: 0.17 s audio features: 0.16 s model: pointwise layers: 1.63 s model: depthwise layers: 0.89 s model: adds and others: 0.09 s language model: 0.17 s hand-off to the caption: 0.01 s 3.56 s 1.2 head start: 200 ms of quiet: 0.08 s audio features: 0.02 s model: pointwise layers: 1.09 s model: depthwise layers: 0.59 s model: adds and others: 0.05 s language model: 0.13 s hand-off to the caption: 0.01 s 1.97 s 1.3 head start: 200 ms of quiet: 0.16 s audio features: 0.02 s model: pointwise layers: 1.01 s model: depthwise layers: 0.36 s model: adds and others: 0.05 s language model: 0.12 s hand-off to the caption: 0.01 s 1.72 s 1.4 head start: 200 ms of quiet: 0.03 s audio features: 0.02 s model: pointwise layers: 1.12 s model: depthwise layers: 0.40 s model: adds and others: 0.05 s language model: 0.06 s hand-off to the caption: 0.01 s 1.70 s 1.5 head start: 200 ms of quiet: 0.13 s audio features: 0.02 s model: pointwise layers: 1.01 s model: depthwise layers: 0.34 s model: adds and others: 0.05 s language model: 0.06 s hand-off to the caption: 0.01 s 1.62 s 1.6 head start: 200 ms of quiet: 0.05 s audio features: 0.02 s model: pointwise layers: 1.08 s model: depthwise layers: 0.37 s model: adds and others: 0.05 s language model: 0.06 s hand-off to the caption: 0.01 s 1.64 s 1.7 head start: 200 ms of quiet: 0.16 s audio features: 0.02 s model: pointwise layers: 0.98 s model: depthwise layers: 0.33 s model: adds and others: 0.05 s language model: 0.05 s hand-off to the caption: 0.01 s 1.61 s 1.8 head start: 200 ms of quiet: 0.06 s audio features: 0.03 s model: pointwise layers: 1.07 s model: depthwise layers: 0.36 s model: adds and others: 0.05 s language model: 0.06 s hand-off to the caption: 0.01 s 1.65 s

The model's split by layer type uses each build's own per-layer timing on the chip: 1.4 changed only the language-model decoder, so it uses 1.3's; 1.5 to 1.8 share one model build, timed in the 1.8 test sessions. 1.4 and 1.8 were run once (7-18 phrases per bar), so their bars are rougher; 2.0 is not shown because it was tested on conversation recordings, where it matched the same build without speaker labels clip by clip (-2 ms). Each bar starts when you stop talking and ends when the caption is ready, and the pieces are in the order they happen. Each bar is the average of the middle half of that group's phrases (so one odd phrase cannot skew it), so totals sit close to the medians above. With the head start, the model runs while the end-of-phrase wait is still going, so the grey wait mostly disappears; the light grey piece is the 200 ms of quiet the P4 needs before it starts. The model's split by layer type comes from each version's own per-layer timing on the chip (1.1: an older build of the same model).

Inside the model run, by layer type (a 5-second phrase)

1.1* pointwise 512x512 (53): 1.07 s pointwise 256x256 (36): 0.14 s pointwise head + first (5): 0.07 s depthwise k75 (15): 0.28 s depthwise k63 (15): 0.16 s depthwise k51 (15): 0.11 s depthwise k87 dilated (1): 0.08 s depthwise k33/k39 (31): 0.07 s adds + ReLU (30): 0.07 s 2.06 s 1.2 pointwise 512x512 (53): 0.87 s pointwise 256x256 (36): 0.13 s pointwise head + first (5): 0.06 s depthwise k75 (15): 0.22 s depthwise k63 (15): 0.14 s depthwise k51 (15): 0.11 s depthwise k87 dilated (1): 0.04 s depthwise k33/k39 (31): 0.07 s adds + ReLU (30): 0.05 s 1.69 s 1.3, 1.4 pointwise 512x512 (53): 0.83 s pointwise 256x256 (36): 0.14 s pointwise head + first (5): 0.05 s depthwise k75 (15): 0.11 s depthwise k63 (15): 0.09 s depthwise k51 (15): 0.08 s depthwise k87 dilated (1): 0.01 s depthwise k33/k39 (31): 0.07 s adds + ReLU (30): 0.05 s 1.44 s 1.5-1.8 pointwise 512x512 (53): 0.84 s pointwise 256x256 (36): 0.13 s pointwise head + first (5): 0.05 s depthwise k75 (15): 0.10 s depthwise k63 (15): 0.09 s depthwise k51 (15): 0.08 s depthwise k87 dilated (1): 0.01 s depthwise k33/k39 (31): 0.06 s adds + ReLU (30): 0.05 s 1.41 s pointwise 512x512 (53) pointwise 256x256 (36) pointwise head + first (5) depthwise k75 (15) depthwise k63 (15) depthwise k51 (15) depthwise k87 dilated (1) depthwise k33/k39 (31) adds + ReLU (30)

Numbers in brackets are how many layers of that type the model has. Depthwise layers are 9% of the model's arithmetic but took about a third of its time: each step read more data than the chip's 64 KB fast cache holds. Working in 64-channel blocks and splitting the channels across both cores fixed that, with bit-identical output. 1.5's prepared layer setups took a further 30 ms off, mostly in the smaller layers, where setup is a larger share; the 512-wide pointwise layers are now 60% of the run. *1.1's split comes from an older per-layer profile of the same model.

What each change did (compared one at a time)

← faster slower → Larger cache line (128 B L2) -344 ms Build the model once per session -187 ms (now in PSRAM) Faster memory clock 250 MHz -68 ms (not used yet) Compiler -O2 for speech code -41 ms Features computed while you talk -120 ms Cheaper pause between model stages -69 ms 16 KB filter tiles -58 ms Early final + silence trim -501 ms Short replies close after 1 s silence -2.2 s (short replies) Both cores for long filters +81 ms (rejected) Smaller frame tiles +255 ms (rejected) Depthwise in 64-channel blocks -320 ms (6.6 s input; 0 on 4 s) + both cores, split by channels -107 ms (4 s input; -96 on 2.5 s) + 64-step pointwise tiles -92 ms (4 s; -136 on 6.6 s) Head start for short replies -590 ms (short replies, median) Bigger 256 KB cache (4 s input) -240 ms (Rejected for excessive RAM consumption) 1.4: faster LM decoder -56 ms (LM step per phrase, same text) 1.5: layer setups prepared once -50 ms (43-57 ms, fixed audio; 0 internal RAM) Tail 100 ms instead of 200 ms (long phrases) -147 ms (rejected: +1.1 WER points) Tail 150 ms instead of 200 ms -22 ms (rejected: no clear gain) Pointwise split by channels +286 ms (rejected (4 s input)) Vector ReLU (branch only) -7 ms (3-10 ms, fixed audio) 1.6: grab audio from a running preview -32 ms (head-start wait, longer phrases) 16 ms mic chunks +24 ms (rejected: -22 ms short, +21-27 ms longer) 1.7: 10 s phrase cap -34 ms (long speech; +3.8 MB PSRAM free) 2.0: speaker labels in background -2 ms (no caption cost (matched))

The chip has no neural-net accelerator; its SIMD units were mostly waiting on memory. That is why the cache line was the biggest early win, and why using both cores only paid off once each core worked on its own channels. The last five rows were measured on fixed test audio (model time) or, for short replies, in speaker sessions.

3b. Through the G2 glasses (Bluetooth)

With the glasses, the speech engine on the P4 is the same; Bluetooth adds the trip in (mic audio) and the trip out (the caption). Green boxes are the parts this work changed.

G2 mic left temple LC3 encode 10 ms frames Bluetooth LE 1 packet / 50 ms C6 radio chip to the P4 (SDIO) LC3 decode on the P4 Phrase detector end + short replies Speech model int8, ESP-DL Language model word search Caption sender wakes on publish Bluetooth LE right temple

End of speech to caption on the glasses (longer phrases)

1.1 mic audio over Bluetooth (est.): 0.08 s end-of-phrase wait: 0.60 s speech processing: 2.79 s caption timer: 0.11 s caption to glasses + ack: 0.07 s 3.65 s 1.2 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.98 s caption to glasses + ack: 0.07 s 2.13 s 1.3 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.76 s caption to glasses + ack: 0.07 s 1.91 s 1.4 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.70 s caption to glasses + ack: 0.07 s 1.85 s 1.5 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.63 s caption to glasses + ack: 0.07 s 1.78 s 1.6 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.67 s caption to glasses + ack: 0.07 s 1.82 s 1.7 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.60 s caption to glasses + ack: 0.07 s 1.75 s 1.8 mic audio over Bluetooth (est.): 0.08 s speech processing: 1.73 s caption to glasses + ack: 0.07 s 1.88 s mic audio over Bluetooth (est.) end-of-phrase wait speech processing caption timer caption to glasses + ack

Short replies ("yeah", "okay")

1.1 mic audio over Bluetooth (est.): 0.08 s wait + processing: 3.40 s caption timer: 0.11 s caption to glasses + ack: 0.07 s 3.66 s 1.2 mic audio over Bluetooth (est.): 0.08 s wait + processing: 1.45 s caption to glasses + ack: 0.07 s 1.60 s 1.3 mic audio over Bluetooth (est.): 0.08 s wait + processing: 0.89 s caption to glasses + ack: 0.07 s 1.04 s 1.4 mic audio over Bluetooth (est.): 0.08 s wait + processing: 0.55 s caption to glasses + ack: 0.07 s 0.70 s 1.5 mic audio over Bluetooth (est.): 0.08 s wait + processing: 0.88 s caption to glasses + ack: 0.07 s 1.03 s 1.6 mic audio over Bluetooth (est.): 0.08 s wait + processing: 0.88 s caption to glasses + ack: 0.07 s 1.03 s 1.7 mic audio over Bluetooth (est.): 0.08 s wait + processing: 0.72 s caption to glasses + ack: 0.07 s 0.87 s 1.8 mic audio over Bluetooth (est.): 0.08 s wait + processing: 0.86 s caption to glasses + ack: 0.07 s 1.01 s mic audio over Bluetooth (est.) wait + processing caption timer caption to glasses + ack

Measured: speech processing (P4, same engine as with the glasses), the caption timer (0-200 ms, about 0.11 s on average; from 1.2 on the sender wakes as soon as a caption is ready) and the caption write to the glasses with its acknowledgement (30-102 ms). Estimated: the mic audio trip over Bluetooth (packets every 50 ms plus radio transfer; not measured directly) and the glasses' own drawing time (not included). From 1.2 on the end-of-phrase wait and the processing overlap (the final starts early), so they are one blue bar. With the glasses 1.2 should land at about 2.1 s for longer phrases and 1.3 at about 1.9 s (short replies about 1.0 s), and 1.4-1.8 at about 1.8-1.9 s (short replies 0.7-1.0 s), against about 3.7 s; none of them has been timed on the glasses yet. 2.0 is not shown: its speaker labels add nothing to the caption path (section 6), so it behaves like 1.8. Earlier glasses logs reported 3.6-5.5 s because up to ~1 s of that was a clock offset in the glasses' audio stream, not real delay.

4. Memory and accuracy by version

Lowest free PSRAM during a session (standard playlist)

1.1 4.15 MB model rebuilt per phrase 1.2 2.49 MB 1.3 2.45 MB 1.4 2.43 MB 1.5 2.27 MB 1.6 2.25 MB 1.7 5.25 MB 1.8 6.25 MB LM 1.6 MB 2.0 (with 2.5 MB LM) 2.04 MB speaker model 3.4 MB 2.0 (with 1.6 MB LM) 3.02 MB shipped combination

Lowest free PSRAM on long unbroken speech

1.6, long speech 1.63 MB 20 s phrase buffers 1.7, long speech 5.45 MB 10 s phrase buffers

Out of 32 MB. 1.1 rebuilt the model for every phrase, so it held less between phrases. From 1.2 the model and its working area stay for the session (about 2.6 MB more). 1.7 halves the five session audio buffers and the feature caches: 2.7 MB more free after warm-up, and 3.8 MB more at the lowest point on long speech, where phrases used to reach 20 s. 1.8's smaller LM gives back 0.9 MB. 2.0's speaker model takes 3.4 MB: with the 1.6 MB LM about 3 MB stays free at the lowest point.

Lowest free internal RAM during a session

1.1 44 KB dip in each phrase rebuild 1.2 116 KB 1.3 117 KB 1.4 112 KB 1.5 116 KB 1.6 116 KB 1.7 143 KB test setup: +17 KB at boot from batch 17 1.8 143 KB 2.0 134 KB speaker model load

Internal RAM is the scarce kind (about 260-280 KB free at boot). 1.1 dipped to about 44 KB while it rebuilt the model for each phrase; from 1.2 the model is built once and its bookkeeping is requested from PSRAM. From test batch 17 on (during 1.6's tests) the same firmware booted with about 17 KB more free internal RAM, so the step to ~140 KB is a test-setup change, not the 10 s cap; compare versions within a batch. In the same batch, 2.0's speaker labels lower the low point by 6-11 KB (about 144 → 134 KB), set while ESP-DL builds the speaker model at session start.

Word errors, scored clip by clip (standard playlist, lower is better)

1.1 19.7 % 3 sessions, 26 of 27 clips 1.2 19.0 % 2 sessions, 26 of 27 clips 1.3 18.9 % 3 sessions, 23 of 27 clips 1.4 20.1 % 1 session, 25 of 27 clips 1.5 18.3 % 11 sessions, 21 of 27 clips 1.6 19.1 % 6 sessions, 22 of 27 clips 1.7 18.2 % 3 sessions, 22 of 27 clips 1.8 18.3 % 1 session, 25 of 27 clips

Clips whose console text line was lost in any of that version's sessions are left out (a test-harness fault, not the firmware), so each version is scored on 21-26 of the 27 clips. Every version lands between 18.2% and 20.1%; sessions of the same build vary by about ±2 points. The speed changes are bit-identical on fixed test audio, so their text cannot change; the changes that alter what the model hears (1.2's early final and short-reply rule, 1.7's cut at 10 s) show no clear accuracy cost here. The one change that did cost accuracy, the 100 ms tail, was rejected.

5. The language model (1.8)

Everyday speech: word errors by LM size (P4 recordings re-decoded on the Mac)

No language model 21.75 % 1.0 MB 18.79 % 1.6 MB 18.70 % 2.5 MB (1.1-1.7) 18.70 % 4.0 MB 18.51 % 1.6 MB + 9.2k tech terms (1.8) 18.56 % chosen

Tech speech: 180 synthesized sentences (word errors; right column: unknown words recognised)

No language model 52.6 % 11 of 126 2.5 MB (1.1-1.7) 50.1 % 8 of 126 1.6 MB + tech terms (1.8) 49.9 % 16 of 126 2.5 MB + terms + Wikipedia 49.4 % 18 of 126 4.0 MB + terms + Wikipedia 49.4 % 18 of 126

The P4 logged each caption's raw speech-model output, and the Mac re-ran the device's exact decoder on it (it reproduces every device caption), so every LM was compared on identical audio. The LM is worth about 3 points; its size barely matters, because the speech model gets most words right on its own and the LM mainly breaks ties that depend on common word pairs, which every size keeps. Decode time is set by the speech, not by memory or the session's load, and is the same for every size. The tech term list doubles the unknown words recognised (8 → 16 of 126), but most tech words still fail: the speech model has never heard them and spells them by sound. The next training run adds synthesized tech speech to fix that at the source.

6. Speaker labels (1.9, 2.0)

Each finished phrase gets a session label, and saved transcript lines read "[Person N] text". The model (clova Fast ResNet-34, 1.4 M weights, 8-bit weights with 16-bit activations) turns a phrase into a voiceprint; phrases join the closest voice heard so far, or start a new one. A different speaker model can be dropped onto the card without new firmware.

Phrases labelled with the right person

Mac: audiobooks 95 % 21 conversations Mac: parliament 94 % 21 conversations Mac: meetings 80 % short turns P4: 4 conversations (1.9) 91 % 2-4 people each

Labelling time per phrase

1.9, first build 1.42 s per phrase in the speech worker 1.9, kernels + 3x3 blocking 0.68 s per phrase in the speech worker 2.0 0.68 s per phrase background task, captions never wait

Device accuracy is from the four known-speaker conversations (2, 4, 3 and 3 people); repeat runs in the background build landed at 76-85% per conversation, within the run-to-run spread, and the device's voiceprints match the Mac's (cosine 0.997). Two similar audiobook readers merged into one person: their voiceprints are too close for any threshold, a limit of this small model. Turns under 2 s never start a new person. In 2.0, captions did not wait at all: matched clip by clip against the same build without labels, speech end to caption changed by -2 ms (95% range -11 to +4 ms). Labels reach the transcript 0.8-1.9 s after the caption (median); every saved line got its label, in order. Costs: 3.4 MB of PSRAM, about 110 KB of flash, about 0.5 s at session start, and 6-11 KB of internal RAM at the low point. The 3x3 layers of the speaker model got their own cache-sized blocking in ESP-DL (195 → 39 ms each, bit-identical).

7. Trade-offs still open

Accuracy: the 1.3 report flagged about 2.5 points more word errors for 1.2 (18.2% → 20.5% in raw session averages). Scored clip by clip with lost console lines left out, 1.1 to 1.8 all land between 18.2% and 20.1%, so that loss does not reproduce. Tech words are the clear weak spot (about 50% errors on tech sentences); the next training run targets them.

Memory: correction to the 1.3 report: 1.1's internal RAM fell to about 44 KB during each phrase's model rebuild, not 117 KB; from 1.2 the low point is about 112-117 KB (about 143 KB in later test batches). The speaker model's load at session start lowers it by 6-11 KB; moving ESP-DL's short-lived build buffers to PSRAM explicitly would win that back. A 256 KB cache was measured 11-22% faster but costs 131 KB of internal RAM, so it was rejected for excessive RAM consumption; the cache stays at 128 KB.

Speaker labels: similar voices can merge (a whole-conversation relabelling pass or a larger model would help, if memory allows); overlapping speech gets one label.

Not used yet: the faster memory clock (-68 ms) needs a long stability test first. Not yet timed on the glasses themselves. 1.4-2.0 ran in test builds; the stt-speedups branch carries all of it but has not run on the P4 as a whole yet (planned for 0.99.98, after the 15x5 switch).

8. How it went

Oct 3General model on the P4 (Broad 23.11). Data moved to one external SSD.
Oct 5Noise-only clip bug found and fixed (they were ~90% of the training signal). User: train from the ground up.
Oct 6-7LR test: 2e-5 hurt early and never fully recovered; 1e-5 matched then beat the old model.
Oct 8general2 1e-5: best model yet (Broad 22.80, invented words 30 -> 9.5 per minute).
Oct 9Deployed to the P4. Speed audit: 9 measurements, 54 ideas, 126 agents. Trainer 3x faster.
Oct 10Six rounds of on-chip tests through the speaker and the P4 mic: caption delay 3.6 s -> ~1.7 s.
Oct 10 (eve)Per-layer timing on the chip: depthwise layers were 9% of the math but 34% of the time. Reordered (bit-identical) and head start for short replies: ~1.7 s -> ~1.1 s. Bigger cache measured, rejected (RAM).
Oct 10 (18:30)1.4: the language-model decoder rewritten, 63% faster on the P4 with identical text. All of 1.3 ported to the local stt-speedups branch.
Oct 11 (00:00-03:30)1.5: every layer's setup prepared once per session (no internal RAM). Its first build pushed free PSRAM under the LM's 6 MB load rule, so the LM did not load; fixed. Tail tests: 100 ms cost 1.1 points of accuracy, 150 ms gained nothing clear; 200 ms kept.
Oct 11 (03:00-05:10)1.6: the head start takes the audio straight away even while a preview runs (wait 77 -> 45 ms). 16 ms mic chunks measured and rejected: the model ran 3% slower.
Oct 11 (05:30)1.7: continuous phrases capped at 10 s. On long unbroken speech the lowest free PSRAM went from 1.6 to 5.4 MB.
Oct 11 (06:00-08:20)LM probe: decode time is set by the speech, not by memory or load, and LM size changes memory, not speed. Tech vocabulary (9.2k terms) built; 1.8 = a 1.6 MB LM with the terms: same accuracy, 0.9 MB less PSRAM.
Oct 11 (08:30-09:35)1.9: speaker labels on the P4 (91% of phrases right). 2.0: labelling moved to a background task; that exposed an ESP-IDF bug (a vector register not saved on task switches), fixed with the upstream patch.
Oct 11 (10:15-11:00)1.6 MB LM installed on the card, speaker kernels trimmed, labelled transcript lines, diagnostics compiled out. Everything ported to the stt-speedups branch (builds, host tests pass).