Full-duplex recognition research #1007
Description
Hi Alex,
I've been building on FluidAudio since March and I love it. Thank you for the library, and for how quickly #827 and the #897 follow-ups landed. That speed is why I went this deep.
Why this is worth your time
End of utterance is the one part of the Parakeet EOU path that CI doesn't exercise today: the benchmark comment on #987 runs 100 files and reports EOU Detections: 0. For the past month I've run StreamingEouAsrManager on an always-open microphone, for full-duplex voice chat in ToolPiper (my local AI app for macOS). That is the missing coverage: every turn ends on the end token, and people pause mid-sentence. This issue hands you what came out of it.
What you get
- Two decoder behaviors after
<EOU>that cost a voice app the user's words, each reduced to one clip, one per-chunk trace and one line of code (findings 1 and 2). - A regression corpus and replay driver for CI, so end of turn has a test. I'll regenerate the clips in a redistributable voice and send the PR.
- A measured two-pass recipe you can document: EOU for the endpoint, TDT v2 for the final text. The final pass costs about 53 ms on an M2 Max and 68 ms on an M1 Max (warm median, in process) and reads every word the 160 ms stream got wrong.
- A tester. I'll run any branch that touches EOU, VAD or TDT v2 on both machines and report back with traces.
What I need from you
Three answers. Everything below them is supporting detail.
- What is the contract after
<EOU>? Must the callerreset()before any more speech, or should the stream keep decoding? Today it does one or the other depending on where the pause falls, and I can't tell which is intended. I'm asking instead of sending a patch because the two obvious fixes both made things worse (finding 2). - Would you take a PR that adds an end-of-turn test to CI? A handful of short clips with trailing silence and mid-utterance pauses, asserting one detection per utterance and no lost or repeated words.
- Would you take additive API for apps that own their endpointing? The raw per-chunk end-token signal with its score, and a
rearm(keepTranscript:). I'll write it once you say what shape you want.
Scope. Measured at v0.17.5 (0b1f4628). RnntDecoder.swift, StreamingEouAsrManager.swift and VadManager+Streaming.swift are byte-identical on main at fc8c3e4 (2026-10-09), so everything here applies to v0.17.7 and main, and line numbers refer to that source. Where I'm inferring instead of measuring, I say so.
| # | Finding | What it is | What it needs |
|---|---|---|---|
| 1 | Words spoken after a mid-sentence <EOU> never decode |
A bug, or an undocumented contract | Answer 1 |
| 2 | Words spoken after <EOU> at a different pause decode twice |
A bug | Answer 1 |
| 3 | The 160 ms tier substitutes short command words that 320 ms reads correctly | A question about the 160 ms export | Your read |
| 4 | The streaming VAD's speechEnd arrives one 256 ms block later than it could |
Faithful to Silero, expensive at this block size | A doc note or an option |
| 5 | Small items: fused model files, an unused score, one comment, the short-audio guard | Housekeeping | PRs if you want them |
If you'd rather track these separately, say so and I'll split them into their own issues.
1. Words spoken after a mid-sentence <EOU> never decode
Clip: "Set the owned fixture counter to [pause] seven." The four P clips put 400, 700, 1000 or 1300 ms of digital silence at the pause and share every speech sample.
| Clip | EOU 160 ms | EOU 320 ms | TDT v2 |
|---|---|---|---|
| P400 to P1300 | set the own fixture counter too | set the own fixture counter to | Set the own fixture counter to seven. |
"seven" is lost at every pause length and at both chunk sizes. P400 at 160 ms (pause at samples 29439 to 35839, "seven" spoken from 35839 to 50122; samples is totalSamplesProcessed after the chunk, so chunk k reads input samples k*1280 to k*1280+2560):
chunk=20 samples=26880 tokens=["▁too"] words="too" eouSignal=false anchor=nil confirmed=false
chunk=21 samples=28160 tokens=[] words="" eouSignal=false anchor=nil confirmed=false
chunk=22 samples=29440 tokens=[] words="" eouSignal=false anchor=nil confirmed=false
chunk=23 samples=30720 tokens=[] words="" eouSignal=true anchor=30720 confirmed=true
chunk=24 samples=32000 tokens=[] words="" eouSignal=true anchor=30720 confirmed=false
# chunks 25 to 27: the same, eouSignal=true
chunk=28 samples=37120 tokens=[] words="" eouSignal=false anchor=30720 confirmed=false
chunk=29 samples=38400 tokens=[] words="" eouSignal=false anchor=30720 confirmed=false
# chunks 30 to 40, while "seven" is spoken: tokens=[] eouSignal=false
The end token fires in chunk 23, the first window wholly inside the pause. When "seven" starts, eouSignal goes false, so the model registers speech, but every frame decodes blank until reset(). The manager isn't gating this: it decodes every chunk whether or not eouDetected has latched.
A nonzero debounce doesn't protect against it. This is my reading of the code, not a separate measurement, because I ran at eouDebounceMs: 0. evaluateEouDebounce clears its anchor only when a chunk decodes words (line 310), and after the raw end token none do. With the default 1280 ms, the timer set in chunk 23 would run straight through the spoken "seven" and confirm, with the word still lost.
Probes on v0.15.7, whose end-token and encoder-cache code is the same:
| Change at the end token | Effect on "seven" |
|---|---|
End only the current frame instead of break outerLoop |
No change, so skipped frames are not the cause |
| Reset the decoder (prediction network) state | Decodes, but 2.5 to 4.5 s late and sometimes twice |
| Reset the encoder caches and decoder state, keep the transcript | Decodes near real time on P400 and P1300, 0.5 to 0.8 s late on P700 and P1000 |
My inference is that the "utterance is over" state lives mainly in the encoder caches. It is also position-dependent: the clips in finding 2 keep decoding after their end token with no re-arm at all, and re-arming there ends the turn inside the pause and loses "by".
What holds up in my app today: hold confirmation while the VAD still reports voice, and re-arm (fresh encoder caches and decoder, transcript kept, audio replayed from the end token) only when the VAD reports voiced audio that the stream isn't decoding. It works, but every open-microphone integrator has to rediscover it. That is the reason for answer 1 and for offering rearm(keepTranscript:).
2. Words spoken after <EOU> at a different pause decode twice
Clip: "Set the owned fixture counter to seven, then increase it [pause] by two." PB400 pauses for 400 ms and PB1000 for 1000 ms.
| Reader | Transcript |
|---|---|
| EOU, 160 ms | set the owned fixture counter to seven then increase it by two by two |
| EOU, 320 ms | set the owned fixture counter to seven then increase it by two |
| TDT v2 | Set the owned fixture counter to seven, then increase it by two |
| EOU, 160 ms, PB1000 | set the owned fixture counter to seven then increase it by two |
chunk=35 samples=46080 tokens=["ase", "▁it"] words="ase it" eouSignal=false anchor=nil confirmed=false
# chunks 36 to 39: tokens=[] eouSignal=false
chunk=40 samples=52480 tokens=[] words="" eouSignal=true anchor=52480 confirmed=true
chunk=41 samples=53760 tokens=[] words="" eouSignal=false anchor=52480 confirmed=false
chunk=42 samples=55040 tokens=[] words="" eouSignal=false anchor=52480 confirmed=false
chunk=43 samples=56320 tokens=["▁by"] words="by" eouSignal=false anchor=nil confirmed=false
chunk=44 samples=57600 tokens=["▁two"] words="two" eouSignal=false anchor=nil confirmed=false
chunk=45 samples=58880 tokens=["▁by", "▁two"] words="by two" eouSignal=false anchor=nil confirmed=false
chunk=46 samples=60160 tokens=[] words="" eouSignal=true anchor=60160 confirmed=false
In RnntDecoder.decodeWithEOU (lines 108 to 111), when the joint returns eouId the decoder sets eouDetected and runs break outerLoop. lastToken, hState and cState aren't updated for that step, and the chunk's remaining frames aren't decoded. processChunkAndDecode (StreamingEouAsrManager.swift lines 616 to 619) then appends each chunk's token IDs straight to accumulatedTokenIds, without the time-gated dedup from #788 or the fresh-state re-decode from #908. My inference, unconfirmed: the overlapping windows that follow re-present audio to a decoder whose state never recorded that the utterance closed.
Two simple fixes made it worse, which is why this is a question and not a patch:
- Committing
<EOU>like any other token (setlastToken,hStateandcState, continue with the next frame) changed no other clip, but both PB clips lost "by two" entirely. - Resetting the decoder state at the token (v0.15.7) also lost the continuation.
A seam dedup or fresh-state re-decode ported from the sliding-window TDT work may be the sturdier route, ideally decided together with finding 1.
3. The 160 ms tier substitutes short command words that 320 ms reads correctly
On 11 clean command clips, the two tiers and TDT v2 agree on 7. The other 4, on the M1 Max:
| Spoken | EOU, 160 ms | EOU, 320 ms | TDT v2 |
|---|---|---|---|
| Reply with exactly ready. | replied with exactly ready | reply with exactly ready | Reply with exactly ready |
| Set the owned fixture counter to seven. | set the owned fixture counter to set | set the owned fixture counter to seven | Set the owned fixture counter to seven. |
| Act on a window outside the owned fixture. | act on a window outside the own fixture | act on a window outside the owned fixture | Act on a window outside the owned fixture. |
| Cancel the turn before the effect executes. | cancel the term before the effect executes | cancel the turn before the effect executes | Cancel the turn before the effect executes. |
320 ms pays for that in end time: across the 11 clips the end token latched a median 218 ms after the labelled end of speech at 160 ms and 828 ms at 320 ms (debounce 0, measured on fed audio). The direction matches your own LibriSpeech numbers in Benchmarks.md (8.23% at 160 ms, 4.88% at 320 ms) against NVIDIA's 3.61% for the same checkpoint at 160 ms (model card).
The 160 ms tier feeds 17 mel frames per call but shifts 1280 samples (shiftSamples, lines 129 to 139, commented "50% overlap ... because the model was trained that way"), while decoding validOutputLen = 2 encoder frames per call. NVIDIA describes cache-aware streaming as "each audio frame is processed exactly once without overlap" (blog), and the same comment gives NeMo's shift_size as 16 mel frames, which is 2560 samples. I tried a 2560-sample shift with the current export and the output collapsed ("said the old fiction counter seven"), so the overlap is part of how this export was converted, and changing the shift alone is not a fix.
Questions: was the 160 ms export built to need the overlap? Would an export that matches NeMo's 160 ms chunk and shift close the gap to 3.61%? And given 8.23% against 4.88%, is 160 ms still the default you'd recommend?
4. The streaming VAD's speechEnd arrives one 256 ms block later than it could
VadManager+Streaming.swift lines 70 to 77 follow Silero's reference VADIterator exactly: the sample counter advances first, and the first quiet block sets tempEndSample = processedSamples, which is the end of that block. So this is a faithful port, not a slip. The difference is the block size. Silero's reference window is 512 samples (32 ms), so starting the silence clock at the end of the first quiet window costs 32 ms. FluidAudio's block is 4096 samples (256 ms), so the same rule costs 256 ms: a 0.6 s minSilenceDuration and the 0.75 s default both end speech after 1024 ms of silence. Starting the clock at the first quiet block's first sample would make both fire after 768 ms. The reported sampleIndex is already right, because line 77 subtracts a chunk. Only the decision is late.
This matters to anyone using the VAD as an end-of-turn ceiling, which is the backstop I'd recommend for the model's late endings (model notes below). A doc note would be enough. If you'd like the earlier anchor as an option, I can send it.
5. Small items
- The fused decoder path can't be tried from the published models.
FLUID_EOU_FUSED=1looks fordecoder_joint_decision_fused.mlmodelc(StreamingEouAsrManager.swiftline 353), which isn't in the160msor320msfolders on Hugging Face. - An end-of-turn score is already exported. The 160 ms
joint_decisionoutputstoken_prob,top_k_idsandtop_k_logits, andRnntDecoderreads onlytoken_id. A per-frame<EOU>score needs no new export. - One comment disagrees with the code. The
evaluateEouDebouncedoc comment says the debounce measures wall-clock silence (line 294). The code measures processed audio fromtotalSamplesProcessed(line 324). The two agree live and differ in offline replay. - Audio under 0.3 s throws
invalidAudioDatain the batch path. Inputs are padded to 240,000 samples anyway, so a short turn could be padded instead of rejected.
Supporting detail
Setup and corpus
- FluidAudio v0.17.5, commit
0b1f46289fe27d95b5e66ad8be46e64f5ee02ae7, exact SwiftPM pin. The streaming EOU and VAD sources are byte-identical to v0.15.7, where the earlier probes ran. - Models:
FluidInference/parakeet-realtime-eou-120m-coreml(160msand320ms),FluidInference/parakeet-tdt-0.6b-v2-coreml, and Silero VAD, all at default compute units unless stated. StreamingEouAsrManager(chunkSize: .ms160, eouDebounceMs: 0). The app applies its own confirmation delay.- Machines, never pooled: MacBook Pro M1 Max 64 GB on macOS 26.7 (all streaming replays), and MacBook Pro M2 Max 32 GB on macOS 27.0 (latency, probes, and a rerun of the 160 ms replay, which matched the M1 on 17 of 18 clips and every end time).
- Corpus: 18 short English command clips of synthetic speech, 16 kHz mono 16-bit, with 300 ms of digital silence before and after. Twelve are plain commands, and one of those turned out to contain a truncated extra syllable in its audio, so the findings use the other 11. Four P clips put 400, 700, 1000 or 1300 ms of digital silence after "...counter to" in "Set the owned fixture counter to seven." Two PB clips put 400 or 1000 ms after "...increase it" in "Set the owned fixture counter to seven, then increase it by two."
- Each clip is fed in 160 ms appends followed by 3 s of silence, with
reset()before each clip. The labelled end of speech is one past the last sample at or above 2% of the clip's peak.
Model behavior, for whoever trains the successor (not port bugs)
These belong to the checkpoint, not to your port. I'm including them because they shape what the port should expose.
- Late end tokens on complete sentences. "Check ToolPiper's version and active model." and "Answer while the configured model is unavailable." decode their last word pieces promptly, then 20 and 22 chunks pass with no tokens and
eouSignalfalse, and the first end token arrives in a word-free chunk.eouDetectedfirst reads true 1756 ms and 1598 ms after the labelled end of speech, against 117 to 358 ms for the other nine clean clips. At 320 ms they are 956 ms and 1438 ms. No earlier end token appears in either trace (under Repro), so this is the model's latency, and a VAD silence ceiling is what bounds it. - Early end tokens on incomplete sentences. The same model fires about 240 ms into the pause after "...counter to", a sentence that can't be complete. Its confidence runs opposite to completeness in exactly the cases conversation depends on.
- For context: NVIDIA's card reports an end-of-utterance latency p50 of 160 ms on TTS audio with 3 s of appended silence. My 218 ms median is in that range. The tails are what hurt.
What I'd ask of a successor, in order:
- Keep decoding after an end token. The token should be a signal, not a state change.
- An end-of-turn probability with an early and a final threshold, not only an argmax token.
- Turn-end confidence that follows sentence completeness.
- Batch-level accuracy on short commands at the lowest-latency tier, with punctuation and casing, so the second pass becomes optional.
- A published, exact streaming protocol (chunk, shift, cache sizes, and what to reset after
<EOU>), so ports can reproduce the reference numbers.
Two-pass latency: TDT v2 as the final pass
The design: streaming EOU for live partials and the candidate end, Silero VAD for the confirmation hold, re-arm and silence ceiling, and TDT v2 over the app-held turn audio, started at the first candidate end so its cost overlaps the confirmation delay. As recorded, TDT v2 read 10 of the 11 clean clips exactly, including every 160 ms substitution (all three readers hear "Mac's" as "Max" in the 11th), read PB400 with no repeat, and adds punctuation and casing.
Method: warm, in process, transcribe([Float], decoderState:) with a fresh TdtDecoderState per call, loaded with AsrModels.loadLocal(from:version: .v2, encoderComputeUnits:), 12 clips over 5 interleaved rounds after three warm-up calls. Cells give two separate runs.
| Milliseconds | M2 Max, ANE (default) | M2 Max, .cpuAndGPU |
M1 Max, ANE (default) | M1 Max, .cpuAndGPU |
|---|---|---|---|---|
| Clip as recorded, median / p90 (n 60) | 52.6 / 55.0, 52.7 / 54.7 | 59.9 / 62.1, 59.7 / 62.0 | 68.7 / 80.4, 67.8 / 70.9 | 74.5 / 84.4, 73.9 / 86.7 |
| With 768 ms of leading silence, median / p90 (n 60) | 52.5 / 55.3, 53.1 / 55.5 | 60.2 / 62.5, 59.9 / 62.9 | 68.9 / 81.7, 67.9 / 70.7 | 74.9 / 84.3, 74.6 / 86.8 |
| 0.86 s turn, median (n 5) | 47.2, 47.6 | 54.7, 54.5 | 60.8, 60.3 | 65.8, 66.2 |
| 0.25 s padded to 0.35 s, median (n 5) | 46.2, 46.6 | 54.3, 53.7 | 58.8, 59.7 | 63.9, 63.4 |
| First call after load | 74.4, 65.2 | 1891, 388 | 86.9, 78.6 | 1693, 273 |
| Load: first at a new path, then cached | 19467, 176 | 7618, 355 | 27075, 184 | 6315, 381 |
- For v2 on these machines the ANE beat
.cpuAndGPUby about 7 ms, the opposite of the v3 encoder numbers inEncoderComputePlacement.md, and the GPU's first call cost 0.27 to 1.9 s. - Cost is flat in turn length: a 0.35 s turn and a 4.5 s turn differ by about 7 ms on the M2, because every turn is padded to the 15 s window. A shorter-window TDT encoder for turn-length audio would be welcome if it is feasible.
- TDT v2's reading of a marginal word moves with leading silence. "outside the owned fixture" reads "owned" behind 0, 100, 400, 500, 900 and 1200 ms of leading digital silence and "own" behind 200, 300, 600, 700 and 768 ms, identically on both machines. For two-pass designs, the span handed to the final pass matters.
- On the M1 only, Core ML printed one line per process to stdout:
E5RT encountered an STL exception. msg = Failed to PropagateInputTensorShapes: std::runtime_error during type inference for ios17.slice_by_index: zero shape error.Results were unaffected.
Repro
The driver is a scratch SwiftPM package in tool/, next to a clone of FluidAudio in FluidAudio/ at 0b1f4628 with the one-line trace patch applied. It replays each clip, then 3 s of silence, through StreamingEouAsrManager (debounce 0) in 160 ms appends and prints one JSON line per clip. With EOU_TRACE=1 it also writes the per-chunk lines to stderr.
The clips aren't attached yet. The current set uses a voice licensed for non-commercial use, so I'm regenerating it in one you can keep in the repo, starting with the clips for findings 1 and 2.
Package.swift and main.swift
// swift-tools-version: 6.0
import PackageDescription
let package = Package(
name: "eou-trace",
platforms: [.macOS(.v14)],
dependencies: [.package(path: "../FluidAudio")],
targets: [.executableTarget(name: "eou-trace", dependencies: [.product(name: "FluidAudio", package: "FluidAudio")])]
)import AVFoundation
import FluidAudio
import Foundation
// eou-trace <160|320> <models-root> <clips-dir> <declarations.json> [clip ...]
let args = CommandLine.arguments
let chunk: StreamingChunkSize = args[1] == "320" ? .ms320 : .ms160
let modelsRoot = URL(fileURLWithPath: args[2])
let clipsDir = URL(fileURLWithPath: args[3])
let declarations = try JSONSerialization.jsonObject(with: Data(contentsOf: URL(fileURLWithPath: args[4]))) as! [String: Any]
let fixtures = (declarations["fixtures"] as! [[String: Any]])
let wanted = Set(args.dropFirst(5))
func samples(_ url: URL) throws -> [Float] {
let file = try AVAudioFile(forReading: url, commonFormat: .pcmFormatFloat32, interleaved: false)
precondition(file.processingFormat.sampleRate == 16000 && file.processingFormat.channelCount == 1, "clip must be 16 kHz mono")
let buffer = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: AVAudioFrameCount(file.length))!
try file.read(into: buffer)
return Array(UnsafeBufferPointer(start: buffer.floatChannelData![0], count: Int(buffer.frameLength)))
}
func pcm(_ values: ArraySlice<Float>) -> AVAudioPCMBuffer {
let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 16000, channels: 1, interleaved: false)!
let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(values.count))!
buffer.frameLength = AVAudioFrameCount(values.count)
values.withUnsafeBufferPointer { buffer.floatChannelData![0].update(from: $0.baseAddress!, count: values.count) }
return buffer
}
let manager = StreamingEouAsrManager(chunkSize: chunk, eouDebounceMs: 0)
if chunk == .ms160 {
try await manager.loadModels(from: modelsRoot.appendingPathComponent("parakeet-eou-streaming/160ms"))
} else {
try await manager.loadModels(to: modelsRoot)
}
let append = 2560
for fixture in fixtures {
let id = fixture["taskId"] as! String
if !wanted.isEmpty && !wanted.contains(id) { continue }
await manager.reset()
FileHandle.standardError.write(Data("TRACE clip=\(id) chunk=\(args[1])\n".utf8))
let audio = try samples(clipsDir.appendingPathComponent(fixture["path"] as! String)) + [Float](repeating: 0, count: 48000)
var fed = 0, latchedAt: Int?
while fed < audio.count {
let end = min(fed + append, audio.count)
_ = try await manager.process(audioBuffer: pcm(audio[fed..<end]))
fed = end
if latchedAt == nil, await manager.eouDetected { latchedAt = fed }
}
let last = fixture["lastSpeechSample"] as! Int
let row: [String: Any] = ["clip": id, "chunkMs": args[1], "transcript": await manager.getPartialTranscript(),
"label": fixture["spokenText"] as! String,
"eouLatchedAtSample": latchedAt.map { $0 as Any } ?? NSNull(),
"lagMs": latchedAt.map { ($0 - last) * 1000 / 16000 as Any } ?? NSNull()]
print(String(decoding: try JSONSerialization.data(withJSONObject: row, options: [.sortedKeys]), as: UTF8.self))
}Trace patch (StreamingEouAsrManager.swift)
--- a/Sources/FluidAudio/ASR/Parakeet/Streaming/EOU/StreamingEouAsrManager.swift
+++ b/Sources/FluidAudio/ASR/Parakeet/Streaming/EOU/StreamingEouAsrManager.swift
@@ -664,6 +664,11 @@ public actor StreamingEouAsrManager {
debounceMs: eouDebounceMs
)
eouFirstDetectedAt = decision.anchorSamples
+ if ProcessInfo.processInfo.environment["EOU_TRACE"] != nil {
+ let words = tokenizer.map { $0.decode(ids: decodeResult.tokenIds) } ?? ""
+ let raw = decodeResult.tokenIds.map { tokenizer?.rawToken(for: $0) ?? "<id:\($0)>" }
+ FileHandle.standardError.write(Data("TRACE chunk=\(processedChunks) samples=\(totalSamplesProcessed) tokens=\(raw) words=\"\(words)\" eouSignal=\(decodeResult.eouDetected) anchor=\(decision.anchorSamples.map(String.init) ?? "nil") confirmed=\(decision.confirmed)\n".utf8))
+ }
if decision.confirmed {
eouDetected = truegit clone https://github.com/FluidInference/FluidAudio && git -C FluidAudio checkout 0b1f46289fe27d95b5e66ad8be46e64f5ee02ae7
patch -p1 -d FluidAudio < trace.patch
cd tool && swift build -c release
# <models-root>/parakeet-eou-streaming/160ms holds the 160ms folder; 320 downloads on first use
EOU_TRACE=1 .build/release/eou-trace 160 <models-root> <clips-dir> <clips-dir>/declarations.json > out-160.json 2> out-160.trace
.build/release/eou-trace 320 <models-root> <clips-dir> <clips-dir>/declarations.json > out-320.json
declarations.json is {"fixtures": [{"taskId", "path", "spokenText", "lastSpeechSample"}]}.
Per-chunk excerpts, 160 ms: the two late endings (2.1)
"Check ToolPiper's version and active model." (labelled end of speech at sample 46135):
chunk=33 samples=43520 tokens=["▁mod"] words="mod" eouSignal=false anchor=nil confirmed=false
chunk=34 samples=44800 tokens=["el"] words="el" eouSignal=false anchor=nil confirmed=false
# chunks 35 to 54: tokens=[] eouSignal=false anchor=nil
chunk=55 samples=71680 tokens=[] words="" eouSignal=true anchor=71680 confirmed=true
chunk=56 samples=72960 tokens=[] words="" eouSignal=true anchor=71680 confirmed=false
"Answer while the configured model is unavailable." (labelled end of speech at sample 43539):
chunk=26 samples=34560 tokens=["▁un"] words="un" eouSignal=false anchor=nil confirmed=false
chunk=28 samples=37120 tokens=["av", "ail", "able"] words="available" eouSignal=false anchor=nil confirmed=false
# chunks 29 to 50: tokens=[] eouSignal=false anchor=nil
chunk=51 samples=66560 tokens=[] words="" eouSignal=true anchor=66560 confirmed=true
chunk=52 samples=67840 tokens=[] words="" eouSignal=true anchor=66560 confirmed=false
Thanks again for FluidAudio. If anyone in the community wants to pick one of these up, I'm glad to help. And if any of the design questions are easier on Discord, I'll meet you there.
Activity
Alex-Wengg commentedon Oct 18, 2026
for issue 1 i am thinking:
Add
rearm(keepTranscript:): rebuild encoder caches and decoder state, keep the transcript, drop the debounce anchor.Keep the audio shifted out since the anchor and replay it through the fresh state, so "seven" gets decoded instead of lost.
Optionally trigger that automatically when voiced audio decodes nothing for two chunks while the debounce window is still open.
or maybe extend the end utter detector to more than 1 seconds. we support this path
BenRacicot commentedon Oct 18, 2026
Ok will report back here after implementing. Is there any better format for feedback or help I can offer?