Real-Time Meeting Intelligence
The live-meeting stack built for Salesbugle: an AI participant that joins real video calls, presents a slide deck as its camera tile, hears the room, answers grounded questions when addressed by name, and resumes where it left off. On top of a real-time transcription and diarization pipeline.
Architecture
- An AI bot joins the meeting as a real participant: it renders the prepared deck in its own camera tile, narrates it, and streams meeting audio back in
- Wake-word conversation flow: the presenter narrates, yields when a participant speaks, answers grounded questions when addressed by name, and resumes at the right point without talking over anyone
- One ordered audio queue shared by narration, clarifications, and answers; a state change invalidates queued speech so nothing stale ever plays
- A second lane captures meeting audio directly from the browser for operator-present sessions, sharing the same voice runtime with no bot in the room
- Real-time ASR pipeline underneath: streaming capture, VAD, sliding windows with overlap, Whisper transcription, speaker diarization, and four-layer hallucination filtering
- Full recording lifecycle handled: consent and bot identity, recording, transcript import, deal association, and retention
Key Decisions
A real meeting participant, not a screen-share hack
Why: An autonomous presenter has to show media as itself with no operator in the room, which only a first-class participant can do
Tradeoff: The full media pipeline (deterministic video frames plus ordered audio) had to be engineered, not borrowed
Conversation discipline as explicit state, not prompt hints
Why: Never narrating over a participant and never double-answering requires race guards across bot callbacks, transcript events, and audio completion
Tradeoff: More state machine to own, but the alternative is an AI that interrupts people
Four-layer hallucination filtering on the ASR pipeline
Why: Whisper hallucinates aggressively on silence and background noise, so single-layer filtering is insufficient
Tradeoff: Added latency from multi-pass processing, but accuracy gains justify it
Technologies
What I Learned
- Live rehearsals beat demos: every real meeting run surfaced a genuine fault that got fixed before the stakes were real.
- Delivery manner is a feature. Pauses, follow-up offers, and spoken admissions decide whether an AI presenter feels human or robotic.
- Real-time transcription needs context: sliding windows with overlap beat fixed chunks, and recent transcript segments as prompts lift accuracy on domain terms.