Agentic Chat Systems
Production chat assistants that do work instead of just answering: tool-calling over live business data, governed writes with one-click confirmation and audit trails, and documents generated inside the conversation. Built for Salesbugle and a climate-tech intelligence platform.
Architecture
- Bounded tool-calling loop over live business data: deals, stakeholders, meetings, products, and call intelligence, streamed over SSE
- A capability registry discovers backend API routes and exposes them as chat tools automatically, with writes withheld until explicitly enabled
- Governed write path: the assistant proposes an action, the user confirms in one click, the action executes and lands in an audit trail
- In-chat artifacts: server-rendered documents and slide decks with a full-screen workspace, inline editing, and PPTX/PDF export
- Knowledge-grounded coaching: objection rebuttals grounded in the org's playbook and each deal's product value drivers, not generic LLM advice
- For the climate-tech platform: an assistant grounded in live sensor telemetry, alert episodes, and household context, streaming two-layer answers (a plain answer first, technical detail on demand)
- Multi-model routing: a fast primary model, an automatic fallback, and a heavyweight model reserved for deep analysis
Key Decisions
Writes require proposal, confirmation, and audit
Why: A chat that can touch a CRM earns trust through governance, not intelligence
Tradeoff: One extra click per action, which users accept because nothing surprising ever happens
Auto-discover tools from the API instead of hand-writing each one
Why: Every new backend endpoint becomes a chat capability without a second implementation to maintain
Tradeoff: Curated tools still shadow the common writes, so the registry needs deliberate curation rules
Two-layer streamed answers for mixed audiences
Why: A homeowner and an engineer need different depth from the same reply; the model writes both in one pass
Tradeoff: The split marker must parse correctly mid-stream so nothing flickers or reflows
Technologies
What I Learned
- Anyone can build a chatbot now. The moat is the tool surface, the grounding, and the write governance around it.
- Propose-confirm-audit is what makes an assistant trustworthy enough to act on real records.
- Streaming UX details (marker splitting, no reflow when the stream ends) are what make an assistant feel finished.