Skip to main content
Streaming delivers a run’s output as it is generated — token by token, straight to your end user’s browser over Server-Sent Events. It is built for chat experiences: the user sends a message, and the answer types itself out in real time. Streaming is a side channel. Runs execute, persist, and complete exactly the same whether anyone is watching or not, and the run record fetched from the API remains the source of truth. If a stream is interrupted, nothing about the run is affected — the protocol below tells the client precisely what it can trust and when to fall back to fetching the result.

​
How it fits together

  1. Your backend mints a stream credential for a thread — one call with any server SDK. The credential grants read access to that thread’s stream and nothing else; your API key never reaches the browser.
  2. Your frontend opens an SSE connection with that credential — most easily via @promptjuggler/browser, which handles the whole protocol.
  3. Your backend triggers runs on the thread as usual. Everything those runs stream — including every prompt node of a workflow — arrives on the connection, tagged by run and channel.

​
Getting a credential

POST /api/v1/threads/{thread}/stream-token (createStreamToken in every SDK) returns:
Hand the response to the browser verbatim — the url is the thread’s SSE endpoint, so the frontend needs no per-environment configuration. Tokens are valid for 24 hours; the browser SDK requests a fresh one on every reconnect, so expiry takes care of itself. The thread does not need to exist yet: mint the token first, connect, then trigger the first run against the same thread ID. Each plan includes a simultaneous streaming connection allowance (10× your concurrency). Connections beyond it are refused with 429 until one closes.

​
The event protocol

The stream is an ordered, replayed log. Every event carries an SSE id; a client that reconnects with Last-Event-ID resumes exactly where it left off, losing nothing across network blips and deploys. Events are JSON, keyed by runId and channel: Two rules keep clients honest:
  • Connect before you trigger. A fresh subscription starts at the live tip of the thread, not in the past. Open the stream first, then start the run, and you see every event from the beginning. A client that can only connect after triggering the run sets the browser SDK’s replay option, or sends Last-Event-ID: 0 from a raw SSE client, to replay the thread’s recent history.
  • stale means fall back. Replay covers reconnects within the retention window (minutes). Beyond it — or if events had to be shed under extreme load — the server says stale instead of leaving the client to guess. Runs that finish under a stale shadow are flagged, and the authoritative text is one API call away.
A tool_end carries no segment or seq. It is a status update rather than a position, found by its ref wherever the chip already sits — which is what lets an async tool (a prompt or workflow call, a knowledge search) resolve seconds after the model moved on. A tool that never ends stays pending, which is exactly what a call still running looks like. A chip is published where the model used the tool, not where its turn ended — including for the tools the provider runs inside its own turn, web search and remote MCP. So a model that keeps talking after calling something gives you two text blocks with the chip between them, live and on a refetch alike. The one thing a fetched run adds is a text block’s citations, which a provider attaches to the finished message rather than to the deltas it was streamed as; see Transcript. The browser SDK folds all of this — segments, resets, resume, staleness — into a single maintained text per run with a gapped flag that tells you the one thing you need to know: whether to fetch the run for the full result. Emit payloads ride the same stream as data events and are maintained the same way, as runs[runId].data (see emit tools). Tool events fold into runs[runId].transcript, the same ordered shape a fetched run returns — see Transcript.

​
Workflows and channels

Every prompt node in a workflow streams. Events are tagged with the node’s channel, so a workflow declares at design time which of its conversations are user-facing: put the user-visible prompt on a support channel, connect with ?channel=support (or channels: ['support'] in the browser SDK), and internal research or synthesis lanes never reach the browser.
Streaming requires API access, so it is available on all paid plans.