Skip to main content
Streaming delivers a run’s output as it is generated — token by token, straight to your end user’s browser over Server-Sent Events. It is built for chat experiences: the user sends a message, and the answer types itself out in real time. Streaming is a side channel. Runs execute, persist, and complete exactly the same whether anyone is watching or not, and the run record fetched from the API remains the source of truth. If a stream is interrupted, nothing about the run is affected — the protocol below tells the client precisely what it can trust and when to fall back to fetching the result.

​
How it fits together

  1. Your backend mints a stream credential for a thread — one call with any server SDK. The credential grants read access to that thread’s stream and nothing else; your API key never reaches the browser.
  2. Your frontend opens an SSE connection with that credential — most easily via @promptjuggler/browser, which handles the whole protocol.
  3. Your backend triggers runs on the thread as usual. Everything those runs stream — including every prompt node of a workflow — arrives on the connection, tagged by run and channel.

​
Getting a credential

POST /api/v1/threads/{thread}/stream-token (createStreamToken in every SDK) returns:
Hand the response to the browser verbatim — the url is the thread’s SSE endpoint, so the frontend needs no per-environment configuration. Tokens are valid for 24 hours; the browser SDK requests a fresh one on every reconnect, so expiry takes care of itself. The thread does not need to exist yet: mint the token first, connect, then trigger the first run against the same thread ID. Each plan includes a simultaneous streaming connection allowance (10× your concurrency). Connections beyond it are refused with 429 until one closes.

​
The event protocol

The stream is an ordered, replayed log. Every event carries an SSE id; a client that reconnects with Last-Event-ID resumes exactly where it left off, losing nothing across network blips and deploys. Events are JSON, keyed by runId and channel: Two rules keep clients honest:
  • Connect before you trigger. A fresh subscription starts at the live tip of the thread, not in the past. Open the stream first, then start the run, and you see every event from the beginning.
  • stale means fall back. Replay covers reconnects within the retention window (minutes). Beyond it — or if events had to be shed under extreme load — the server says stale instead of leaving the client to guess. Runs that finish under a stale shadow are flagged, and the authoritative text is one API call away.
The browser SDK folds all of this — segments, resets, resume, staleness — into a single maintained text per run with a gapped flag that tells you the one thing you need to know: whether to fetch the run for the full result.

​
Workflows and channels

Every prompt node in a workflow streams. Events are tagged with the node’s channel, so a workflow declares at design time which of its conversations are user-facing: put the user-visible prompt on a support channel, connect with ?channel=support (or channels: ['support'] in the browser SDK), and internal research or synthesis lanes never reach the browser.
Streaming requires API access, so it is available on all paid plans.