> ## Documentation Index > Fetch the complete documentation index at: https://docs.promptjuggler.com/llms.txt > Use this file to discover all available pages before exploring further. # Reliability > What happens when a provider wobbles, a network blips, or we ship a release mid-run. Everything between your request and the model's answer can fail: providers rate-limit and time out, networks drop, tool endpoints go down, and we deploy new code while your runs are in flight. Most of that is transient, and transient trouble should not become your problem. So PromptJuggler absorbs it. A run either **completes**, or it **ends with a recorded reason** — never silently, never half-applied, and never because a release happened to land while it was executing. This page describes what the platform does on your behalf, and — at the end — what it deliberately doesn't. ## Two kinds of failure Every failure gets classified before anything else happens, because the right response depends entirely on what kind it is. Retrying a malformed request is pointless; giving up on a 503 is wasteful. | Kind | Examples | What we do | | ------------- | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------- | | **Transient** | Rate limits, provider 5xx and overload, timeouts, torn streams, a tool endpoint mid-deploy | Back off and retry, with the run left intact | | **Terminal** | Malformed request, unknown model, invalid credentials, hard quota exhaustion | Stop immediately and record why — retrying cannot change the answer | | **Ours** | A worker being shut down for a release | Hand the work back to the queue so another worker picks it up from the start | ## Retries Retries happen at two altitudes, independently. **Inside a call**, the runner retries the provider itself. Delays grow exponentially and carry random jitter, so a provider recovering from an incident doesn't get a synchronised stampede from every worker at once. **Around the run**, a run that fails transiently is re-dispatched as a whole — a fresh attempt against the model, from the beginning of that step. The run-level ladder is **bounded**: attempts are finite, and when they are spent the run is marked failed with the error that stopped it. Provider-level backoff deliberately isn't — while a provider is rate-limiting or unavailable, the run waits for it to come back rather than being failed over an outage that was never yours, each wait longer than the last up to a ceiling. Either way, nothing retries silently: the attempt count and the last error are always on the run. A retry generates tokens again, and generated tokens are billed. Retries protect your run's outcome, not your bill — a run that succeeds on its third attempt has paid for three. ## Circuit breaking When a provider is genuinely unwell, continuing to call it wastes time and money and slows its recovery. A circuit breaker watches for sustained failure against each of your provider API keys, and when it trips, further calls on that key fail fast for a short cooling-off period instead of piling on. It then lets traffic through again on its own — if the provider has recovered, everything resumes; if it hasn't, the breaker re-trips. Nobody has to intervene, and there is no stuck-open state waiting on a human. The breaker is shared across every worker, so it reflects what the whole fleet is seeing rather than one process's local bad luck. The part worth knowing is what the breaker refuses to count. **Only failures that are actually evidence about the provider's health move it.** A rejected request means the provider answered — it was up, and it said no. A remote MCP server timing out is the provider *telling you* that someone else's server is down. Counting either would take a healthy provider offline for all of your runs sharing that key, over an outage it had no part in. ## Rate limits, learned rather than guessed Request and token budgets are tracked centrally and shared across all workers, so your concurrency is a real fleet-wide number rather than a per-process guess that collectively overshoots. Those budgets are also **read back from the provider on every call** — including on a 429. Providers adjust limits per account and over time, so the limiter tracks what a provider currently says rather than what was true when the model was added. When you're at the limit, work queues and waits instead of hammering the API into a longer backoff — and you choose the order it waits in, since every run takes a `priority`: an interactive request takes the next free slot ahead of the batch job you queued that morning. ## Retries can't double-apply Retrying is only safe if a duplicate can't do damage — and in any queue-based system, duplicate and late deliveries are a matter of *when*, not *if*. A worker can finish a job and lose the connection before its answer lands; the job runs again; both answers eventually arrive. Every response carries the attempt it belongs to, and a run only accepts a result while it is still waiting for one. Once a run has completed or failed, later deliveries are **ignored, not applied**. Concurrent writes to the same run are serialised, so two handlers racing on the same record cannot interleave into a corrupted state. The practical consequence: a retry never double-appends a message to a thread, never double-counts token usage, and never resurrects a run you already saw finish. ## Deploys don't fail your runs When a worker is asked to shut down — a release, a scale-down, a node being recycled — it stops accepting new work and keeps executing what it already has. Runs that finish inside the grace window complete normally. Anything still executing when the window closes is **returned to the queue rather than failed**, and another worker starts it over. From your side, a deploy looks like nothing at all. ## Failure stays contained Failures are scoped as tightly as the work they belong to — as, on paid plans, is the capacity underneath them: your runs execute on queues and workers dedicated to your tenant, so nobody else's backlog is ever in front of yours. **Within a workflow**, each node retries on its own ladder. A node having a bad time retries in place — it does not restart the workflow, re-run completed nodes, or disturb branches executing beside it. Only if that node exhausts its own retries does the workflow fail, and then it fails as a whole rather than limping onward: no half-finished workflow ever reports a result assembled from the nodes that happened to succeed. **Within a prompt run**, a failed tool call doesn't have to be fatal. Each tool has a [fail-fast toggle](/tools/overview#attaching-tools): leave it off and the error is handed back to the model as the tool's result, so it can try another approach or explain the problem in its answer. Turn it on for tools where a wrong answer is worse than no answer. **Streaming is a side channel.** Runs execute, persist, and complete identically whether anyone is watching. If the streaming infrastructure has a problem, streaming degrades and the run is untouched — and a dropped connection resumes exactly where it left off, since [the stream is a replayable log](/concepts/streaming), not a live firehose. A run's record is always the source of truth. **Webhooks** that fail are retried on their own schedule, independently of the run that triggered them. A webhook endpoint being down never affects the run's outcome. ## When a run does fail, you can see why Failure is data, not a dead end. * The run's status and error are on the run record — one fetch, no log-diving. * The **partial output of the failed attempt is kept** alongside it. When a run dies mid-generation, what the model had produced by then is usually the fastest route to understanding why. * The [transcript](/concepts/transcript) shows which tools were called, in order, and which of them errored — so a failure caused by a tool is visibly a tool failure. * Clients streaming a run receive an explicit signal when a retry re-generates text, so a reader never sees two overlapping attempts spliced together. ## What we don't promise Robustness claims are worth exactly as much as the limits stated alongside them. * **Delivery is at-least-once, not exactly-once.** We make duplicates harmless on our side. Your own webhook handlers should be idempotent too — a delivery can arrive twice. * **Terminal errors are not retried.** An invalid request, a bad key, or exhausted quota fails immediately and stays failed. That is the correct behaviour, but it does mean some failures reach you on the first attempt. * **Retries cost tokens.** See the note above. * **Streams are ephemeral.** The replay buffer behind a resumable stream has bounded retention. Reconnect promptly and lose nothing; come back much later and fetch the run instead — which is why the run record, not the stream, is the source of truth. * **A model's answer is still a model's answer.** None of this makes an LLM deterministic or correct. For that, use [evaluations](/concepts/evaluations). * **The last hop is yours.** Everything on this page begins once a request reaches us. The network between your code and our API is the one stretch we can't make reliable for you, so if a call does fail, retry it — a short retry with backoff around your PromptJuggler calls, the same as you would give any remote service.