Run lifecycle

Every state a run can occupy, what moves it, and which endings are retryable.

A run moves through a small, explicit state machine. Nothing is inferred from absence — a run that stopped has a state and a category saying why.

States

queued ──▶ claimed ──▶ running ──┬──▶ succeeded
                                 ├──▶ failed
                                 ├──▶ timed_out
                                 └──▶ cancelled
StateMeaning
queuedWritten to the durable queue, waiting for a worker.
claimedA worker has taken the run atomically. No other worker can take it.
runningExecution has started; startedAt is set.
succeededThe model finished within its limits and produced a result.
failedA terminal error occurred. error.category says which.
timed_outThe run exceeded the routine's timeoutSeconds deadline.
cancelledExecution was aborted externally rather than by its own deadline.

succeeded, failed, timed_out, and cancelled are terminal. Poll for membership of that set rather than for succeeded alone.

Error categories

Every non-successful ending carries a category. Categories marked retryable are reported as retryable: true in webhook payloads.

CategoryCauseRetryable
authenticationA credential was rejected.No
configurationThe routine or connection is not runnable as configured.No
invalid_modelThe model identifier is not in the current directory.No
gateway_rate_limitLLMGateway rate limited the request.Yes
gateway_timeoutThe gateway did not respond in time.Yes
gateway_errorThe gateway returned an error.Yes
mcp_discoveryTool discovery against the MCP server failed.No
mcp_transportAn MCP call failed in transport mid-run.Yes
tool_deniedThe model asked for a tool it is not permitted to call.No
tool_errorAn approved tool ran and returned an error.No
step_limitThe run reached maxSteps without completing.No
timeoutThe run exceeded timeoutSeconds.Yes
cancelledThe run was aborted externally.No
internalAn unexpected failure inside the service.No

Retryable describes the category, not an automatic retry

A failed run is not re-executed on its own. retryable tells your systems whether trying the same work again is reasonable. An active routine simply runs again at its next scheduled occurrence.

Concurrency and claiming

Runs are claimed atomically from a durable Postgres queue with bounded worker concurrency. Overlapping scheduler ticks cannot execute the same run twice, and a worker that dies mid-run does not strand the queue — the claim expires and the run becomes available again.

The Node process runs a polling worker; on Ploy, a one-minute cron trigger drives the same operations tick, and a manually dispatched run executes on the request's waitUntil.

What happens after a terminal state

  • The result and usage are persisted on the run.
  • If the routine has a webhook URL, a delivery is written and then attempted.
  • For an active routine, the next occurrence is already computed; a failure does not pause the schedule.

On this page

Edit this page on GitHub