> ## Documentation Index
> Fetch the complete documentation index at: https://synthetiq.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Error handling

> Retries, checkpointing, idempotency, and recovery strategies

The workflow engine provides automatic retries, checkpointing for recovery, and patterns for ensuring exactly-once execution of side effects.

## Retries

### Step-level retries

Service steps retry automatically on failure with exponential backoff:

```typescript theme={null}
{
  service: "flaky-api",
  operation: "fetchData",
  params: { ... },
  outputTable: "results",
  outputSchema: { ... },
  maxRetries: 5,
  retryDelayMs: 2000,
}
```

| Field          | Default | Description                                        |
| -------------- | ------- | -------------------------------------------------- |
| `maxRetries`   | `3`     | Number of retry attempts                           |
| `retryDelayMs` | `1000`  | Base delay in milliseconds (doubles on each retry) |

### forEach item retries

When using `forEach`, retries apply per item:

```typescript theme={null}
forEach: {
  source: "records",
  itemKey: "id",
  maxRetries: 3,
  retryDelayMs: 1000,
  onItemError: "skip",
}
```

With `onItemError: "skip"`, failed items are logged and skipped after exhausting retries. With `"fail"` (default), the first item failure after retries stops the entire step.

### Pagination retries

Each page request can retry independently:

```typescript theme={null}
paginate: {
  type: "cursor",
  cursorField: "nextCursor",
  cursorParam: "cursor",
  maxRetries: 5,
  retryDelayMs: 2000,
}
```

## Checkpointing

The workflow engine checkpoints progress at multiple levels. If a worker crashes or the job is interrupted, it resumes from where it left off:

* **Step-level** — completed steps are not re-executed
* **Page-level** — paginated steps resume after the last fully completed page
* **Item-level** — forEach steps skip already-completed items

This means long-running workflows are resilient to transient failures — a paginated fetch through thousands of pages or a forEach over hundreds of items will pick up where it stopped, not start over.

## Idempotency

Prevent duplicate active jobs for the same logical operation using an idempotency key:

```typescript theme={null}
await client.submitJob({
  job: syncOrders,
  params: { orgId: "org_abc123" },
  idempotencyKey: "app--my-app:sync-orders",
});
```

If a job with the same `idempotencyKey` is already active (pending or running), the submission is rejected. This is especially useful for manual triggers and webhook-initiated workflows.

## Exactly-once side effects

For operations with real-world consequences (payments, notifications), ensure they execute exactly once even across retries.

### Native idempotency keys

If the target API supports idempotency keys, pass a deterministic key derived from the data:

```typescript theme={null}
{
  service: "stripe",
  operation: "createCharge",
  params: {
    amount: "{{item.amount}}",
    currency: "usd",
    idempotency_key: "charge-{{item.order_id}}",
  },
  forEach: {
    source: "pending_charges",
    itemKey: "order_id",
  },
  outputTable: "charges",
  outputSchema: { ... },
}
```

### preExecute checks

For APIs without native idempotency, use `preExecute` to check whether the operation has already been performed:

```typescript theme={null}
{
  service: "legacy_payment",
  operation: "create_charge",
  params: {
    order_id: "{{item.order_id}}",
    amount: "{{item.amount}}",
  },
  preExecute: {
    service: "legacy_payment",
    operation: "list_charges",
    params: { order_id: "{{item.order_id}}" },
    condition: { path: "results.length", gt: 0 },
    resultMapping: {
      charge_id: "results[0].id",
      recovered: true,
    },
  },
  forEach: {
    source: "pending_orders",
    itemKey: "order_id",
  },
  outputTable: "charges",
  outputSchema: { ... },
}
```

If `preExecute` finds an existing result matching the condition, the main operation is skipped and the mapped values are used as the step output instead.