> ## Documentation Index
> Fetch the complete documentation index at: https://synthetiq.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Execution

> Worker service, job lifecycle, concurrency, and data storage

Workflows execute on a dedicated worker service, separate from the app service. The worker claims jobs, executes steps in dependency order, and manages checkpointing and data storage.

## Job lifecycle

| Status      | Description                                                                                            |
| ----------- | ------------------------------------------------------------------------------------------------------ |
| `PENDING`   | Job queued, waiting for a worker to claim it                                                           |
| `RUNNING`   | Worker is actively executing steps                                                                     |
| `PAUSED`    | Waiting for a [wait step](/docs/platform-docs/workflows-framework/steps/wait-for) or scheduled continuation |
| `RESUMING`  | Resuming from a checkpoint after a pause or recovery                                                   |
| `COMPLETED` | All steps finished successfully                                                                        |
| `FAILED`    | A step failed after exhausting retries                                                                 |
| `CANCELLED` | Manually cancelled                                                                                     |

## Worker architecture

### Job claiming

Workers claim jobs atomically from the queue using a claim epoch. If a job is reclaimed (e.g. after a heartbeat timeout), the original worker detects the epoch mismatch and stops processing. This prevents duplicate execution in multi-worker deployments.

### Heartbeats

Workers send heartbeats while processing jobs. If a worker stops sending heartbeats (crash, network failure) after the specified timeout, the job becomes eligible for another worker to claim and resume from the last checkpoint.

### Concurrency

Workflow execution has multiple levels of concurrency:

* **Jobs per worker** — each worker processes multiple jobs concurrently. Configured via the max concurrency setting in [workflow settings](/docs/platform-docs/deployments/configuration#workflow-settings).
* **Parallel steps per job** — independent steps within a job execute simultaneously. Configured via the max parallel steps setting in [workflow settings](/docs/platform-docs/deployments/configuration#workflow-settings).
* **forEach parallelism** — iterations within a forEach step can execute concurrently. Configured per step in the [forEach](/docs/platform-docs/workflows-framework/foreach) configuration.

## Guarantees

| Guarantee               | Description                                                                                                                                                                                                                                                                                                 |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| At-least-once execution | Each step executes at least once. If a worker crashes after executing a step but before checkpointing the result, the step may re-execute on recovery. Use [idempotency patterns](/docs/platform-docs/workflows-framework/error-handling#exactly-once-side-effects) for operations with real-world consequences. |
| No concurrent execution | The claim epoch mechanism prevents multiple workers from making progress on the same job simultaneously.                                                                                                                                                                                                    |
| Resumable progress      | Checkpointing at step, page, and item levels means recovery continues from where it left off, not from the beginning.                                                                                                                                                                                       |
| Data isolation          | Each job run uses a unique table prefix, preventing cross-job data interference.                                                                                                                                                                                                                            |

## Data processing & storage

### DuckDB

Workflow steps execute on [DuckDB](https://duckdb.org/), a high-performance columnar database. DuckDB runs in-process on the worker and handles SQL transformations, bulk data transfer to and from the app database, and intermediate storage operations. No external database connection is required for data processing. Step resource limits scale with worker configuration — see [configuration](/docs/platform-docs/deployments/configuration#step-resource-limits) for details.

### Step results

Step outputs are stored as [Parquet](https://parquet.apache.org/) files in the app's configured workflow storage (i.e. S3 on AWS) or on the local filesystem during development. Each job run's data is logically isolated from other concurrent runs. Step results are available for querying and debugging, and are automatically cleaned up after the configured TTL (default 48 hours).

## Monitoring

Track workflow execution through the platform's monitoring tools:

* **Job status**: Current state, start time, duration, step progress
* **Step details**: Per-step execution time, row counts, errors
* **Logs**: Structured logs for each step execution
* **History**: Past runs with outcomes and timing

See [workflow monitoring](/docs/platform-docs/deployments/monitoring/workflows) for details.