Skip to main content
Transform data using SQL queries. SQL steps read from tables produced by previous steps and produce new output tables:
{
  sql: `
    SELECT
      o.id,
      o.order_number,
      CAST(o.total_price AS DOUBLE) AS revenue,
      o.created_at,
      c.email AS customer_email
    FROM raw_orders o
    LEFT JOIN raw_customers c ON o.customer_id = c.id
    WHERE o.created_at >= '{{var.startDate}}'
  `,
  outputTable: "enriched_orders",
  outputSchema: {
    id: "VARCHAR",
    order_number: "INTEGER",
    revenue: "DOUBLE",
    created_at: "VARCHAR",
    customer_email: "VARCHAR",
  },
}
FieldRequiredDescription
sqlYesDuckDB SQL query (SELECT statements and CTEs only)
outputTableYesName of the table this step produces
outputSchemaYesColumn names and types for the result output table

SQL safety

  • Only SELECT statements and CTEs (WITH ... AS) are allowed
  • Mutations (i.e. INSERT, UPDATE, DELETE) are blocked
  • Variables are parameterized to prevent SQL injection
  • Tables are isolated per job — one job cannot access the results of any other job

Validation

Check
Referenced tables exist in previous steps
SQL syntax is valid
Column references are validated against source table schemas