Skip to main content
Version: Next

Changes Refresh Mode

The changes refresh mode applies incremental inserts, updates, and deletes from a Change Data Capture (CDC) source. Unlike append, changes mode reflects modifications and deletions in the acceleration, keeping it consistent with sources where rows mutate over time.

Use changes when:

  • The source supports CDC (e.g., a database with a transaction log).
  • Rows in the source are updated or deleted, not just inserted.
  • The acceleration must reflect the current state of the source row-for-row.

Configuration​

refresh_mode: changes requires a CDC-capable data connector. Spice supports CDC via PostgreSQL Logical Replication, MySQL Binlog Replication, MongoDB Change Streams, DynamoDB Streams, and Debezium (over Kafka). See Supported Data Connectors for details.

note

Apache Kafka is a real-time streaming source but is append-only — it uses refresh_mode: append, not changes.

Any accelerator engine that supports writes can be a changes sink — arrow, duckdb, sqlite, and cayenne. Spice Cayenne is recommended for large-scale CDC (incremental materialized views, in-memory CDC tier, and replication-lag/freshness SLOs).

datasets:
  - from: debezium:cdc.public.customer_orders
    name: customer_orders
    acceleration:
      enabled: true
      refresh_mode: changes
      engine: duckdb
      mode: file

The Debezium connector streams change events from a Kafka topic produced by Debezium. Each event is applied to the acceleration in order, preserving inserts, updates, and deletes from the source.

Behavior​

  • The acceleration is bootstrapped from the source snapshot, then continuously updated from the change stream.
  • refresh_check_interval, refresh_cron, on-demand refresh, refresh_data_window, and retention_period do not apply — updates are driven by the change stream rather than periodic polling.
  • refresh_sql can only modify selected columns in changes mode and cannot apply row filters.