Scheduled Iceberg Table Maintenance with Trino

Iceberg tables degrade quietly. Streaming and micro-batch writes pile up small files that slow every scan, and each write adds a snapshot that keeps old data files alive on storage. This blueprint runs the two standard countermeasures through io.kestra.plugin.jdbc.trino.Query on a weekly schedule: ALTER TABLE ... EXECUTE optimize to compact small files, and ALTER TABLE ... EXECUTE expire_snapshots to drop snapshots older than seven days. Both statements run with fetchType: NONE since they return no result set, and Discord confirms every run.

How it works

1. optimize_table (io.kestra.plugin.jdbc.trino.Query, fetchType: NONE) runs ALTER TABLE iceberg.analytics.events EXECUTE optimize, rewriting small data files into larger ones sized for efficient scans.
2. expire_snapshots runs ALTER TABLE iceberg.analytics.events EXECUTE expire_snapshots(retention_threshold => '7d'), removing snapshot metadata and the data files only those snapshots reference. Time travel remains available within the seven-day window.
3. Each maintenance action is its own task with exactly one statement, so a failure in snapshot expiry never hides a failure in compaction, and each step is retryable on its own.
4. notify posts a Discord confirmation naming the table, and the errors block posts a distinct alert when either step fails.
5. A disabled-by-default Schedule trigger runs the flow every Sunday at 03:00, outside business hours.

What you get

• Consistent read performance, because small files from streaming writes are compacted weekly.
• Bounded storage growth, because snapshots and their orphaned data files expire on a fixed retention.
• Independent, observable maintenance steps with per-task logs and retries.
• A Discord audit trail confirming when maintenance actually ran, and an alert when it did not.

Who it's for

• Lakehouse teams running Iceberg behind Trino who currently run OPTIMIZE by hand, or not at all.
• Data platform engineers formalizing day 2 operations for tables fed by streaming ingestion.
• Anyone whose Trino queries got slower every month without the data getting bigger.

Why orchestrate this with Kestra

Maintenance SQL is easy to write and easy to forget. Kestra turns it into an operated process: a schedule that survives restarts, per-step execution history showing exactly when each table was last compacted, retries for transient cluster errors, and notifications on both success and failure. Adding the next table is a copy of two tasks, not a new cron entry on a forgotten VM.

Prerequisites

• A Trino cluster with an iceberg catalog and a table matching iceberg.analytics.events, or adjust the table name in both statements.
• Sufficient permissions for the Trino user to execute table procedures on that table.
• A Discord incoming webhook for confirmations and alerts.

Secrets

• TRINO_URL: JDBC URL, e.g. jdbc:trino://host:443/iceberg/analytics (use https on 443 in production; password authentication requires TLS).
• TRINO_USERNAME: Trino username.
• TRINO_PASSWORD: Trino password.
• DISCORD_WEBHOOK_URL: Discord incoming webhook URL.

Quick start

1. Add the four secrets to your Kestra namespace.
2. Replace iceberg.analytics.events with your catalog, schema, and table in both statements.
3. Execute the flow once manually and confirm the Discord message arrives.
4. Set disabled: false on the weekly trigger.

How to extend

• Add remove_orphan_files as a third task for tables where failed writes leave unreferenced files.
• Duplicate the two tasks per table, or loop over a table list with io.kestra.plugin.core.flow.Loop.
• Tune retention_threshold per table to match your time travel and rollback requirements.
• Add a WHERE clause to optimize to compact only recent partitions on very large tables.

Links

• Trino plugin documentation
• Trino Iceberg connector documentation