Transform Aggregate

Transform Aggregate

Certified

Aggregate records by group

Group records by one or more fields and compute typed summary values such as counts, sums, minimums, maximums, first, or last values.

yaml
type: io.kestra.plugin.transform.Aggregate

Aggregate totals

yaml
id: aggregate_totals_records
namespace: company.team

tasks:
  - id: normalize
    type: io.kestra.plugin.core.output.OutputValues
    values:
      records:
        - customer_id: c1
          country: FR
          total_spent: 10
        - customer_id: c1
          country: FR
          total_spent: 5

  - id: aggregate
    type: io.kestra.plugin.transform.Aggregate
    from: "{{ outputs.normalize.values.records }}"
    groupBy:
      - customer_id
      - country
    aggregates:
      order_count:
        expr: count()
        type: INT
      total_spent:
        expr: sum(total_spent)
        type: DECIMAL
    onError: FAIL

Aggregate with stored output

yaml
id: aggregate_totals
namespace: company.team

tasks:
  - id: fetch
    type: io.kestra.plugin.core.output.OutputValues
    values:
      records:
        - customer_id: "c1"
          country: "FR"
          total_spent: 10
        - customer_id: "c1"
          country: "FR"
          total_spent: 5

  - id: aggregate
    type: io.kestra.plugin.transform.Aggregate
    from: "{{ outputs.fetch.values.records }}"
    outputType: STORE
    groupBy:
      - customer_id
      - country
    aggregates:
      order_count:
        expr: count()
        type: INT
      total_spent:
        expr: sum(total_spent)
        type: DECIMAL
Properties

Aggregate definitions

Output fields to compute for each group. Each value can be a shorthand expression string or an object with expr and optional type.

Input records

Ion list or struct to transform, or a storage URI pointing to an Ion file.

SubTypestring

Group by

Fields to group on.

DefaultFAIL
Possible Values
FAILSKIPNULL

On error behavior

FAIL stops the task on aggregate errors, SKIP drops failing input records, and NULL sets failing aggregate outputs to null.

DefaultTEXT
Possible Values
TEXTBINARY

Output format

Experimental: TEXT or BINARY. Only transform tasks can read binary Ion. Use TEXT as the final step.

DefaultAUTO
Possible Values
AUTORECORDSSTORE

Output type

AUTO stores to internal storage when the input is a storage URI; otherwise it returns records.

Reference (ref) of the pluginDefaults to apply to this task.

Aggregated records

JSON-safe records when output mode is RECORDS or AUTO resolves to RECORDS.

Stored Ion file URI

URI to the stored Ion file when output mode is STORE or AUTO resolves to STORE.

More Plugins in this Category

Core Plugins and tasks
Core Plugins and tasks

Tasks that provide Kestra's built-in orchestration, I/O, and observability capabilities.

Core

122 tasks

398 blueprints

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks