AssetShipper | Kestra

AssetShipper

Forward asset lineage to one or more desired destinations.

yaml
type: io.kestra.plugin.ee.assets.AssetShipper

Examples

yaml
id: assetShipper
namespace: company.team

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

tasks:
  - id: logSync
    type: io.kestra.plugin.ee.assets.AssetShipper
    assetExporters:
      - id: file
        type: io.kestra.plugin.ee.openlineage.OpenLineageAssetExporter
        uri: http://localhost:3000DO

Properties

assetExporters *

Min items1

Definitions

Ship asset lineage events to a file inside Kestra's internal storage.

Example

yaml
id: assetShipper
namespace: company.team

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

tasks:
  - id: logSync
    type: io.kestra.plugin.ee.assets.AssetShipper
    assetExporters:
      - id: file
        type: io.kestra.plugin.ee.assets.FileAssetExporter
        format: JSON
        maxLinesPerFile: 100

id*string

Validation RegExp^[a-zA-Z0-9][a-zA-Z0-9_-]*

Min length1

type*object

formatstring

DefaultION

Possible Values

IONJSON

logFilePrefixstring

Defaultkestra-log-file

maxLinesPerFileintegerstring

Export asset lineage to OpenLineage

Example

yaml
id: assetShipper
namespace: company.team

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

tasks:
  - id: logSync
    type: io.kestra.plugin.ee.assets.AssetShipper
    assetExporters:
      - id: file
        type: io.kestra.plugin.ee.openlineage.OpenLineageAssetExporter
        uri: http://localhost:3000

yaml
id: assetShipper
namespace: company.team

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

tasks:
  - id: logSync
    type: io.kestra.plugin.ee.assets.AssetShipper
    assetExporters:
      - id: file
        type: io.kestra.plugin.ee.openlineage.OpenLineageAssetExporter
        uri: http://localhost:3000
        mappings:
          io.kestra.plugin.ee.assets.Table:
            namespace: namespace
            name: name
          io.kestra.plugin.ee.assets.Dataset:
            namespace: namespace
            name: name

id*string

Validation RegExp^[a-zA-Z0-9][a-zA-Z0-9_-]*

Min length1

type*object

uri*string

mappingsobject

SubTypeobject

assetNamespace string

assetTypes array

SubTypestring

delete booleanstring

flowId string

flowNamespace string

lookbackPeriod string

DefaultP1D

offsetKey string

taskId string

Outputs

endFetchedDate string

Formatdate-time

outputs object

Definitions

io.kestra.core.models.tasks.Output

size integer

Register S3 JSON files as governed datasets and curate them with DuckDB

Execute R scripts as part of Kestra data workflows.

Kestra Plugin Transformation for records.

Execute Bun scripts as part of Kestra data workflows.

AssetShipper

Forward asset lineage to one or more desired destinations.

Create automations with Core Asset Ee AssetShipper

Register S3 JSON files as governed datasets and curate them with DuckDB

More Plugins in this Category

AssetShipper Forward asset lineage to one or more desired destinations.

Create automations with Core Asset Ee AssetShipper

Register S3 JSON files as governed datasets and curate them with DuckDB

More Plugins in this Category

AssetShipper

Forward asset lineage to one or more desired destinations.