Compression ArchiveCompress

Compression ArchiveCompress
Certified
Create an archive from multiple files

Builds an archive from rendered file map inputs stored in internal storage, optionally wrapping it with a stream compressor (for example TAR + GZIP). Fails for algorithms that are extract-only.

yaml
type: io.kestra.plugin.compress.ArchiveCompress

Examples

Compress an input file

yaml
id: archive_compress
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: "archive_compress"
    type: "io.kestra.plugin.compress.ArchiveCompress"
    from:
      myfile.txt: "{{ inputs.file }}"
    algorithm: ZIP

Download two files, compress them together and upload to S3 bucket

yaml
id: archive_compress
namespace: company.team

tasks:
  - id: products_download
    type: io.kestra.plugin.core.http.Download
    uri: "http://huggingface.co/datasets/kestra/datasets/raw/main/csv/products.csv"

  - id: orders_download
    type: io.kestra.plugin.core.http.Download
    uri: "https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv"

  - id: archive_compress
    type: "io.kestra.plugin.compress.ArchiveCompress"
    from:
      products.csv: "{{ outputs.products_download.uri }}"
      orders.csv: "{{ outputs.orders_download.uri }}"
    algorithm: TAR
    compression: GZIP

  - id: upload_compressed
    type: io.kestra.plugin.aws.s3.Upload
    bucket: "example"
    region: "{{ secret('AWS_REGION') }}"
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    from: "{{ outputs.archive_compress.uri }}"
    key: "archive.gz"

Properties

algorithm *string

Possible Values

ARARJCPIODUMPJARTARZIP

Archive container format to create

Required archive format. Compression supports AR, CPIO, JAR, TAR, and ZIP; ARJ and DUMP are extract-only and will fail on compression.

from *object

Structured data items, either as a map, a list of map, a URI, or a JSON string.

Structured data items can be defined in the following ways:

A single item as a map (a document).
A list of items as a list of maps (a list of documents).
A URI, supported schemes are kestra for internal storage files, file for host local files, and nsfile for namespace files.
A JSON String that will then be serialized either as a single item or a list of items.

compression string

Possible Values

BROTLIBZIP2DEFLATEDEFLATE64GZIPLZ4BLOCKLZ4FRAMELZMASNAPPYSNAPPYFRAMEXZZZSTD

Optional compressor applied to the archive stream

Use a single-file compressor such as GZIP alongside TAR. Leave null to store the archive uncompressed. Brotli, Deflate64, and Snappy variants are decode-only and will fail during compression.