PluginsFile systemTasksHttpDownload

Download​Download

Download Download

yaml
type: "io.kestra.plugin.core.http.Download"

Download a file from a HTTP server.

This task connects to a HTTP server and copy a file to Kestra's internal storage.

Examples

Download a CSV file.

yaml
id: download
namespace: company.team

tasks:
  - id: extract
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv

Properties

uri

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The fully-qualified URI that points to the HTTP destination

allowFailed

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

If true, allow a failed response code (response code >= 400)

body

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The full body as a string

contentType

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The request content type

failOnEmptyResponse

  • Type: boolean
  • Dynamic: ❌
  • Required: ❌
  • Default: true

Should the task fail when downloading an empty file.

formData

  • Type: object
  • Dynamic: ✔️
  • Required: ❌

The form data to be send

headers

  • Type: object
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

The headers to pass to the request

method

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: GET
  • Possible Values:
    • OPTIONS
    • GET
    • HEAD
    • POST
    • PUT
    • DELETE
    • TRACE
    • CONNECT
    • PATCH
    • CUSTOM

The HTTP method to use

options

The HTTP request options

sslOptions

The SSL request options

Outputs

code

  • Type: integer
  • Required: ❌

The status code of the response.

headers

  • Type: object
  • SubType: array
  • Required: ❌

The headers of the response.

length

  • Type: integer
  • Required: ❌

The content-length of the response.

uri

  • Type: string
  • Required: ❌
  • Format: uri

The URL of the downloaded file on Kestra's internal storage.

Definitions

java.nio.charset.Charset

Properties

io.kestra.plugin.core.http.HttpInterface-RequestOptions

Properties

basicAuthPassword
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The password for HTTP basic authentication.

basicAuthUser
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The username for HTTP basic authentication.

connectTimeout
  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Format: duration

The time allowed to establish a connection to the server before failing.

connectionPoolIdleTimeout
  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: 0.0
  • Format: duration

The time an idle connection can remain in the client's connection pool before being closed.

defaultCharset
  • Type: Charset
  • Dynamic: ❌
  • Required: ❌
  • Default: UTF-8

The default charset for the request.

followRedirects
  • Type: boolean
  • Dynamic: ❌
  • Required: ❌
  • Default: true

Whether redirects should be followed automatically.

logLevel
  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Possible Values:
    • ALL
    • TRACE
    • DEBUG
    • INFO
    • WARN
    • ERROR
    • OFF
    • NOT_SPECIFIED

The log level for the HTTP client.

maxContentLength
  • Type: integer
  • Dynamic: ❌
  • Required: ❌
  • Default: 10485760

The maximum content length of the response.

proxyAddress
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The address of the proxy server.

proxyPassword
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The password for proxy authentication.

proxyPort
  • Type: integer
  • Dynamic: ❌
  • Required: ❌

The port of the proxy server.

proxyType
  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: DIRECT
  • Possible Values:
    • DIRECT
    • HTTP
    • SOCKS

The type of proxy to use.

proxyUsername
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The username for proxy authentication.

readIdleTimeout
  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: 300.000000000
  • Format: duration

The time allowed for a read connection to remain idle before closing it.

readTimeout
  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: 10.000000000
  • Format: duration

The maximum time allowed for reading data from the server before failing.

io.kestra.plugin.core.http.HttpInterface-SslOptions

Properties

insecureTrustAllCertificates
  • Type: boolean
  • Dynamic: ❌
  • Required: ❌

Whether to disable checking of the remote SSL certificate.

Only applies if no trust store is configured. Note: This makes the SSL connection insecure and should only be used for testing. If you are using a self-signed certificate, set up a trust store instead.

Metrics

response.length

  • Type: counter

Was this page helpful?