Airbyte Connections Reset

Airbyte Connections Reset

Certified

Reset streams of a self-hosted Airbyte connection

Starts a reset of the selected streams of a self-hosted Airbyte connection and returns the ID of the reset job. At least one stream is required.

**Warning: ** resetting a stream deletes the data this stream generated in the destination and resets its cursors back to their initial state. The next sync of the connection then replicates the stream from scratch.

The task does not wait for the job to finish. To wait, pass the returned jobId to io.kestra.plugin.airbyte.connections.CheckStatus.

This task calls Airbyte's POST /api/v1/connections/reset/stream config API. To reset a whole connection on Airbyte Cloud, use io.kestra.plugin.airbyte.cloud.jobs.Reset.

yaml
type: io.kestra.plugin.airbyte.connections.Reset

Reset one stream of a self-hosted Airbyte connection and wait for the job

yaml
id: airbyte_reset_stream
namespace: company.team

tasks:
  - id: reset
    type: io.kestra.plugin.airbyte.connections.Reset
    url: http://localhost:8000
    username: "{{ secret('AIRBYTE_USERNAME') }}"
    password: "{{ secret('AIRBYTE_PASSWORD') }}"
    connectionId: e3b1ce92-547c-436f-b1e8-23b6936c12cd
    streams:
      - streamName: customers
        streamNamespace: public

  - id: wait_for_reset
    type: io.kestra.plugin.airbyte.connections.CheckStatus
    url: http://localhost:8000
    username: "{{ secret('AIRBYTE_USERNAME') }}"
    password: "{{ secret('AIRBYTE_PASSWORD') }}"
    jobId: "{{ outputs.reset.jobId }}"

Reset several streams, with and without a namespace

yaml
id: airbyte_reset_streams
namespace: company.team

tasks:
  - id: reset
    type: io.kestra.plugin.airbyte.connections.Reset
    url: http://localhost:8000
    token: "{{ secret('AIRBYTE_TOKEN') }}"
    connectionId: e3b1ce92-547c-436f-b1e8-23b6936c12cd
    streams:
      - streamName: customers
        streamNamespace: public
      - streamName: orders
Properties

Connection ID

Airbyte connection ID that owns the streams to reset.

Min items1

Streams to reset

Streams to reset, each with a streamName and an optional streamNamespace. At least one stream is required.

Resetting a stream deletes its data in the destination and resets its cursors.

Definitions
streamName*string

Stream name

Name of the stream to reset, as it appears in the source.

streamNamespacestring

Stream namespace

Namespace of the stream to reset, such as a database schema. Leave it out for streams without a namespace.

Airbyte API URL

Base URL of the Airbyte instance to call. This value is rendered from the current run context

Application credentials

Client credentials used to request an Airbyte application access token from /api/v1/applications/token. Use this instead of a static token when Airbyte application authentication is enabled

Definitions
clientId*string

Client ID

Airbyte application client ID

clientSecret*string

Client secret

Airbyte application client secret. Store this value in a Secret

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150

HTTP client options

HTTP client configuration applied to Airbyte API requests

Definitions
allowFailedbooleanstring
Defaultfalse

If true, allow a failed response code (response code >= 400)

allowedResponseCodesarray
SubTypeinteger

List of response code allowed for this request

auth

The authentication to use.

type*object
passwordstring

The password for HTTP basic authentication.

usernamestring

The username for HTTP basic authentication.

type*object
tokenstring

The token for bearer token authentication.

type*object
passwordstring

The password for HTTP Digest authentication.

usernamestring

The username for HTTP Digest authentication.

defaultCharsetstring
DefaultUTF-8

The default charset for the request.

enabledTcpExtendedKeepAlivebooleanstring
Defaulttrue

Whether to enable TCP Keep-Alive extended socket options (TCP_KEEPIDLE, TCP_KEEPINTERVAL, TCP_KEEPCOUNT).

Set to false when running on Windows workers, as these extended socket options are not supported by the Windows JDK and will cause connection failures.

followRedirectsbooleanstring
Defaulttrue

Whether redirects should be followed automatically.

logsarray
SubTypestring
Possible Values
REQUEST_HEADERSREQUEST_BODYRESPONSE_HEADERSRESPONSE_BODY

The enabled log.

proxy

The proxy configuration.

addressstring

The address of the proxy server.

passwordstring

The password for proxy authentication.

portintegerstring

The port of the proxy server.

typestring
DefaultDIRECT
Possible Values
DIRECTHTTPSOCKS

The type of proxy to use.

usernamestring

The username for proxy authentication.

retry

Retry strategy for HTTP requests.

retry is disabled by default.

Retry with a fixed delay between attempts.

interval*string
Formatduration
type*object
behaviorstring
DefaultRETRY_FAILED_TASK
Possible Values
RETRY_FAILED_TASKCREATE_NEW_EXECUTION
maxAttemptsinteger
Minimum>= 1
maxDurationstring
Formatduration
warningOnRetryboolean
Defaultfalse

Retry with exponentially increasing delays between attempts.

interval*string
Formatduration
maxInterval*string
Formatduration
type*object
behaviorstring
DefaultRETRY_FAILED_TASK
Possible Values
RETRY_FAILED_TASKCREATE_NEW_EXECUTION
delayFactornumber
maxAttemptsinteger
Minimum>= 1
maxDurationstring
Formatduration
warningOnRetryboolean
Defaultfalse

Retry with a random delay within a configurable range between attempts.

maxInterval*string
Formatduration
minInterval*string
Formatduration
type*object
behaviorstring
DefaultRETRY_FAILED_TASK
Possible Values
RETRY_FAILED_TASKCREATE_NEW_EXECUTION
maxAttemptsinteger
Minimum>= 1
maxDurationstring
Formatduration
warningOnRetryboolean
Defaultfalse
retryOnStatusCodesarray
SubTypeinteger
Default[502,503,504]

HTTP status codes that should be retried.

Used for idempotent methods (GET, HEAD) without their own entry in retryOnStatusCodesByMethod. Non-idempotent methods are never retried on a status code unless listed in retryOnStatusCodesByMethod. Defaults to 502, 503 and 504 for idempotent methods (GET, HEAD) only.

retryOnStatusCodesByMethodobject
SubTypearray

Per-HTTP-method overrides for which status codes should be retried.

Keys are HTTP methods (e.g. GET, POST). A method with no entry here falls back to retryOnStatusCodes. This lets a caller retry more broadly for safe, read-only methods (for example, every 5xx and a 429 for GET/HEAD) while keeping a narrower list for methods that write, where retrying a request that may have already been processed by the origin risks duplicating the effect. Example: {GET: [429, 500, 502, 503, 504], HEAD: [429, 500, 502, 503, 504]}.

retryableTransportFailureMethodsarray
SubTypestring
Default["GET","HEAD"]

HTTP methods for which any transport-level failure may be retried.

A transport-level failure is a connection or timeout error rather than an HTTP response (e.g. a read timeout or a connection reset mid-request). For a method listed here, any such failure is retried. Defaults to GET and HEAD HTTP methods.

ssl

The SSL request options

insecureTrustAllCertificatesbooleanstring

Whether to disable checking of the remote SSL certificate.

Only applies if no trust store is configured. Note: This makes the SSL connection insecure and should only be used for testing. If you are using a self-signed certificate, set up a trust store instead.

timeout

The timeout configuration.

connectTimeoutstring

The time allowed to establish a connection to the server before failing.

readIdleTimeoutstring
DefaultPT5M

The time allowed for a read connection to remain idle before closing it.

Basic auth password

Password for Airbyte basic authentication. Store this value in a Secret

Bearer token

Bearer token used for Airbyte API requests. This value is rendered from the current run context

Basic auth username

Username for Airbyte basic authentication. If both basic auth and a bearer token are configured, basic auth is sent last and takes precedence

Airbyte web app URL

Base URL of the Airbyte web app, used to build the jobUrl output. Defaults to url. Set it when the web app is served from a different address than the API

Job ID

Airbyte job ID of the reset. Pass it to io.kestra.plugin.airbyte.connections.CheckStatus to wait for the job to finish.

More Plugins in this Category

Firecrawl
Firecrawl

Tasks that scrape and crawl websites into clean, LLM-ready content with Firecrawl.

Data

2 tasks

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks

8 blueprints