Copy Copy 

type: "io.kestra.plugin.gcp.bigquery.Copy"

Copy a BigQuery table or partition to other one

# Examples

id: "copy"
type: "io.kestra.plugin.gcp.bigquery.Copy"
sourceTables:
- "my_project.my_dataset.my_table$20130908"
destinationTable: "my_project.my_dataset.my_table"

# Properties

# createDisposition

  • Type: string

  • Dynamic: ❌

  • Required: ❌

  • Possible Values:

    • CREATE_IF_NEEDED
    • CREATE_NEVER

Whether the job is allowed to create tables

# destinationTable

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The destination table

If not provided a new table is created.

# dryRun

  • Type: boolean
  • Dynamic: ❌
  • Required: ❌
  • Default: false

Sets whether the job has to be dry run or no.

A valid query will return a mostly empty response with some processing statistics, while an invalid query will return the same error it would if it wasn't a dry run.

# jobTimeout

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Format: duration

Job timeout.

If this time limit is exceeded, BigQuery may attempt to terminate the job.

# labels

  • Type: object
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

The labels associated with this job.

You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.

# location

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The geographic location where the dataset should reside

This property is experimental and might be subject to change or removed.

See Dataset Location

# operationType

  • Type: string

  • Dynamic: ✔️

  • Required: ✔️

  • Possible Values:

    • COPY
    • SNAPSHOT
    • RESTORE
    • CLONE

Sets the supported operation types in table copy job.

  • COPY: The source and destination table have the same table type.
  • SNAPSHOT: The source table type is TABLE and the destination table type is SNAPSHOT.
  • RESTORE: The source table type is SNAPSHOT and the destination table type is TABLE.
  • CLONE: The source and destination table have the same table type, but only bill for unique data.

# projectId

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP project id

# retryAuto

# retryMessages

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: [due to concurrent update, Retrying the job may solve the problem]

The message that are valid for a automatic retry.

Message is tested as a substring of the full message and case insensitive

# retryReasons

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: [rateLimitExceeded, jobBackendError, internalError, jobInternalError]

The reason that are valid for a automatic retry.

# scopes

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: [https://www.googleapis.com/auth/cloud-platform]

The GCP scopes to used

# serviceAccount

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP service account key

# sourceTables

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ✔️

The source tables

Can be table or partitions.

# writeDisposition

  • Type: string

  • Dynamic: ❌

  • Required: ❌

  • Possible Values:

    • WRITE_TRUNCATE
    • WRITE_APPEND
    • WRITE_EMPTY

The action that should occur if the destination table already exists

# Outputs

# jobId

  • Type: string

The job id

# Definitions

# Constant

# interval

  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration

# maxAttempt

  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Minimum: >= 1

# maxDuration

  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Format: duration

# warningOnRetry

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

# Random

# maxAttempt

  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Minimum: >= 1

# maxDuration

  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Format: duration

# maxInterval

  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration

# minInterval

  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration

# warningOnRetry

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

# Exponential

# delayFactor

  • Type: number
  • Dynamic: ❓
  • Required: ❌

# interval

  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration

# maxAttempt

  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Minimum: >= 1

# maxDuration

  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Format: duration

# maxInterval

  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration

# warningOnRetry

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

← CopyPartitions →

 Tweet  Share  Share