CloudRun
Task runner that executes a task inside a job in Google Cloud Run.
: : alert{type="info"} This plugin is only available in the Enterprise Edition (EE). : :
This task runner is container-based so the containerImage property must be set.
You need to have roles 'Cloud Run Developer' and 'Logs Viewer' to be able to use it.
To access the task's working directory, use the {{workingDir}} Pebble expression or the WORKING_DIR environment variable. Input files and namespace files will be available in this directory.
To generate output files you can either use the outputFiles task's property and create a file with the same name in the task's working directory, or create any file in the output directory which can be accessed by the {{outputDir}} Pebble expression or the OUTPUT_DIR environment variables.
To use inputFiles, outputFiles or namespaceFiles properties, make sure to set the bucket property. The bucket serves as an intermediary storage layer for the task runner. Input and namespace files will be uploaded to the cloud storage bucket before the task run. Similarly, the task runner will store outputFiles in this bucket during the task run. In the end, the task runner will make those files available for download and preview from the UI by sending them to internal storage.
The task runner will generate a folder in the configured bucket for each task run. You can access that folder using the {{ bucketPath }} Pebble expression or the BUCKET_PATH environment variable.
Warning, contrarily to other task runners, this task runner didn't run the task in the working directory but in the root directory. You must use the {{ workingDir }} Pebble expression or the WORKING_DIR environment variable to access files.
Note that when the Kestra Worker running this task is terminated, the Cloud Run Job will still run until completion.
type: "io.kestra.plugin.ee.gcp.runner.CloudRun"Execute a Shell command.
id: new-shell
namespace: company.team
variables:
projectId: "projectId"
region: "europe-west2"
tasks:
- id: shell
type: io.kestra.plugin.scripts.shell.Commands
taskRunner:
type: io.kestra.plugin.ee.gcp.runner.CloudRun
projectId: "{{ vars.projectId }}"
region: "{{ vars.region }}"
serviceAccount: "{{ secret('GOOGLE_SA') }}"
commands:
- echo "Hello World"Pass input files to the task, execute a Shell command, then retrieve output files.
id: new-shell-with-file
namespace: company.team
inputs:
- id: file
type: FILE
variables:
projectId: "projectId"
bucket: "bucket"
region: "europe-west2"
tasks:
- id: shell
type: io.kestra.plugin.scripts.shell.Commands
inputFiles:
data.txt: "{{ inputs.file }}"
outputFiles:
- out.txt
containerImage: centos
taskRunner:
type: io.kestra.plugin.ee.gcp.runner.CloudRun
projectId: "{{ vars.projectId }}"
region: "{{ vars.region }}"
bucket: "{{ vars.bucket }}"
serviceAccount: "{{ secret('GOOGLE_SA') }}"
commands:
- cp {{ workingDir }}/data.txt {{ workingDir }}/out.txtThe GCP region.
Google Cloud Storage Bucket to use to upload (inputFiles and namespaceFiles) and download (outputFiles) files.
It's mandatory to provide a bucket if you want to use such properties.
Determines how often Kestra should poll the container for completion. By default, the task runner checks every 5 seconds whether the job is completed. You can set this to a lower value (e.g. PT0.1S = every 100 milliseconds) for quick jobs and to a lower threshold (e.g. PT1M = every minute) for long-running jobs. Setting this property to a lower value will reduce the number of API calls Kestra makes to the remote service — keep that in mind in case you see API rate limit errors.
The GCP project ID.
The GCP scopes to be used.
The GCP service account key.
Additional time after the job ends to wait for late logs.
The maximum duration to wait for the job completion unless the task timeout property is set which will take precedence over this property.
Google Cloud Run will automatically timeout the Job upon reaching such duration and the task will be failed.