​Google ​Cloud ​Run ​Task ​Runner

Run tasks as containers on Google Cloud Run.

How to use the Google Cloud Run task runner

The Google Cloud Run task runner deploys the container for each task to Google Cloud Run.

How the Google Cloud Run task runner works

To use the inputFiles, outputFiles, or namespaceFiles properties, you must set the bucket property. The bucket acts as an intermediary storage layer for the task runner:

• Input and namespace files are uploaded to the Cloud Storage bucket before the task runs.
• Output files are stored in the same bucket and made available for download and preview in the Kestra UI after execution.

To help track where files are stored, the task runner creates a unique folder for each run. You can access this folder using the {{ bucketPath }} Pebble expression or the BUCKET_PATH environment variable.

Because Cloud Run environments are ephemeral, tasks run in the root directory instead of the working directory. Therefore, use the {{ workingDir }} Pebble expression or the WORKING_DIR environment variable to access your input or namespace files.

Example flows

Basic example

The following example runs a simple shell command inside a Cloud Run container:

id: new-shell
namespace: company.team

variables:
  projectId: myProjectId
  region: europe-west2

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.ee.gcp.runner.CloudRun
      projectId: "{{ vars.projectId }}"
      region: "{{ vars.region }}"
      serviceAccount: "{{ secret('GOOGLE_SA') }}"
    commands:
      - echo "Hello World"

Example with file inputs

This flow runs a shell command and demonstrates how to use file inputs and outputs with Cloud Run:

id: new-shell-with-file
namespace: company.team

variables:
  projectId: myProjectId
  region: europe-west2

inputs:
  - id: file
    type: FILE

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    inputFiles:
      data.txt: "{{ inputs.file }}"
    outputFiles:
      - out.txt
    containerImage: centos
    taskRunner:
      type: io.kestra.plugin.ee.gcp.runner.CloudRun
      projectId: "{{ vars.projectId }}"
      region: "{{ vars.region }}"
      bucket: "{{ vars.bucket }}"
      serviceAccount: "{{ secret('GOOGLE_SA') }}"
    commands:
      - cp {{ workingDir }}/data.txt {{ workingDir }}/out.txt

How to run tasks on Google Cloud Run

Before you begin

You’ll need the following:

1. A Google Cloud account.
2. A Kestra instance (version 0.16.0 or later) with Google credentials stored as secrets or as environment variables.

Google Cloud Console setup

Create a project

If you don’t already have one, create a new project in the Google Cloud Console.

Ensure the new project is selected in the top navigation bar.

Enable the Cloud Run Admin API

Open APIs & Services → Enable APIs and Services, then search for and enable Cloud Run Admin API.

Create a service account

After enabling the API, create a service account to allow Kestra to access Cloud Run resources.

In the search bar, find Service Accounts and select Create Service Account.

Assign the following roles to the service account:

• Cloud Run Developer
• Logs Viewer
• Storage Admin (required for file upload and download)

Refer to the Google credentials guide for details on adding the service account to Kestra as a secret.

To grant access to the Compute Engine default service account, go to IAM & Admin → Service Accounts → Permissions → Grant Access, and assign the Service Account User role to your new service account.

Create a storage bucket

Search for “Bucket” in the Cloud Console and create a new GCS bucket. Use default settings unless otherwise required.

Create a flow

Below is an example flow that runs a shell script using the Google Cloud Run task runner. The task copies a file under a new name.

containerImage: centos
taskRunner:
  type: io.kestra.plugin.ee.gcp.runner.CloudRun
  projectId: "{{ secret('GCP_PROJECT_ID') }}"
  region: "{{ vars.region }}"
  bucket: "{{ secret('GCP_BUCKET') }}"
  serviceAccount: "{{ secret('GOOGLE_SA') }}"

The containerImage property is required because Cloud Run executes each task as a container. You can use any image from a public or private registry. In this example, we use centos.

Here’s the complete flow:

id: new-shell-with-file
namespace: company.team

variables:
  projectId: myProjectId
  region: europe-west2

inputs:
  - id: file
    type: FILE

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    inputFiles:
      data.txt: "{{ inputs.file }}"
    outputFiles:
      - out.txt
    containerImage: centos
    taskRunner:
      type: io.kestra.plugin.ee.gcp.runner.CloudRun
      projectId: "{{ secret('GCP_PROJECT_ID') }}"
      region: "{{ vars.region }}"
      bucket: "{{ secret('GCP_BUCKET') }}"
      serviceAccount: "{{ secret('GOOGLE_SA') }}"
    commands:
      - cp {{ workingDir }}/data.txt {{ workingDir }}/out.txt

When you execute the flow, the logs in Kestra confirm the creation of the Cloud Run task runner:

You can also verify job creation in the Google Cloud Console:

Once the task completes, the Cloud Run container is automatically terminated to free up resources.