Use Cloudflare R2 with MinIO Gateway for Kestra
This guide demonstrates how to use Cloudflare R2 as an object storage backend through an S3-compatible interface, exposed to Kestra via a MinIO Gateway.
This setup enables Kestra to continue using S3 storage without requiring configuration changes.
This guide assumes that MinIO runs locally in gateway mode to access Cloudflare R2. It is intended for local development and QA environments, and is not optimized for production deployments.
Create an R2 Bucket
Log into Cloudflare Dashboard and create a new R2 bucket:
- Navigate to R2 → Create Bucket
- Choose a name like
kestra-bucket
Generate Access Keys
Go to API Tokens → R2 Keys and create a new key pair:
access_key_id
: Your user access keysecret_access_key
: Your secret key
Be sure to save these credentials securely.
Retrieve the R2 Endpoint
Cloudflare R2 provides a static S3-compatible endpoint:
https://<ACCOUNT_ID>.r2.cloudflarestorage.com
Replace <ACCOUNT_ID>
with your Cloudflare account ID, found in the R2 dashboard.
Set Up MinIO Gateway to R2
MinIO will act as a gateway, forwarding all S3 traffic to Cloudflare R2.
docker-compose.yml
version: '3.8'
services:
minio:
image: minio/minio:latest
container_name: minio-r2-gateway
command: gateway s3 https://<ACCOUNT_ID>.r2.cloudflarestorage.com
environment:
MINIO_ROOT_USER: <R2_ACCESS_KEY>
MINIO_ROOT_PASSWORD: <R2_SECRET_KEY>
ports:
- "9000:9000"
restart: always
Replace
<ACCOUNT_ID>
,<R2_ACCESS_KEY>
, and<R2_SECRET_KEY>
with your actual Cloudflare and access values.
Validate Setup with MinIO Client
Install the MinIO Client (mc):
mc alias set r2 http://localhost:9000 <R2_ACCESS_KEY> <R2_SECRET_KEY>
mc mb r2/kestra-bucket
mc ls r2
Configure Kestra (No Changes Required)
Since Kestra supports MinIO-compatible S3 endpoints, no changes to your configuration are required:
storage:
type: minio
minio:
endpoint: localhost
port: 9000
bucket: kestra-bucket
accessKey: <R2_ACCESS_KEY>
secretKey: <R2_SECRET_KEY>
Kestra will interact with MinIO, which in turn proxies to R2.
Test with a Flow
id: r2_test_flow
namespace: company.team
tasks:
- id: write_output
type: io.kestra.plugin.scripts.python.Script
taskRunner:
type: io.kestra.plugin.scripts.runner.docker.Docker
containerImage: ghcr.io/kestra-io/pydata:latest
outputFiles:
- r2-output.json
script: |
import json
from kestra import Kestra
data = {'message': 'stored in R2'}
Kestra.outputs(data)
with open('r2-output.json', 'w') as f:
json.dump(data, f)
Then, verify the file was stored correctly using:
mc cat r2/kestra-bucket/main/company/team/r2_test_flow/...
Expected output:
{"message": "stored in R2"}%
References
You now have Cloudflare R2 configured as your object storage backend for Kestra, fully integrated via MinIO Gateway.
Was this page helpful?