Install Kestra on AWS EC2 with Postgres RDS database and S3 internal storage backend.
This guide provides instructions for deploying Kestra on Amazon Web Services (AWS). We’ll use EC2 with Docker to host Kestra server, a PostgreSQL RDS database and AWS S3 as storage backend.
- basic knowledge about using a command line interface
- basic knowledge about EC2, S3 and PostgreSQL.
You can find the corresponding full Terraform configuration in this repository.
First, create an EC2 instance. To do so, go to the AWS console and choose EC2.
- Give a name to your instance.
- Choose Ubuntu as OS.
- Instance type: Kestra needs at least 4GiB Memory and 2vCPU to run correctly. Choosing t3-medium is a good starting point.
- Create a key-pair to securely connect to your instance. This key is needed to connect through SSH in the following steps.
- Create a security group that allows SSH traffic from your IP. Also allow HTTPS traffic.
You can now click on “Launch instance” and wait a few seconds for the compute to be up and running.
Then you can open a terminal on your laptop and connect to your instance through SSH:
ssh -i <your-key-pair.pem> ubuntu@<your-EC2-public-IP>
Kestra can be run directly from the .jar binary or thanks to Docker. We’ll use docker here for quicker setup:
- Install Docker on the EC2 instance. You can find the last updated instruction on Docker website.
- Install docker-compose.
To check your installation run
docker version and
docker-compose version. You're now ready to download and launch Kestra server.
Download the official Docker-Compose file:
curl -o docker-compose.yml https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml
Use editor such as Vim to modify the
docker-compose.yml and set basic authentication to true, and configure your basic authentication credentials to secure your Kestra instance.
Then use the following command to start Kestra server:
docker-compose up -d
Kestra is now running and the Kestra server exposes traffic on the 8080 port. To be connect through your web browser, update the inbound traffic rules in the EC2 security group.
Go to the EC2 console, and select Security Group. Choose the security group attached to your EC2 instance and add a new inbound rule to open access to the 8080 port.
If you want to only allow traffic coming from your IP address, set the source to your own IP. If you want to make it open to the entire Internet, leave it at 0.0.0.0.
Note that if you haven't set up basic authentication in the previous step, your Kestra instance will be publicly available to anyone without any access restriction.
You can now access your Kestra instance and create, edit and run Flows.
This first installation relies on a Postgres database running alongside the Kestra server - on the EC2 instance (see the postgres service running thanks to the docker-compose).
For a simple PoC, you can keep that Postgres database running in Docker.
However, for a production-grade installation, we recommend a managed database service such as AWS RDS.
Create a AWS RDS database
- Go to the RDS console.
- Create a database and choose Postgres (Kestra also supports MySQL, but Postgres is recommended)
- Set a username and password.
- On the connectivity configuration choose “Connect to an EC2 compute resource” and choose your EC2 instance.
- Also select the existing DB subnet group and existing VPC security group and choose the one attached to your EC2 instance.
- Fine tune instance class, storage type to avoid import AWS costs. For a first step a small Postgres instance is enough
- Hit create and wait for completion
Create Kestra database
Before attaching your Kestra server to our new database backend, initialize the database with a base schema as follows:
- Connect to your EC2 instance with ssh
- Install a PostgresSQL client:
sudo apt-get install postgresql-client
- Create the Kestra database:
createdb -h <your-rds-url-endpoint> -U <your-username> -p 5432 kestra
Update Kestra configuration
In the docker-compose configuration, edit the
datasources property of the Kestra service in the following way:
Because you now use RDS service, you don't need the postgres service anymore. Remove it from the
In order for the changes to take effect, restart the docker services with
docker compose restart or
docker compose up -d.
By default, internal storage is implemented using the local file system. This section will guide you on how to change the storage backend to S3 to ensure a more reliable, durable and scalable storage.
- Go to the S3 console and create a bucket.
- Go to IAM and create a new User Group with AWS full access.
- Create a new user and attach it to the user group.
- For the new user, go to Security Credentials and create an access key. Choose “Application running on an AWS compute service” and retrieve the access and secret keys.
- Edit Kestra storage configuration.
- Restart docker services.
This guide walked you through installing Kestra on an AWS EC2 instance with RDS database and S3 storage backend.
This setup provides the easiest starting point for running Kestra in production on a single machine. For a deployment to a distributed cluster, check the Kubernetes deployment guide.
Reach out via Slack if you encounter any issues or if you have any questions regarding deploying Kestra to production.
Make sure to also check the CI/CD guide to automate your workflow deployments based on changes in Git.