Blueprints

Logging configuration in a Python script using Loguru

Source

yaml
id: loguru
namespace: company.team

inputs:
  - id: nr_logs
    type: INT
    defaults: 100

tasks:
  - id: reproducer
    type: io.kestra.plugin.scripts.python.Script
    warningOnStdErr: false
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
    containerImage: ghcr.io/kestra-io/pydata:latest
    script: |
      from loguru import logger
      from faker import Faker
      import time
      import sys

      logger.remove()
      logger.add(sys.stdout, level="INFO")
      logger.add(sys.stderr, level="WARNING")

      def generate_logs(fake, num_logs):
          logger.debug("This message will not show up as the log level is set to INFO")
          logger.warning("Starting to generate log messages")
          for _ in range(num_logs):
              log_message = fake.sentence()
              logger.info(log_message)
              time.sleep(0.01)
          logger.warning("Finished generating log messages")

      if __name__ == "__main__":
          faker_ = Faker()
          generate_logs(faker_, int("{{ inputs.nr_logs }}"))
See more

About this blueprint

Python

This flow demonstrates how to configure logging in a Python script using Loguru. The Script task will generate, by default, 100 random log messages, but this number of logs can be changed at runtime using the input parameter nr_logs.

  • The warningOnStdErr property is set to false to prevent the Script task from failing when the logger.warning method is used.
  • The containerImage property is set to ghcr.io/kestra-io/pydata:latest to use the pydata Docker image that contains the loguru and faker libraries.
  • The script property contains the Python code that will be executed by the Script task.

The log level is set to INFO in the Script task. Therefore, the logger.debug message will NOT show up in the logs. The logger.warning messages will be translated to WARN-level logs in Kestra. The logger.info messages will be translated to INFO-level logs in Kestra.

Script

Docker

More Related Blueprints

Namespace Files Trigger Python Schedule

Run specific tasks only on business days for a specific country

Python SQL Outputs API DuckDB

Extract data from a REST API, process it in Python with Polars in a Docker container, then run DuckDB query and preview results as a table in the Outp...

AWS Python Task Runner

Run a Python script on AWS ECS Fargate with AWS Batch

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra