Schedule icon
Request icon
TransformValue icon
Loop icon
Pause icon
Get icon
If icon
SlackIncomingWebhook icon
Set icon

Python Requirements Dependency Health Check with PyPI Registry

Audit Python requirements.txt dependencies against PyPI for yanked, outdated, and stale packages with automated Slack alerting and Kestra KV diffing.

Categories
Infrastructure

Dependencies in a Python project can easily fall behind, become deprecated, or remain unmaintained for years without developers noticing. Rather than suffering alert fatigue from noisy daily pull requests for routine patch updates, this workflow periodically inspects your requirements.txt, queries the official PyPI registry API, and sends a concise Slack digest only when yanked releases, severe major-version lag, or stale unmaintained packages are found.

How it works

  1. weekly (io.kestra.plugin.core.trigger.Schedule) triggers the workflow every Monday at 08:00 UTC (disabled by default until configured).
  2. fetch_requirements (io.kestra.plugin.core.http.Request) downloads the raw requirements.txt file from requirements_url.
  3. list_packages (io.kestra.plugin.transform.jsonata.TransformValue) parses package names and pinned versions, ignores comments and editable flags (-e), strips bracketed extras (such as package[extra]), and caps the list to max_packages.
  4. check_packages (io.kestra.plugin.core.flow.Loop) iterates through each package. fetch_pypi_metadata queries the public PyPI JSON API, and assess_health evaluates the package against your pinned version, classifying it as YANKED, UNKNOWN_VERSION, STALE, OUTDATED, or OK. pause_for_rate_limit pauses briefly between calls to respect PyPI rate limits.
  5. summarize (io.kestra.plugin.transform.jsonata.TransformValue) formats a consolidated list of packages requiring attention.
  6. previous_report (io.kestra.plugin.core.kv.Get) retrieves the previous execution report from Kestra's KV store. alert_if_new (io.kestra.plugin.core.flow.If) posts to Slack only if actionable issues are detected and the report differs from the previous run. remember_report updates the KV store with the latest report state.
  7. The errors block catches network or parsing failures and posts an execution alert to Slack.

What you get

  • Automated, scheduled auditing of Python dependencies against official PyPI registry metadata.
  • Early detection of yanked releases and stale packages with no updates for over a year.
  • Zero alert spam: identical reports are deduplicated via Kestra's KV store and sent only when status changes.
  • Dedicated failure handling informing the team if your repository or the PyPI registry becomes unreachable.

Who it's for

  • Python backend engineers, data scientists, and platform teams maintaining Python applications and microservices.
  • DevOps engineers monitoring software supply-chain health and library obsolescence.

Why orchestrate this with Kestra

Building this logic as an ad-hoc cron script requires maintaining custom scheduling infrastructure, secret encryption, rate-limiting loops, state storage across runs, and error handling. Kestra provides declarative scheduling, built-in KV state persistence, visual execution logs, and robust retry mechanics in a portable YAML workflow.

Prerequisites

  • HTTPS network access to pypi.org and your raw requirements.txt URL.
  • A Slack incoming webhook URL for receiving notifications.

Secrets

  • SLACK_WEBHOOK_URL: Slack Incoming Webhook URL used to dispatch status reports and failure alerts (required).

Quick start

  1. Store the SLACK_WEBHOOK_URL secret in your Kestra namespace (Namespaces → Secrets).
  2. Set requirements_url to the raw HTTPS URL of your target requirements.txt.
  3. Click Execute in the Kestra UI to run an initial audit and inspect the summarize output.
  4. Enable the weekly schedule trigger to run automated recurring checks.

How to extend

  • Increase max_packages to audit larger dependency manifests or monorepos.
  • Replace the Slack webhook task with Microsoft Teams, PagerDuty, or Discord notifications.
  • Chain this workflow to trigger automated dependency bump pipelines when yanked packages are identified.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.