Alibaba Cloud OSS Storage Class Cost Governance Sentinel with Kestra

id: alibaba-oss-storage-class-cost-governance-sentinel
namespace: company.team
description: |
  List objects in an Alibaba Cloud OSS bucket on a schedule, classify the
  result HEALTHY or STORAGE_CLASS_DRIFT_DETECTED based on how many objects
  are still sitting on the Standard storage class well past a configured
  age threshold, log clean passes, and alert Slack with the exact keys and
  ages before unmanaged storage-class drift becomes a cost problem.

triggers:
  - id: periodic_storage_class_check
    type: io.kestra.plugin.core.trigger.Schedule
    description: "Checks once a day - object age and storage class change slowly, so
      daily resolution is enough to catch lifecycle drift well before it
      compounds into a meaningful cost difference. Shipped disabled; enable it
      once alibaba_region and bucket_name point at a real OSS bucket and a
      manual run has been verified."
    cron: "0 9 * * *"
    disabled: true
    inputs:
      alibaba_region: "cn-hangzhou"
      bucket_name: "REPLACE-WITH-YOUR-BUCKET-NAME"
      object_prefix: ""
      max_object_age_days: 90
      max_files_to_scan: 1000

inputs:
  - id: alibaba_region
    type: STRING
    displayName: "Alibaba Cloud Region"
    defaults: "cn-hangzhou"
    description: "Passed to the List task's own region property; the OSS endpoint is
      derived from it unless endpointOverride is set."

  - id: bucket_name
    type: STRING
    displayName: "OSS Bucket Name"
    defaults: "REPLACE-WITH-YOUR-BUCKET-NAME"
    description: "Passed to the List task's own bucket property. Must be replaced
      with a real bucket name - there is no safe universal default."

  - id: object_prefix
    type: STRING
    displayName: "Object Key Prefix"
    defaults: ""
    description: "Passed to the List task's own prefix property. Leave empty to scan
      the whole bucket, or scope the audit to one logical folder."

  - id: max_object_age_days
    type: INT
    displayName: "Max Standard-Tier Object Age (days)"
    defaults: 90
    description: "A Standard-storage-class object older than this (by its
      lastModified timestamp) is flagged as storage-class drift - it should
      typically have transitioned to a cheaper tier (Infrequent Access, Archive,
      or Cold Archive) under a lifecycle policy by now."

  - id: max_files_to_scan
    type: INT
    displayName: "Max Files To Scan"
    defaults: 1000
    description: "Passed to the List task's own maxFiles property, capped at 1000
      (the task's own documented hard limit). A warning is logged by the task
      itself if more objects match than this cap allows."

tasks:
  - id: probe_list_objects
    type: io.kestra.plugin.alibaba.oss.List
    description: "Lists up to max_files_to_scan objects (verified from the task's
      own Javadoc: capped between 1 and 1000) under object_prefix, returning
      each object's key, size, lastModified, and storageClass - a read-only
      call, it cannot delete, copy, or modify any object."
    accessKeyId: "{{ secret('ALIBABA_ACCESS_KEY_ID') }}"
    accessKeySecret: "{{ secret('ALIBABA_ACCESS_KEY_SECRET') }}"
    region: "{{ inputs.alibaba_region }}"
    bucket: "{{ inputs.bucket_name }}"
    prefix: "{{ inputs.object_prefix }}"
    maxFiles: "{{ inputs.max_files_to_scan }}"

  - id: evaluate_storage_class_drift
    type: io.kestra.plugin.core.flow.If
    description: "STORAGE_CLASS_DRIFT_DETECTED whenever at least one returned object
      (verified output fields storageClass and lastModified from the List task's
      own OssObject model) is still on the Standard class and older than
      max_object_age_days. fromdateiso8601 and now are both real, documented jq
      builtins used elsewhere in this repository's AWS CLI-based sentinels for
      the same date-window pattern."
    condition: "{{ (outputs.probe_list_objects.objects ?? []) |
      jq('map(select(.storageClass == \"Standard\" and (now - (.lastModified |
      fromdateiso8601)) > (' ~ inputs.max_object_age_days ~ ' * 86400))) |
      length') > 0 }}"
    then:
      - id: alert_storage_class_drift
        type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
        description: "Always alert on STORAGE_CLASS_DRIFT_DETECTED, listing each flagged
          object's key, size in bytes, and age in whole days so the team can see
          exactly what should be transitioned or whether a lifecycle rule is
          missing or misconfigured."
        url: "{{ secret('SLACK_WEBHOOK_URL') }}"
        payload: |
          {{ {
            "text": "Alibaba OSS storage class sentinel: STORAGE_CLASS_DRIFT_DETECTED in bucket " ~ inputs.bucket_name ~ " (" ~ inputs.alibaba_region ~ "). Objects still on Standard tier past " ~ inputs.max_object_age_days ~ " days: " ~ ((outputs.probe_list_objects.objects ?? []) | jq('map(select(.storageClass == \"Standard\" and (now - (.lastModified | fromdateiso8601)) > (' ~ inputs.max_object_age_days ~ ' * 86400))) | map("\(.key) (\(.size) bytes, \(((now - (.lastModified | fromdateiso8601)) / 86400) | floor) days old)") | join("; ")')) ~ ". Execution " ~ execution.id ~ "."
          } | toJson }}
    else:
      - id: log_storage_class_healthy
        type: io.kestra.plugin.core.log.Log
        message: "HEALTHY: no Standard-tier objects in oss://{{ inputs.bucket_name }}/{{
          inputs.object_prefix }} exceed {{ inputs.max_object_age_days }} days
          old, out of {{ (outputs.probe_list_objects.objects ?? []) | length }}
          objects scanned."

  - id: log_audit_complete
    type: io.kestra.plugin.core.log.Log
    description: "Final status line for every run, independent of which branch fired."
    message: "Storage class audit complete for oss://{{ inputs.bucket_name }}/{{
      inputs.object_prefix }}. objects_scanned={{
      (outputs.probe_list_objects.objects ?? []) | length }}."

errors:
  - id: alert_execution_failure
    type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
    description: "Alert when the flow itself fails outside the handled
      HEALTHY/STORAGE_CLASS_DRIFT_DETECTED branches, for example a bucket that
      does not exist, or invalid credentials."
    url: "{{ secret('SLACK_WEBHOOK_URL') }}"
    payload: |
      {{ {
        "text": "Alibaba OSS storage class sentinel FAILED in execution " ~ execution.id ~ " for bucket " ~ inputs.bucket_name ~ " (" ~ inputs.alibaba_region ~ "). List did not complete; investigate before trusting the last known HEALTHY/STORAGE_CLASS_DRIFT_DETECTED status."
      } | toJson }}

An Alibaba Cloud OSS lifecycle rule that is missing, misconfigured, or scoped to the wrong prefix lets objects sit on the expensive Standard storage class indefinitely instead of transitioning to Infrequent Access, Archive, or Cold Archive - a cost drift that is invisible until a monthly bill review. This blueprint lists objects on a schedule and alerts Slack with the exact keys, sizes, and ages of anything still on Standard well past a configured age threshold.

How it works

periodic_storage_class_check (io.kestra.plugin.core.trigger.Schedule) runs once a day, explicitly passing alibaba_region, bucket_name, object_prefix, max_object_age_days, and max_files_to_scan through its own inputs: block so the flow is self-contained when the trigger fires. Shipped disabled so a manual run can be verified first.
probe_list_objects (io.kestra.plugin.alibaba.oss.List) lists up to max_files_to_scan objects under object_prefix, capturing each object's key, size, lastModified, and storageClass.
evaluate_storage_class_drift (io.kestra.plugin.core.flow.If) branches on whether any returned object is still Standard class and older than max_object_age_days.
- then (STORAGE_CLASS_DRIFT_DETECTED): alert_storage_class_drift reports every flagged object's key, size, and age to Slack.
- else (HEALTHY): log_storage_class_healthy records the clean pass along with how many objects were scanned.
log_audit_complete always runs last, printing the total objects scanned regardless of which branch fired.
The errors block alerts Slack separately if the flow itself fails outright - a missing bucket or invalid credentials.

What you get

A cost governance signal built on OSS's own storageClass and lastModified fields, not a separately-maintained spreadsheet of what should have transitioned by when.
The exact key, size, and age of every drifted object in the Slack alert itself, so a lifecycle policy fix can be scoped precisely instead of guessing which prefix is affected.
A worked example of io.kestra.plugin.alibaba.oss.List combined with jq's fromdateiso8601/now date-window pattern, reusing the same approach this repository's AWS Certificate Manager sentinel established, applied here to a completely different plugin with no prior blueprint coverage at all.

Who it's for

Platform and FinOps teams running workloads on Alibaba Cloud who want to catch storage-class drift before it shows up as an unexplained cost increase.
Teams who configured an OSS lifecycle rule once and have no ongoing way to verify it is still matching the objects it was meant to, especially after a prefix or naming convention changes.
Anyone evaluating the Alibaba plugin who wants a worked example of its OSS List task used for governance monitoring rather than for basic data ingestion.

Why orchestrate this with Kestra

OSS lifecycle rules run silently and have no built-in "alert me if objects aren't transitioning as expected" primitive - a rule that stopped matching due to a prefix change produces no error, just a growing Standard-tier footprint. Kestra supplies the schedule, the HEALTHY/STORAGE_CLASS_DRIFT_DETECTED classification as a first-class branch, and an execution history showing exactly when drift was first detected and for which objects.

Prerequisites

An Alibaba Cloud account with an OSS bucket containing objects (update alibaba_region, bucket_name, object_prefix).
Alibaba Cloud AccessKey credentials with read access to the bucket - entirely read-only, cannot delete, copy, or modify any object.
A Slack incoming webhook for alerts.

Local testing: Alibaba Cloud publishes its own lightweight, Ruby-based OSS-Emulator for testing and debugging against the same OSS APIs, and the List task's own endpointOverride and pathStyleAccess properties are explicitly documented as being "useful with custom endpoints such as a local emulator." Exact Docker packaging and current maintenance status of that emulator were not independently confirmed; a real Alibaba Cloud account remains the most complete way to exercise this blueprint.

Secrets

ALIBABA_ACCESS_KEY_ID / ALIBABA_ACCESS_KEY_SECRET: credentials for io.kestra.plugin.alibaba.oss.List.
SLACK_WEBHOOK_URL: Slack incoming webhook used by alert_storage_class_drift and the errors block.
In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded - for example SECRET_ALIBABA_ACCESS_KEY_SECRET=$(echo -n '<your-secret-key>' | base64) - and read back in flows with {{ secret('NAME') }}.

Inputs

Input	Type	Default	Description
`alibaba_region`	STRING	`cn-hangzhou`	Alibaba Cloud region the OSS endpoint is derived from
`bucket_name`	STRING	`REPLACE-WITH-YOUR-BUCKET-NAME`	The target OSS bucket to audit
`object_prefix`	STRING	`""`	Optional key prefix to scope the audit
`max_object_age_days`	INT	`90`	Inactivity age threshold in days for Standard-tier objects
`max_files_to_scan`	INT	`1000`	Max objects returned by List (capped at 1000 limit)

Outputs

outputs.probe_list_objects.objects (list): every scanned object's key, size, etag, lastModified, and storageClass (verified fields from the List task's own OssObject model).

Quick start

Identify a real OSS bucket with objects on the Standard storage class.
Add the ALIBABA_ACCESS_KEY_ID, ALIBABA_ACCESS_KEY_SECRET, and SLACK_WEBHOOK_URL secrets.
Run the flow manually and confirm log_storage_class_healthy fires if nothing is drifted.
Temporarily lower max_object_age_days below the real age of a known Standard-tier object, and re-run to confirm alert_storage_class_drift fires with the correct key and age.
Enable periodic_storage_class_check once you trust the check against your real bucket.

How to extend

Loop over several bucket_name/object_prefix combinations with io.kestra.plugin.core.flow.Loop to audit multiple buckets or logical folders in one run.
Lower max_object_age_days progressively and track the flagged count over several executions (via Kestra's own execution history) to see whether drift is growing or shrinking after a lifecycle policy fix.
Add a remediation branch using io.kestra.plugin.alibaba.oss.Copy to re-upload a drifted object with an explicit storage class override, behind an explicit approval gate - deliberately kept out of this sentinel so the monitoring path never changes storage class on its own.

Pitfalls

List caps results at maxFiles (hard limit 1000, default 25) - verified directly from the task's own Javadoc, which logs a warning when more objects match. A bucket or prefix with more objects than max_files_to_scan will only have its first page audited; this blueprint does not paginate beyond that cap.
The regular-expression-based regexp filter is intentionally unused here - the task's own Javadoc notes it is time-boxed against catastrophic backtracking and limited to 256 characters, which is a safety feature worth knowing about if you extend this blueprint to filter by key pattern.
This blueprint is UNTESTED against a live Alibaba Cloud account. The task class, its OssObject output fields (key, size, etag, lastModified, storageClass), and the OSS storage class names are taken verbatim from the plugin's own source - but no Kestra engine was run to execute this flow end to end.

Alibaba OSS Storage Class Cost Governance Sentinel

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Inputs

Outputs

Quick start

How to extend

Pitfalls

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

New to Kestra?

Alibaba OSS Storage Class Cost Governance Sentinel

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Inputs

Outputs

Quick start

How to extend

Pitfalls

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

More Infrastructure Blueprints

MongoDB Document Validation Drift Repair Guard

Reconcile MLflow Registry Drift Against Deployed Serving Models

MongoDB Duplicate Document Dedup Guard

MongoDB Dangling Reference Orphan Cleanup Guard

Back up a MongoDB collection and prove it restores, every night

ML Training Dataset Security Guard and Poisoning Prevention Gate

New to Kestra?