Huawei GeminiDB TTL Hygiene Sentinel

GeminiDB for NoSQL is Huawei Cloud's DynamoDB-compatible database, and like DynamoDB itself, a time-to-live attribute only works if every write path actually sets it. A single service that forgets, or a migration that backfilled rows without one, leaves a growing set of items that will never expire and that nothing in GeminiDB itself will ever flag. This blueprint scans for exactly that: items with no expiry attribute at all, or one that has already passed, and treats the count as a governance signal.

This blueprint was created by Mohit Jeswani.

How it works

1. scan_missing_ttl (io.kestra.plugin.huawei.geminidb.Scan) reads a single response page of table_name, server-side filtered to items where ttl_attribute is absent (attribute_not_exists) or its value is less than today's date, read as FETCH so the matching rows are available for the alert. Read-only; nothing is written.
2. evaluate (io.kestra.plugin.core.output.OutputValues) compares the matched count against max_missing_ttl_count.
3. route (io.kestra.plugin.core.flow.If):
   • compliant: logs that nothing in the scanned page is missing or past its expiry.
   • alert: posts the flagged count to Slack.
4. summary logs the flagged count and severity, closing the governance cycle.
5. errors alerts Slack if the flow itself fails, for example wrong database-account credentials or an unreachable endpoint.

What you get

• An early signal for unbounded GeminiDB table growth, the same cost and compliance problem DynamoDB users hit when TTL silently stops being set on new writes.
• A check that reads cleanly whether it is pointed at a real GeminiDB instance or a local DynamoDB-compatible endpoint, since Scan talks the DynamoDB wire protocol either way.
• A record, in Kestra's execution history, of the flagged count on every run, not just a point-in-time console query.

Who it's for

• Teams running GeminiDB for NoSQL who rely on a TTL-style attribute for data retention and want to know when a write path stops setting it, rather than discovering it in next quarter's bill.
• Platform teams evaluating the Huawei Cloud plugin who want a worked example of Scan with a server-side filterExpression, distinct from reading the whole table into memory and filtering client-side.

Why orchestrate this with Kestra

GeminiDB enforces nothing about whether a TTL-style attribute is actually set, and its own console has no alert for a growing count of unexpiring items. Kestra turns that into a scheduled check with the threshold as a flow input, a direct Slack page naming the count, and a full history of every run so a slow regression in write-path hygiene is visible before it is a storage bill.

Prerequisites

• A Huawei Cloud GeminiDB for NoSQL instance, or for local testing, the open-source amazon/dynamodb-local image: GeminiDB's data plane is DynamoDB wire-compatible, so Scan works against either one by only changing endpoint.
• The instance's database account: accessKeyId is always the literal rwuser, and secretAccessKey is the instance admin password set when it was purchased (not a Huawei IAM key - IAM credentials are rejected by this data plane with an opaque auth error).
• At least one item in table_name carrying ttl_attribute as an ISO yyyy-MM-dd (or longer ISO) string, to confirm the filter actually excludes compliant items.
• A Slack incoming webhook URL.

Local testing: docker run -d --name dynamodb-local -p 8000:8000 amazon/dynamodb-local docker network connect YOUR_KESTRA_NETWORK dynamodb-local (check existing networks first with docker network ls; only needed if the Kestra Worker runs in a separate Docker network than this container; set endpoint to http://dynamodb-local:8000 from inside that network, or http://localhost:8000 from the host). accessKeyId/secretAccessKey can be any non-empty strings against dynamodb-local, which does not validate them.

Secrets

• GEMINIDB_ADMIN_PASSWORD: the instance's database admin password, paired with the literal accessKeyId: rwuser. In Kestra OSS, provide it as the base64-encoded environment variable SECRET_GEMINIDB_ADMIN_PASSWORD.
• SLACK_WEBHOOK_URL: used by alert and the errors block. In Kestra OSS, provide it as the base64-encoded environment variable SECRET_SLACK_WEBHOOK_URL.

Inputs

Input	Type	Default	Purpose
endpoint	String	http://localhost:8000	DynamoDB-compatible connection address
region	String	cn-north-1	SigV4 signing region only, does not affect routing
table_name	String	<your-geminidb-table>	Table to scan
ttl_attribute	String	expires_at	Attribute expected to hold an ISO expiry date
scan_limit	Integer	1000	Items evaluated in the single response page (1-1000)
max_missing_ttl_count	Integer	1	Flagged-item count that triggers the Slack alert

Outputs

• {{ outputs.scan_missing_ttl.rows }}: the matched items themselves, present because fetchType: FETCH is used.
• {{ outputs.scan_missing_ttl.size }}: how many items in the scanned page matched the filter.
• {{ outputs.evaluate.values.severity }}: OK or TTL_HYGIENE_BREACH.

Pitfalls

• Scan reads a single response page only; it never follows LastEvaluatedKey. A table larger than scan_limit items is not fully covered by one run, and the task logs when the response was truncated so this isn't silent. Narrow with a smaller, sharded table_name strategy or accept partial coverage per run.
• The comparison {{ inputs.ttl_attribute }} < :cutoff only behaves correctly if every item actually stores ttl_attribute as a string, and specifically one that starts with yyyy-MM-dd. GeminiDB (like the AWS DynamoDB task this plugin ports from) stores numeric values written through its own PutItem task as strings already, but data loaded by other means could store it as a native number (N) attribute, which this filter will not match correctly against a string :cutoff.
• filterExpression is a server-side filter applied after items are read; it does not reduce GeminiDB's read cost or the amount of data scanned, only the amount returned.
• This Scan task has no expressionAttributeNames property. If ttl_attribute collides with a DynamoDB reserved word, the filter expression fails outright rather than needing an alias.
• accessKeyId/secretAccessKey here are the instance's database account, never a Huawei IAM AK/SK pair - a perfectly valid IAM key is rejected with a bare AccessDeniedException that gives no indication the credential type itself is the problem.

Quick start

1. Start amazon/dynamodb-local with the command above, or point at a real GeminiDB instance.
2. Add the GEMINIDB_ADMIN_PASSWORD and SLACK_WEBHOOK_URL secrets.
3. Create a table and put a few items, some with expires_at set to a past date and some with it omitted entirely.
4. Run the flow with the defaults and confirm alert fires with the count you expect.
5. Remove or correct the offending items and re-run to confirm compliant logs instead.
6. Enable the daily trigger once endpoint and table_name point at the real instance.

How to extend

• Add a remediation branch, gated behind the auto_remediate/dry_run pair used elsewhere in this repo's guard blueprints, that loops over outputs.scan_missing_ttl.rows and calls io.kestra.plugin.huawei.geminidb.PutItem to backfill a default ttl_attribute on each flagged item rather than deleting it - left out of this blueprint since PutItem replaces the whole item, and doing that safely needs the exact attribute types already in use in your table.
• Loop over several table_name values with io.kestra.plugin.core.flow.Loop to cover more than one table in a single scheduled run.
• Track missing_ttl_count over successive runs to catch a slow regression, not just a one-time breach.

Links

• Huawei Cloud plugin: https://kestra.io/plugins/plugin-huawei
• GeminiDB for NoSQL (DynamoDB-compatible API): https://www.huaweicloud.com/intl/en-us/product/geminidb.html
• DynamoDB Local Docker image: https://hub.docker.com/r/amazon/dynamodb-local
• Slack incoming webhooks: https://api.slack.com/messaging/webhooks