id: phpipam-stale-ip-reservation-reclaim-guard
namespace: company.team
description: >-
Reclaim phpIPAM address reservations whose host is gone. An address is deleted
only when it is unreachable now AND a second operator-chosen signal agrees,
past a grace window, outside every protected range, under a per-run ceiling,
behind a Pause. A single failed ping never frees an address.
labels:
owner: network-operations
system: phpipam
inputs:
- id: phpipam_base_url
type: STRING
displayName: phpIPAM base URL
description: Root URL of the phpIPAM instance with no trailing slash. Used by
every phpIPAM task.
defaults: http://localhost
- id: phpipam_app_id
type: STRING
displayName: phpIPAM API app id
description: The API application identifier created under Administration, API in
phpIPAM.
defaults: myapp
- id: phpipam_app_token
type: STRING
displayName: phpIPAM API app token
description: >-
App code token for the API application. Leave the default so the value
comes from the PHPIPAM_APP_TOKEN secret. Override it only in a throwaway
lab.
defaults: "{{ secret('PHPIPAM_APP_TOKEN') }}"
- id: subnet_id
type: STRING
displayName: Subnet id
description: Numeric phpIPAM id of the subnet to reclaim. One subnet per run
keeps the blast radius small.
defaults: "1"
- id: second_signal
type: SELECT
displayName: Second signal required before a delete
description: >-
The second independent condition an address must also meet.
OWNER_INVENTORY means the address is absent from an owner source fetched
over HTTP. PHPIPAM_LAST_SEEN means phpIPAM itself last saw the host longer
ago than stale_after_days. DECOMMISSIONED_TAG means the address carries
the tag id in decommissioned_tag_id.
values:
- OWNER_INVENTORY
- PHPIPAM_LAST_SEEN
- DECOMMISSIONED_TAG
defaults: OWNER_INVENTORY
- id: owner_inventory_url
type: STRING
displayName: Owner inventory URL
description: >-
HTTP endpoint returning any JSON document that lists the addresses still
owned by a live host. Every IPv4 literal anywhere in the document counts
as owned. Required for OWNER_INVENTORY, ignored otherwise.
defaults: ""
- id: stale_after_days
type: INT
displayName: Last-seen staleness threshold in days
description: >-
For PHPIPAM_LAST_SEEN, how long phpIPAM must have failed to see a host
before its last-seen stamp counts as the second signal.
defaults: 30
- id: decommissioned_tag_id
type: INT
displayName: Decommissioned tag id
description: >-
For DECOMMISSIONED_TAG, the phpIPAM address tag id that means an operator
already marked the host retired. Stock tags are 1 Offline, 2 Used, 3
Reserved, 4 DHCP, so a custom tag is normal here.
defaults: 5
- id: grace_hours
type: INT
displayName: Grace window in hours
description: >-
An address whose phpIPAM record changed more recently than this is never a
candidate, however dead it looks. This is what protects a reservation cut
minutes ago for a host still booting.
defaults: 72
- id: protected_tag_ids
type: STRING
displayName: Protected tag ids
description: >-
Comma separated phpIPAM tag ids that are never reclaimed. Defaults cover 3
Reserved plus 4 DHCP, so DHCP pool entries plus reserved infrastructure
stay put.
defaults: "3,4"
- id: protected_ranges
type: STRING
displayName: Protected address ranges
description: >-
Comma separated allowlist of single addresses or inclusive first-last
ranges that are never reclaimed, for example 10.0.0.1,10.0.0.20-10.0.0.40.
Addresses flagged as a gateway in phpIPAM or carrying its exclude-ping
flag are protected without being listed here.
defaults: ""
- id: max_deletes_per_run
type: INT
displayName: Per-run delete ceiling
description: >-
Hard ceiling on candidates in one run. More candidates than this fails the
run and deletes nothing, because a whole-network outage looks exactly like
every host being dead.
defaults: 5
- id: ping_count
type: INT
displayName: Probes per address
description: How many ICMP echo requests each address gets before it is called
unreachable.
defaults: 2
- id: ping_timeout_seconds
type: INT
displayName: Probe reply timeout in seconds
description: How long each echo request waits for a reply.
defaults: 2
- id: probe_image
type: STRING
displayName: Probe container image
description: Image used for the ICMP sweep. Needs a working ping binary only.
defaults: alpine:3.20
- id: probe_network_mode
type: STRING
displayName: Probe container network
description: >-
Docker network the probe and the inventory fetch join, so they reach the
subnet being audited. Leave empty when the Kestra worker already sits on
that network.
defaults: ""
- id: insecure_tls
type: BOOL
displayName: Skip TLS verification
description: Set true only for a lab instance with a self-signed certificate.
defaults: false
- id: dry_run
type: BOOL
displayName: Dry run
description: True reports the candidates and deletes nothing. The flow ships this way.
defaults: true
- id: auto_remediate
type: BOOL
displayName: Allow reclaim
description: >-
Must be true, with dry_run false and the Pause approved, before any
address is deleted. The flow ships this way.
defaults: false
tasks:
- id: read_subnet_before
type: io.kestra.plugin.phpipam.ipam.subnet.Get
description: Read the subnet so the run knows its mask. Capacity plus
utilisation are derived from that mask and the record count, before
anything is touched.
baseUrl: "{{ inputs.phpipam_base_url }}"
appId: "{{ inputs.phpipam_app_id }}"
auth:
appToken: "{{ inputs.phpipam_app_token }}"
insecureTls: "{{ inputs.insecure_tls }}"
subnetId: "{{ inputs.subnet_id }}"
- id: read_addresses_before
type: io.kestra.plugin.phpipam.ipam.address.List
description: Read every address phpIPAM holds for this subnet, with its tag,
owner, gateway flag, exclude-ping flag, last-seen stamp and record edit
date.
baseUrl: "{{ inputs.phpipam_base_url }}"
appId: "{{ inputs.phpipam_app_id }}"
auth:
appToken: "{{ inputs.phpipam_app_token }}"
insecureTls: "{{ inputs.insecure_tls }}"
subnetId: "{{ inputs.subnet_id }}"
fetchType: FETCH
- id: probe_reachability
type: io.kestra.plugin.scripts.shell.Commands
description: ICMP sweep of every address in the subnet. This is signal one and
on its own it is never enough to free an address.
containerImage: "{{ inputs.probe_image }}"
taskRunner:
type: io.kestra.plugin.scripts.runner.docker.Docker
networkMode: "{{ inputs.probe_network_mode }}"
inputFiles:
addresses.json: "{{ outputs.read_addresses_before.addresses | toJson }}"
outputFiles:
- reachability.json
env:
PING_COUNT: "{{ inputs.ping_count }}"
PING_TIMEOUT: "{{ inputs.ping_timeout_seconds }}"
commands:
- |
set -u
IPS=$(grep -oE '"ip"[[:space:]]*:[[:space:]]*"[0-9.]+"' addresses.json | grep -oE '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' || true)
DEADLINE=$(( PING_COUNT * PING_TIMEOUT + 1 ))
UP=0; DOWN=0; FIRST=1
printf '{' > reachability.json
for IP in $IPS; do
if ping -c "$PING_COUNT" -W "$PING_TIMEOUT" -w "$DEADLINE" "$IP" >/dev/null 2>&1; then
STATE=true; UP=$(( UP + 1 ))
else
STATE=false; DOWN=$(( DOWN + 1 ))
fi
[ "$FIRST" -eq 1 ] || printf ',' >> reachability.json
printf '"%s":%s' "$IP" "$STATE" >> reachability.json
FIRST=0
echo "probe $IP reachable=$STATE"
done
printf '}' >> reachability.json
echo "probe finished: up=$UP down=$DOWN"
- id: select_candidates
type: io.kestra.plugin.scripts.python.Script
description: >-
Correlate the phpIPAM records with the sweep. An address becomes a
candidate only when it is unreachable AND the chosen second signal agrees
AND its record is older than the grace window AND it sits outside every
protected range. Everything else is spared with a stated reason.
containerImage: python:3.12-slim
taskRunner:
type: io.kestra.plugin.scripts.runner.docker.Docker
networkMode: "{{ inputs.probe_network_mode }}"
inputFiles:
addresses.json: "{{ outputs.read_addresses_before.addresses | toJson }}"
subnet.json: "{{ outputs.read_subnet_before.subnet | toJson }}"
reachability.json: "{{ outputs.probe_reachability.outputFiles['reachability.json'] }}"
env:
SECOND_SIGNAL: "{{ inputs.second_signal }}"
OWNER_INVENTORY_URL: "{{ inputs.owner_inventory_url }}"
STALE_AFTER_DAYS: "{{ inputs.stale_after_days }}"
DECOMMISSIONED_TAG_ID: "{{ inputs.decommissioned_tag_id }}"
GRACE_HOURS: "{{ inputs.grace_hours }}"
PROTECTED_TAG_IDS: "{{ inputs.protected_tag_ids }}"
PROTECTED_RANGES: "{{ inputs.protected_ranges }}"
MAX_DELETES_PER_RUN: "{{ inputs.max_deletes_per_run }}"
script: |
import json, os, re, sys, urllib.request
from datetime import datetime, timezone
EPOCH_SENTINELS = ("1970-01-01", "0000-00-00", "")
def pick(row, *names):
for n in names:
if n in row and row[n] is not None:
return row[n]
return None
def to_int(value, default=0):
try:
return int(str(value).strip())
except (TypeError, ValueError):
return default
def ip_to_long(text):
parts = str(text).split(".")
if len(parts) != 4:
return None
try:
octets = [int(p) for p in parts]
except ValueError:
return None
if any(o < 0 or o > 255 for o in octets):
return None
return (octets[0] << 24) + (octets[1] << 16) + (octets[2] << 8) + octets[3]
def parse_stamp(value):
text = str(value or "").strip()
if not text or text[:10] in EPOCH_SENTINELS:
return None
text = text.replace("T", " ").split(".")[0]
try:
return datetime.strptime(text[:19], "%Y-%m-%d %H:%M:%S").replace(tzinfo=timezone.utc)
except ValueError:
return None
def parse_protected(spec):
spans = []
for chunk in str(spec or "").split(","):
chunk = chunk.strip()
if not chunk:
continue
if "-" in chunk:
lo, hi = [c.strip() for c in chunk.split("-", 1)]
else:
lo = hi = chunk
lo_long, hi_long = ip_to_long(lo), ip_to_long(hi)
if lo_long is None or hi_long is None:
raise SystemExit(f"protected_ranges entry is not an IPv4 range: {chunk}")
spans.append((min(lo_long, hi_long), max(lo_long, hi_long)))
return spans
signal = os.environ["SECOND_SIGNAL"]
grace_hours = to_int(os.environ["GRACE_HOURS"], 72)
stale_days = to_int(os.environ["STALE_AFTER_DAYS"], 30)
decommissioned_tag = to_int(os.environ["DECOMMISSIONED_TAG_ID"], -1)
ceiling = to_int(os.environ["MAX_DELETES_PER_RUN"], 0)
protected_tags = {t.strip() for t in os.environ["PROTECTED_TAG_IDS"].split(",") if t.strip()}
protected_spans = parse_protected(os.environ["PROTECTED_RANGES"])
def capacity(mask):
bits = to_int(mask, 32)
hosts = 2 ** (32 - bits)
return hosts - 2 if bits <= 30 else hosts
def free_percent(used, total):
return round((total - used) * 100.0 / total, 2) if total else 0.0
addresses = json.load(open("addresses.json")) or []
subnet = json.load(open("subnet.json")) or {}
reachable = json.load(open("reachability.json")) or {}
used_before = len(addresses)
max_hosts = capacity(subnet.get("mask"))
free_pct_before = free_percent(used_before, max_hosts)
owned = set()
if signal == "OWNER_INVENTORY":
url = os.environ["OWNER_INVENTORY_URL"].strip()
if not url:
raise SystemExit("second_signal is OWNER_INVENTORY but owner_inventory_url is empty")
with urllib.request.urlopen(url, timeout=20) as response:
body = response.read(4_000_000).decode("utf-8", "replace")
json.loads(body)
owned = set(re.findall(r"\b\d{1,3}(?:\.\d{1,3}){3}\b", body))
print(f"owner inventory listed {len(owned)} addresses")
now = datetime.now(timezone.utc)
candidates, spared = [], []
for row in addresses:
ip = str(pick(row, "ip", "ip_addr") or "")
addr_id = str(pick(row, "id") or "")
tag = str(to_int(pick(row, "tag", "state"), -1))
gateway = to_int(pick(row, "isGateway", "is_gateway"), 0) == 1
no_ping = to_int(pick(row, "excludePing", "exclude_ping"), 0) == 1
ip_long = ip_to_long(ip)
in_protected = ip_long is not None and any(lo <= ip_long <= hi for lo, hi in protected_spans)
answered = bool(reachable.get(ip, False))
edited = parse_stamp(pick(row, "editDate", "edit_date"))
last_seen = parse_stamp(pick(row, "lastSeen", "last_seen"))
if gateway:
reason = "phpIPAM marks it a gateway"
elif no_ping:
reason = "phpIPAM exclude-ping flag is set, so silence proves nothing"
elif tag in protected_tags:
reason = f"tag {tag} is in protected_tag_ids"
elif in_protected:
reason = "address sits inside protected_ranges"
elif answered:
reason = "answers ICMP now"
elif edited is not None and (now - edited).total_seconds() < grace_hours * 3600:
reason = f"record changed inside the {grace_hours}h grace window"
elif signal == "OWNER_INVENTORY" and ip in owned:
reason = "unreachable but still listed in the owner inventory, one signal only"
elif signal == "PHPIPAM_LAST_SEEN" and last_seen is None:
reason = "unreachable but phpIPAM never recorded a last-seen stamp, one signal only"
elif signal == "PHPIPAM_LAST_SEEN" and (now - last_seen).days < stale_days:
reason = f"unreachable but phpIPAM saw it {(now - last_seen).days}d ago, one signal only"
elif signal == "DECOMMISSIONED_TAG" and tag != str(decommissioned_tag):
reason = f"unreachable but tag {tag} is not the decommissioned tag, one signal only"
else:
candidates.append({"id": addr_id, "ip": ip, "hostname": pick(row, "hostname") or "", "tag": tag})
print(f"CANDIDATE {ip} id={addr_id} host={pick(row, 'hostname') or '-'}")
continue
spared.append({"ip": ip, "id": addr_id, "reason": reason})
print(f"SPARED {ip} id={addr_id} {reason}")
candidates.sort(key=lambda c: ip_to_long(c["ip"]) or 0)
breached = len(candidates) > ceiling
print(f"subnet {subnet.get('subnet')}/{subnet.get('mask')} used={used_before}/{max_hosts} free={free_pct_before}%")
print(f"candidates={len(candidates)} spared={len(spared)} ceiling={ceiling} breached={breached}")
result = {
"finding_count": len(candidates),
"spared_count": len(spared),
"ceiling_breached": breached,
"candidate_ids_json": json.dumps([c["id"] for c in candidates]),
"candidate_ips": ", ".join(c["ip"] for c in candidates) or "none",
"keep_ids_json": json.dumps([s["id"] for s in spared]),
"spared_summary": "; ".join(f"{s['ip']} {s['reason']}" for s in spared) or "none",
"used_before": used_before,
"max_hosts": max_hosts,
"free_percent_before": str(free_pct_before),
}
sys.stdout.write("::" + json.dumps({"outputs": result}) + "::\n")
- id: log_findings
type: io.kestra.plugin.core.log.Log
description: One line an operator can act on, naming the real candidate
addresses plus every spare decision and its reason.
message: |
phpIPAM subnet {{ inputs.subnet_id }}: {{ outputs.select_candidates.vars.used_before }}/{{ outputs.select_candidates.vars.max_hosts }} addresses used, {{ outputs.select_candidates.vars.free_percent_before }}% free.
Reclaim candidates ({{ outputs.select_candidates.vars.finding_count }}), unreachable plus {{ inputs.second_signal }}: {{ outputs.select_candidates.vars.candidate_ips }}
Spared ({{ outputs.select_candidates.vars.spared_count }}): {{ outputs.select_candidates.vars.spared_summary }}
dry_run={{ inputs.dry_run }} auto_remediate={{ inputs.auto_remediate }} ceiling={{ inputs.max_deletes_per_run }}
- id: refuse_on_ceiling
type: io.kestra.plugin.core.flow.If
description: More candidates than the ceiling means the probe itself is suspect,
so the run fails and nothing is deleted.
condition: "{{ outputs.select_candidates.vars.ceiling_breached }}"
then:
- id: fail_on_ceiling
type: io.kestra.plugin.core.execution.Fail
errorMessage: >-
Refusing to reclaim: {{ outputs.select_candidates.vars.finding_count
}} candidates exceed the per-run ceiling of {{
inputs.max_deletes_per_run }} in subnet {{ inputs.subnet_id }}. A
broken probe or a network outage looks exactly like this. Review {{
outputs.select_candidates.vars.candidate_ips }} by hand, then raise
max_deletes_per_run only if every one is genuinely dead.
- id: reclaim_gate
type: io.kestra.plugin.core.flow.If
description: Writes happen only when auto_remediate is true, dry_run is false
and there is something to reclaim.
condition: "{{ inputs.auto_remediate and (inputs.dry_run == false) and
(outputs.select_candidates.vars.finding_count > 0) }}"
then:
- id: confirm_reclaim
type: io.kestra.plugin.core.flow.Pause
description: A human approves the exact address list before any reservation is
deleted.
onResume:
- id: approved
type: BOOL
description: >-
Set true to delete the listed reservations from phpIPAM. Anything
else, including resuming with no answer, leaves the subnet
untouched.
- id: apply_if_approved
type: io.kestra.plugin.core.flow.If
description: An unanswered Pause leaves approved null, so the gate defaults to
refusing.
condition: "{{ outputs.confirm_reclaim.onResume.approved ?? false }}"
then:
- id: reclaim_addresses
type: io.kestra.plugin.core.flow.Loop
description: Delete each candidate reservation one at a time. Serial on purpose,
concurrent writes to one subnet race in phpIPAM.
values: "{{ outputs.select_candidates.vars.candidate_ids_json }}"
concurrencyLimit: 1
tasks:
- id: delete_address
type: io.kestra.plugin.phpipam.ipam.address.Delete
description: Remove one stale reservation, freeing the address for the next
allocation.
baseUrl: "{{ inputs.phpipam_base_url }}"
appId: "{{ inputs.phpipam_app_id }}"
auth:
appToken: "{{ inputs.phpipam_app_token }}"
insecureTls: "{{ inputs.insecure_tls }}"
addressId: "{{ item.value }}"
- id: read_addresses_after
type: io.kestra.plugin.phpipam.ipam.address.List
description: Read the subnet's address list again. The delete task going green
proves nothing on its own.
baseUrl: "{{ inputs.phpipam_base_url }}"
appId: "{{ inputs.phpipam_app_id }}"
auth:
appToken: "{{ inputs.phpipam_app_token }}"
insecureTls: "{{ inputs.insecure_tls }}"
subnetId: "{{ inputs.subnet_id }}"
fetchType: FETCH
- id: read_subnet_after
type: io.kestra.plugin.phpipam.ipam.subnet.Get
description: Read the subnet again after the deletes, so the after utilisation
is computed from the same mask.
baseUrl: "{{ inputs.phpipam_base_url }}"
appId: "{{ inputs.phpipam_app_id }}"
auth:
appToken: "{{ inputs.phpipam_app_token }}"
insecureTls: "{{ inputs.insecure_tls }}"
subnetId: "{{ inputs.subnet_id }}"
- id: verify_reclaim
type: io.kestra.plugin.scripts.python.Script
description: >-
Fail the run if any candidate survived the delete or if anything
the selector spared has disappeared. A refused write reads as
success at the task, so this re-read is the only thing that
decides whether the change landed.
containerImage: python:3.12-slim
inputFiles:
after.json: "{{ outputs.read_addresses_after.addresses | toJson }}"
subnet_after.json: "{{ outputs.read_subnet_after.subnet | toJson }}"
env:
CANDIDATE_IDS: "{{ outputs.select_candidates.vars.candidate_ids_json }}"
KEEP_IDS: "{{ outputs.select_candidates.vars.keep_ids_json }}"
USED_BEFORE: "{{ outputs.select_candidates.vars.used_before }}"
FREE_BEFORE: "{{ outputs.select_candidates.vars.free_percent_before }}"
script: |
import json, os, sys
def pick(row, *names):
for n in names:
if n in row and row[n] is not None:
return row[n]
return None
def capacity(mask):
try:
bits = int(str(mask).strip())
except (TypeError, ValueError):
return 0
hosts = 2 ** (32 - bits)
return hosts - 2 if bits <= 30 else hosts
after = json.load(open("after.json")) or []
subnet_after = json.load(open("subnet_after.json")) or {}
targeted = set(json.loads(os.environ["CANDIDATE_IDS"]))
keep = set(json.loads(os.environ["KEEP_IDS"]))
present = {str(pick(row, "id")) for row in after}
survivors = sorted(targeted & present)
lost = sorted(keep - present)
freed = sorted(targeted - present)
used_after = len(after)
max_hosts = capacity(subnet_after.get("mask"))
free_after = round((max_hosts - used_after) * 100.0 / max_hosts, 2) if max_hosts else 0.0
print(f"used before={os.environ['USED_BEFORE']} after={used_after} of {max_hosts}")
print(f"free percent before={os.environ['FREE_BEFORE']} after={free_after}")
print(f"freed={len(freed)} ids={freed}")
if survivors:
sys.exit(f"re-verify failed: {len(survivors)} targeted reservations are still in phpIPAM: {survivors}")
if lost:
sys.exit(f"re-verify failed: {len(lost)} spared reservations disappeared: {lost}")
print(f"re-verify passed: {len(freed)} addresses recovered, {len(keep)} spared records intact")
sys.stdout.write("::" + json.dumps({"outputs": {
"freed_count": len(freed),
"used_after": str(used_after),
"free_percent_after": str(free_after),
}}) + "::\n")
- id: log_reclaimed
type: io.kestra.plugin.core.log.Log
description: The number an operator cares about. Addresses recovered plus
utilisation before and after, read back from phpIPAM.
message: >-
Reclaimed {{ outputs.verify_reclaim.vars.freed_count }} stale
reservations from phpIPAM subnet {{ inputs.subnet_id }}. Used {{
outputs.select_candidates.vars.used_before }} to {{
outputs.verify_reclaim.vars.used_after }} of {{
outputs.select_candidates.vars.max_hosts }}. Free {{
outputs.select_candidates.vars.free_percent_before }}% to {{
outputs.verify_reclaim.vars.free_percent_after }}%. Freed
addresses: {{ outputs.select_candidates.vars.candidate_ips }}.
else:
- id: log_declined
type: io.kestra.plugin.core.log.Log
level: WARN
description: The approval was refused or left unanswered, so nothing was
deleted.
message: >-
Reclaim declined for phpIPAM subnet {{ inputs.subnet_id }}. No
address was deleted. {{
outputs.select_candidates.vars.finding_count }} candidates stay
recorded: {{ outputs.select_candidates.vars.candidate_ips }}.
else:
- id: log_read_only
type: io.kestra.plugin.core.log.Log
description: Read-only outcome. Either there was nothing to reclaim or the write
gate is closed.
message: >-
No reservation deleted in phpIPAM subnet {{ inputs.subnet_id }}.
dry_run={{ inputs.dry_run }}, auto_remediate={{ inputs.auto_remediate
}}, candidates={{ outputs.select_candidates.vars.finding_count }} ({{
outputs.select_candidates.vars.candidate_ips }}).
errors:
- id: alert_failure
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
description: Tell the network on-call that the reclaim guard failed, so a silent
failure cannot hide a refused delete.
url: "{{ secret('SLACK_WEBHOOK_URL') }}"
payload: |
{{ {"text": "phpIPAM stale reservation reclaim guard FAILED on subnet " ~ inputs.subnet_id ~ " (execution " ~ execution.id ~ "). No reclaim is confirmed. Check the execution logs before touching the subnet."} | toJson }}
triggers:
- id: weekly_reclaim_review
type: io.kestra.plugin.core.trigger.Schedule
description: Shipped disabled. Enable it once the thresholds match your network.
Keep the safe inputs.
cron: "15 6 * * 1"
disabled: true
inputs:
dry_run: true
auto_remediate: false