Segment Event Taxonomy and Property Contract Validator
Normalize Segment webhook batches, check event names and required property types against your taxonomy contract, and escalate violations to GitHub and Slack.
Compare a fund manager's 13F holdings quarter over quarter and get its new positions, exits and largest changes in Slack and the logs.
id: sec-13f-holdings-change-report
namespace: company.team
inputs:
- id: filer_ciks
type: ARRAY
itemType: STRING
displayName: Institutional managers (CIK)
description: SEC CIKs of the 13F filers to follow, for example 1067983 for
Berkshire Hathaway. Keep the trigger's `ciks` in sync for automatic runs.
defaults:
- "1067983"
- "1061768"
- "1656456"
- id: min_change_pct
type: FLOAT
displayName: Position change (%)
description: Report an existing position when its share count changed by at
least this percentage.
defaults: 20
- id: top_n
type: INT
displayName: Largest changes per manager
description: How many increases and how many decreases to list per manager, by
change in reported value.
defaults: 5
tasks:
- id: each_filer
type: io.kestra.plugin.core.flow.Loop
# On a filing-triggered run, only the managers that just filed.
values: "{{ trigger is defined and trigger.filings is defined ? (trigger.filings
| jq('[.[].cik] | unique') | first) : inputs.filer_ciks }}"
# One at a time: SEC allows 10 requests per second across all your tools.
concurrencyLimit: 1
outputs:
- id: filings
type: JSON
value: >-
{{ outputs.filings.rows | jq('[.[] | .cik + ";" + .accessionNumber +
";" + .filingDate + ";" + .form]') | first | toJson }}
tasks:
- id: filings
type: io.kestra.plugin.edgar.filings.List
description: The manager's 13F reports and amendments of the last 13 months,
notices included.
userAgent: "{{ kv('SEC_EDGAR_USER_AGENT', errorOnMissing=false) ?? '' }}"
cik: "{{ item.value }}"
forms:
- 13F-HR
- 13F-HR/A
- 13F-NT
- 13F-NT/A
startDate: "{{ now() | dateAdd(-13, 'MONTHS') | date('yyyy-MM-dd') }}"
- id: each_filing
type: io.kestra.plugin.core.flow.Loop
values: "{{ outputs.each_filer.outputs | jq('[.[].outputs.filings[]] | unique')
| first }}"
concurrencyLimit: 1
outputs:
- id: cover
type: STRING
value: "{{ outputs.download.primaryDocument.uri }}"
- id: holdings
type: STRING
value: "{{ outputs.download.exhibits | length > 0 ?
outputs.download.exhibits[0].uri : '' }}"
tasks:
- id: download
type: io.kestra.plugin.edgar.filings.Download
description: The cover page (amendment type, period) and the information table
(the holdings).
userAgent: "{{ kv('SEC_EDGAR_USER_AGENT', errorOnMissing=false) ?? '' }}"
cik: "{{ item.value | split(';') | first }}"
accessionNumber: "{{ (item.value | split(';'))[1] }}"
exhibits:
- INFORMATION TABLE
- id: parse
type: io.kestra.plugin.scripts.python.Script
description: One row per information-table entry, with the filing's period and
amendment type (standard library only).
containerImage: python:3.13-slim
# Each filing is saved as <CIK>_<accession number>_<filing date>_<form>, with 13F-HR/A written as 13F-HR-A.
inputFiles: |
{
{%- for f in outputs.each_filing.outputs ?? [] %}
"{{ f.item.value | replace({';': '_', '/': '-'}) }}.cover.xml": "{{ f.outputs.cover }}"{% if f.outputs.holdings is not empty %},
"{{ f.item.value | replace({';': '_', '/': '-'}) }}.table.xml": "{{ f.outputs.holdings }}"{% endif %}{{ loop.last ? '' : ',' }}
{%- endfor %}
}
script: |
import csv, glob, os
import xml.etree.ElementTree as ET
from datetime import datetime
def local(tag):
return tag.rsplit("}", 1)[-1]
def find(root, name):
return next((el.text.strip() for el in root.iter() if local(el.tag) == name and el.text and el.text.strip()), None)
FIELDS = ["filer_cik", "filer_name", "period", "accession_number", "form", "filing_date", "report_type", "amendment_type",
"cusip", "issuer", "title_of_class", "put_call", "share_type", "shares", "value_usd"]
with open("holdings.csv", "w", newline="") as out:
writer = csv.DictWriter(out, FIELDS)
writer.writeheader()
for cover_path in sorted(glob.glob("*.cover.xml")):
stem = cover_path[:-len(".cover.xml")]
cik, accession, filing_date, form = stem.split("_")
cover = ET.parse(cover_path).getroot()
period = datetime.strptime(find(cover, "periodOfReport"), "%m-%d-%Y").date().isoformat()
base = {"filer_cik": cik.lstrip("0"), "filer_name": find(cover, "name"), "period": period,
"accession_number": accession, "form": form.replace("-A", "/A"), "filing_date": filing_date,
"report_type": find(cover, "reportType"),
# RESTATEMENT replaces the report; NEW HOLDINGS adds positions to it (often ones first kept confidential).
"amendment_type": find(cover, "amendmentType")}
table = stem + ".table.xml"
if not os.path.exists(table):
# A 13F notice: the holdings are reported by another manager. Kept as one row without a security.
writer.writerow(base)
continue
for entry in ET.parse(table).getroot():
if local(entry.tag) != "infoTable":
continue
writer.writerow({**base,
"cusip": find(entry, "cusip").upper(),
"issuer": find(entry, "nameOfIssuer"),
"title_of_class": find(entry, "titleOfClass"),
"put_call": find(entry, "putCall"),
"share_type": find(entry, "sshPrnamtType"),
"shares": find(entry, "sshPrnamt"),
# In dollars since 2023 (before, in thousands).
"value_usd": find(entry, "value")})
outputFiles:
- holdings.csv
- id: changes
type: io.kestra.plugin.jdbc.duckdb.Query
description: Compares each manager's latest quarter with the quarter before, per
security.
inputFiles:
holdings.csv: "{{ outputs.parse.outputFiles['holdings.csv'] }}"
fetchType: STORE
sql: |
WITH rows AS (
SELECT * FROM read_csv('{{ workingDir }}/holdings.csv', header = true,
types = {'filer_cik': 'VARCHAR', 'filer_name': 'VARCHAR', 'period': 'DATE', 'accession_number': 'VARCHAR',
'form': 'VARCHAR', 'filing_date': 'DATE', 'amendment_type': 'VARCHAR', 'cusip': 'VARCHAR',
'issuer': 'VARCHAR', 'title_of_class': 'VARCHAR', 'put_call': 'VARCHAR', 'share_type': 'VARCHAR', 'report_type': 'VARCHAR',
'shares': 'DOUBLE', 'value_usd': 'DOUBLE'})
),
units AS (
-- Values are in dollars since 2023, but some managers still report thousands: a median price per share
-- under $1 across a filing's common-stock rows means thousands.
SELECT accession_number, median(value_usd / shares) < 1 AS in_thousands
FROM rows WHERE cusip IS NOT NULL AND share_type = 'SH' AND put_call IS NULL AND shares > 0
GROUP BY accession_number
),
scaled AS (
SELECT r.* REPLACE (CASE WHEN u.in_thousands THEN r.value_usd * 1000 ELSE r.value_usd END AS value_usd),
coalesce(u.in_thousands, false) AS in_thousands
FROM rows r LEFT JOIN units u USING (accession_number)
),
notices AS (
SELECT filer_cik, max(period) AS notice_period FROM rows WHERE cusip IS NULL GROUP BY filer_cik
),
filings AS (
SELECT DISTINCT filer_cik, period, accession_number, filing_date, amendment_type FROM scaled WHERE cusip IS NOT NULL
),
position_keys AS (
SELECT DISTINCT filer_cik, period, accession_number, filing_date, cusip, coalesce(put_call, '') AS put_call, share_type
FROM scaled WHERE cusip IS NOT NULL
),
overlap AS (
SELECT a.accession_number, count(*) AS positions,
count(*) FILTER (WHERE EXISTS (
SELECT 1 FROM position_keys b
WHERE b.filer_cik = a.filer_cik AND b.period = a.period AND b.filing_date < a.filing_date
AND b.cusip = a.cusip AND b.put_call = a.put_call AND b.share_type = a.share_type)) AS repeated
FROM position_keys a GROUP BY a.accession_number
),
originals AS (
SELECT filer_cik, period, arg_min(positions, filing_date) AS original_positions
FROM filings JOIN overlap USING (accession_number) GROUP BY filer_cik, period
),
typed AS (
-- The original report and a RESTATEMENT are complete reports. A NEW HOLDINGS amendment adds positions,
-- sometimes more shares of a security already reported, but some managers file the whole report again
-- under that label: one that repeats at least half of the original's positions is a complete report.
SELECT f.*, CASE WHEN f.amendment_type IS NULL OR f.amendment_type = 'RESTATEMENT' OR o.repeated >= 0.5 * g.original_positions
THEN 'complete' ELSE 'addition' END AS role
FROM filings f JOIN overlap o USING (accession_number) JOIN originals g USING (filer_cik, period)
),
base AS (
-- The latest complete report of each quarter, with the additions filed after it.
SELECT filer_cik, period, arg_max(accession_number, filing_date) AS base_accession, max(filing_date) AS base_date
FROM typed WHERE role = 'complete' GROUP BY filer_cik, period
),
kept AS (
SELECT s.* FROM scaled s JOIN base b ON s.accession_number = b.base_accession
UNION ALL
SELECT s.* FROM scaled s
JOIN typed t USING (accession_number)
JOIN base b ON b.filer_cik = s.filer_cik AND b.period = s.period
WHERE t.role = 'addition' AND t.filing_date >= b.base_date AND t.accession_number <> b.base_accession
),
periods AS (
SELECT filer_cik, period, dense_rank() OVER (PARTITION BY filer_cik ORDER BY period DESC) AS age
FROM (SELECT DISTINCT filer_cik, period FROM kept)
),
positions AS (
-- A manager reports one security on several rows (per sub-manager and discretion): add them up.
-- Options (put_call) and principal amounts (PRN) are kept apart from shares.
SELECT k.filer_cik, any_value(k.filer_name) AS filer_name, p.age, k.period, k.cusip,
coalesce(k.put_call, '') AS put_call, k.share_type,
any_value(k.issuer) AS issuer, any_value(k.title_of_class) AS title_of_class,
sum(k.shares) AS shares, sum(k.value_usd) AS value_usd, bool_or(k.in_thousands) AS in_thousands
FROM kept k JOIN periods p USING (filer_cik, period)
WHERE p.age <= 2
GROUP BY k.filer_cik, p.age, k.period, k.cusip, coalesce(k.put_call, ''), k.share_type
),
cur AS (SELECT * FROM positions WHERE age = 1),
prev AS (SELECT * FROM positions WHERE age = 2),
joined AS (
SELECT coalesce(c.filer_cik, p.filer_cik) AS filer_cik, coalesce(c.filer_name, p.filer_name) AS filer_name,
max(c.period) OVER (PARTITION BY coalesce(c.filer_cik, p.filer_cik)) AS period,
max(p.period) OVER (PARTITION BY coalesce(c.filer_cik, p.filer_cik)) AS previous_period,
coalesce(c.cusip, p.cusip) AS cusip, coalesce(c.issuer, p.issuer) AS issuer,
coalesce(c.title_of_class, p.title_of_class) AS title_of_class,
coalesce(c.put_call, p.put_call) AS put_call, coalesce(c.share_type, p.share_type) AS share_type,
coalesce(p.shares, 0) AS previous_shares, coalesce(c.shares, 0) AS shares,
coalesce(p.value_usd, 0) AS previous_value_usd, coalesce(c.value_usd, 0) AS value_usd,
coalesce(c.in_thousands, false) OR coalesce(p.in_thousands, false) AS reported_in_thousands
FROM cur c FULL OUTER JOIN prev p
ON p.filer_cik = c.filer_cik AND p.cusip = c.cusip AND p.put_call = c.put_call AND p.share_type = c.share_type
)
SELECT j.*, n.notice_period,
CASE WHEN previous_period IS NULL THEN 'no previous quarter'
WHEN previous_shares = 0 THEN 'new'
WHEN shares = 0 THEN 'exited'
WHEN shares > previous_shares THEN 'increased'
WHEN shares < previous_shares THEN 'decreased'
ELSE 'unchanged' END AS status,
round(100 * (shares - previous_shares) / nullif(previous_shares, 0), 1) AS shares_change_pct,
value_usd - previous_value_usd AS value_change_usd,
-- A stock split multiplies the share count and divides the price per share by the same factor.
previous_shares > 0 AND shares > 0 AND previous_value_usd > 0 AND value_usd > 0
AND (shares / previous_shares >= 1.9 OR shares / previous_shares <= 0.53)
AND abs((value_usd / shares) / (previous_value_usd / previous_shares) * (shares / previous_shares) - 1) < 0.25 AS possible_split
FROM joined j LEFT JOIN notices n ON n.filer_cik = j.filer_cik AND n.notice_period > j.period
ORDER BY filer_cik, status, value_change_usd DESC
- id: changes_csv
type: io.kestra.plugin.serdes.csv.IonToCsv
from: "{{ outputs.changes.uri }}"
- id: digest
type: io.kestra.plugin.jdbc.duckdb.Query
description: One message per manager with the portfolio change, new and exited
positions, and the largest moves.
inputFiles:
changes.csv: "{{ outputs.changes_csv.uri }}"
fetchType: FETCH
sql: |
WITH c AS (SELECT * FROM read_csv_auto('{{ workingDir }}/changes.csv')),
label AS (
-- Name the share class when a manager holds several classes of one issuer (Alphabet class A and C).
SELECT *, issuer || CASE WHEN count(DISTINCT cusip) OVER (PARTITION BY filer_cik, issuer) > 1 THEN ' ' || title_of_class ELSE '' END
|| CASE WHEN put_call <> '' THEN ' (' || put_call || ' options)'
WHEN share_type = 'PRN' THEN ' (principal amount)' ELSE '' END AS name
FROM c
),
ranked AS (
SELECT *, abs(shares_change_pct) >= {{ inputs.min_change_pct }} AS large_move,
row_number() OVER (PARTITION BY filer_cik, status, abs(shares_change_pct) >= {{ inputs.min_change_pct }}
ORDER BY abs(value_change_usd) DESC) AS rn,
count(*) OVER (PARTITION BY filer_cik, status) AS status_count
FROM label
),
lines AS (
SELECT filer_cik, any_value(filer_name) AS filer_name, any_value(period) AS period, any_value(previous_period) AS previous_period,
sum(value_usd) AS value_usd, sum(previous_value_usd) AS previous_value_usd,
count(*) FILTER (WHERE shares > 0) AS positions,
string_agg(name || format(' (${:,.1f}M)', value_usd / 1e6), ', ' ORDER BY value_usd DESC) FILTER (WHERE status = 'new' AND rn <= {{ inputs.top_n }})
|| CASE WHEN max(status_count) FILTER (WHERE status = 'new') > {{ inputs.top_n }}
THEN format(' and {} more', max(status_count) FILTER (WHERE status = 'new') - {{ inputs.top_n }}) ELSE '' END AS new_positions,
string_agg(name || format(' (was ${:,.1f}M)', previous_value_usd / 1e6), ', ' ORDER BY previous_value_usd DESC) FILTER (WHERE status = 'exited' AND rn <= {{ inputs.top_n }})
|| CASE WHEN max(status_count) FILTER (WHERE status = 'exited') > {{ inputs.top_n }}
THEN format(' and {} more', max(status_count) FILTER (WHERE status = 'exited') - {{ inputs.top_n }}) ELSE '' END AS exits,
string_agg(name || format(' {:+.1f}% shares', shares_change_pct) || CASE WHEN possible_split THEN ' (possible split)' ELSE '' END, ', ' ORDER BY value_change_usd DESC)
FILTER (WHERE status = 'increased' AND large_move AND rn <= {{ inputs.top_n }}) AS increases,
string_agg(name || format(' {:+.1f}% shares', shares_change_pct) || CASE WHEN possible_split THEN ' (possible split)' ELSE '' END, ', ' ORDER BY value_change_usd)
FILTER (WHERE status = 'decreased' AND large_move AND rn <= {{ inputs.top_n }}) AS decreases,
bool_or(status = 'no previous quarter') AS first_report, max(notice_period) AS notice_period,
bool_or(reported_in_thousands) AS in_thousands
FROM ranked GROUP BY filer_cik
)
SELECT filer_cik, filer_name, period,
CASE WHEN notice_period IS NOT NULL THEN format('For {}, {} filed a 13F notice: its holdings are in another manager''s report. ', notice_period, filer_name) ELSE '' END ||
CASE WHEN first_report THEN format('{} (CIK {}): {} positions, ${:,.1f}B at {}; no earlier 13F in the last 13 months to compare with.',
filer_name, filer_cik, positions, value_usd / 1e9, period)
ELSE format('{} (CIK {}), {} vs {}: {} positions, ${:,.1f}B ({:+.1f}%).{}{}{}{}', filer_name, filer_cik, period, previous_period,
positions, value_usd / 1e9, 100 * (value_usd / nullif(previous_value_usd, 0) - 1),
CASE WHEN new_positions IS NOT NULL THEN ' New: ' || new_positions || '.' ELSE '' END,
CASE WHEN exits IS NOT NULL THEN ' Exited: ' || exits || '.' ELSE '' END,
CASE WHEN increases IS NOT NULL THEN ' Added to: ' || increases || '.' ELSE '' END,
CASE WHEN decreases IS NOT NULL THEN ' Cut: ' || decreases || '.' ELSE '' END) END ||
CASE WHEN in_thousands THEN ' Values were reported in thousands of dollars and are converted.' ELSE '' END AS message
FROM lines
ORDER BY filer_name
- id: log_digest
type: io.kestra.plugin.core.log.Log
message: |
13F holdings changes:
{% for d in outputs.digest.rows %}- {{ d.message }}
{% endfor %}
- id: slack
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
description: Optional. Without the SLACK_WEBHOOK_URL secret this step fails as a
warning; the digest above is already logged.
runIf: "{{ outputs.digest.size > 0 }}"
allowFailure: true
url: "{{ secret('SLACK_WEBHOOK_URL') }}"
# A full JSON payload: `messageText` starting with a number is cut to that number (kestra-io/plugin-slack#103).
payload: >-
{{ outputs.digest.rows | jq('{text: ("13F holdings changes:\n" + (map("• "
+ .message) | join("\n")))}') | first | toJson }}
outputs:
- id: changes
type: FILE
description: Every position of each manager in the two quarters, with its status
and changes (CSV).
value: "{{ outputs.changes_csv.uri }}"
triggers:
- id: new_13f
type: io.kestra.plugin.edgar.filings.Trigger
description: Runs when a followed manager files a 13F holdings report or an
amendment. Enable after setting SEC_EDGAR_USER_AGENT.
userAgent: "{{ kv('SEC_EDGAR_USER_AGENT', errorOnMissing=false) ?? '' }}"
ciks:
- "1067983"
- "1061768"
- "1656456"
forms:
- 13F-HR
- 13F-HR/A
interval: PT1H
disabled: true
Follow what institutional managers buy and sell, from the 13F holdings reports they file with the SEC each quarter. No API key or paid data feed.
new_13f (EDGAR trigger) runs the flow when a followed manager files a 13F or an amendment; a
triggered run reports only the managers that filed.each_filer lists 13 months of 13F filings per CIK, and each_filing downloads each one's cover
page and information table.parse (Python, standard library) writes one row per information-table entry.changes (DuckDB) compares each manager's latest quarter with the one before:digest writes one message per manager, logged and posted to Slack.changes CSV of every position in both quarters with its status and change.Equity analysts, investor relations teams and allocators who follow other managers' portfolios.
The EDGAR trigger replaces a polling script, each filing's files stay with the run, and the 13F rules are one SQL query versioned with the flow.
Kestra 2.0 with plugin-edgar 1.0.0 (auto-installed on save in a local OSS instance, otherwise
kestra plugins install io.kestra.plugin:plugin-edgar:1.0.0), and Docker for the Python task.
SLACK_WEBHOOK_URL (optional): without it the Slack step ends in a warning; the digest stays logged.company.team > KV Store: key SEC_EDGAR_USER_AGENT, type STRING, value
your name and email (Example Co admin@example.com). SEC rejects requests without one.changes.ciks and enable it.changes on its CUSIP.Normalize Segment webhook batches, check event names and required property types against your taxonomy contract, and escalate violations to GitHub and Slack.
When a company you follow files a 10-Q or 10-K, read its XBRL numbers from SEC EDGAR, compute current ratio, debt-to-equity and margins in DuckDB, and alert on thresholds and sharp moves.
Extract CSV data, mask PII columns with DuckDB SQL, then load the anonymized result into BigQuery.
Turn real Sentinel-2 COG windows into a calibrated, cloud-masked NDVI raster with provenance and read-back checks.
Pick up PGP files from SFTP, decrypt them, verify the partner's signature, refuse replays and quarantine anything that fails.
Audits Apache SeaTunnel Zeta Engine REST API for failed streaming or batch jobs and alerts on pipeline stalls.
React to signed Sent webhook events in real time, alert Slack when a message's own status has not reached DELIVERED, and log a governance audit trail.
Normalize Segment webhook batches, check event names and required property types against your taxonomy contract, and escalate violations to GitHub and Slack.
Answer vendor security questionnaires from your own policies with verified citations, reuse approved answers, and send only uncertain answers to the security team.
Drive a headless browser via Selenium Grid to check whether a page sets tracking cookies before consent, and alert Slack on a GDPR-style violation.
Poll SEC EDGAR for new 8-K filings from the companies you follow, summarize each one and rate its sentiment with a local Ollama model, check every figure against the filing, and remember what was already processed.
Sync external API data into a custom ServiceNow table with automatic upsert (update or insert) logic.
