If icon
Loop icon
Subflow icon
Commands icon
Docker icon
Download icon
WorkingDirectory icon
List icon
OutputValues icon
Upload icon
Fail icon
Log icon
Request icon
Trigger icon
Webhook icon

Publish verified searchable PDF archives with OCRmyPDF

Run OCRmyPDF, verify PDF/A with veraPDF and page text with PDFium, then safely publish or quarantine PDFs in S3-compatible storage.

Categories
DataInfrastructure

Publish independently verified searchable PDFs without modifying sources.

How it works

  1. Manual runs/webhooks select source_key. S3 polling queues one child per new/changed PDF each minute without waiting, releasing the single-execution slot.
  2. Inspect the source, then run OCRmyPDF with --skip-text and PDF/A-2 output. Invalid sources bypass OCR.
  3. veraPDF checks PDF/A-2b; PDFium checks page count and text on every page. Missing evidence cannot pass.
  4. ZIP the PDF, text, report and SHA-256 manifest. Rejections contain the original and diagnostics. One S3 PUT publishes the bundle; exact-key checks prevent repeat overwrites.

What you get

Download outputs.verify_and_bundle.outputFiles['bundle.zip'] and ['report.json']. outputs.publication_plan.values exposes destination, action and artifact URIs. Content/settings-derived keys use accepted/ or quarantine/. Rejections fail and alert, even in dry runs; queued child failures alert independently.

Who it's for

Document platform engineers can use this flow to turn scanned PDFs arriving in object storage into searchable archive bundles. Records teams can apply it to digitized contracts, historical correspondence or scanned technical manuals that need extractable text and independently checked PDF/A output. Teams migrating an existing PDF collection can preview the results before enabling uploads to the archive bucket. Operators can inspect quarantined originals and diagnostic reports to identify documents that need rescanning, different language packs or manual correction.

Why orchestrate this with Kestra

Kestra connects object-storage ingestion, OCRmyPDF, veraPDF and per-page text checks in a visible execution history for each document. Its S3 trigger dispatches processing executions for new or changed PDFs, while the concurrency limit bounds how many documents the flow processes at once. Conditional publication keeps rejected documents out of the accepted archive, and content-derived keys with existence checks preserve previously published bundles on repeat runs. Storage retries, downloadable evidence and failure alerts help operators diagnose failures and manually retry affected documents without restarting an entire collection.

Prerequisites

Use Kestra 2.0.8, AWS/Shell plugins, Docker socket access, distinct buckets and an alert receiver. S3 polling requires stateful triggers. Request validation uses the image user. Shared-file tasks require root with Kestra 2.0 VOLUME staging; non-root writes fail. veraPDF needs amd64/emulation. Processing has no network access.

Secrets

  • OCR_S3_ACCESS_KEY_ID / OCR_S3_SECRET_ACCESS_KEY allow source read/list, archive list and bundle uploads.
  • OCR_WEBHOOK_KEY protects the webhook.
  • OCR_ALERT_WEBHOOK_URL receives JSON failure alerts.

Quick start

  1. Configure inbox_bucket (ocr-inbox), inbox_prefix (samples/), s3_endpoint (http://seaweedfs:8333) and s3_region (us-east-1) in variables. They supply the manual connection defaults. archive_bucket is ocr-archive; source_key is samples/scanned.pdf.
  2. Preview with dry_run BOOL true. language is STRING eng; use + for installed packs, such as eng+deu. max_pages is INT 25 (1–100), max_size_mb INT 20 (1–100, checked after download), min_text_chars_per_page INT 10 (1–1000). Other inputs are STRING.
  3. Polling children use input defaults; review before changing dry_run. Enable inbox_pdf for polling or pdf_arrived for webhook requests; both start disabled. Polling scans at most 1,000 objects, so partition larger inboxes by prefix. Dedup records detection, not success: retry failed children manually.

How to extend

Add indexing or installed language packs. Blank pages fail the text threshold; coverage does not establish OCR accuracy. Limit source sizes upstream.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.