Skip to content
Webfold Docs
Esc
↑↓navigate↵open⌘Jpreview
On this page

PII and secrets

Detect and redact personal data and credentials with a command-line scan, a library, or a self-hosted HTTP service.

PII detection comes from pii-svc: a set of Rust crates and a single static binary. It runs on your own infrastructure with no managed cloud and no data egress.

Three ways to use it

Way Use it for
corpusforge scan and corpusforge redact Scan a folder in CI, or redact a file
pii-svc scan The same detectors as a CI gate in the PII project
HTTP service Redact text from any language 
corpusforge scan ./data                       # exit 2 if anything is found
corpusforge redact ./notes.txt -o notes.redacted.txt

The HTTP service

cargo run -p pii-svc          # listens on 0.0.0.0:${PII_SVC_PORT:-7780}
curl http://127.0.0.1:7780/healthz
curl -s http://127.0.0.1:7780/redact \
  -H 'content-type: application/json' \
  -d '{"text":"email jane.doe@example.com, TFN 123456782, call 0412 345 678"}'
{
  "redacted_text": "email [EMAIL_ADDRESS], TFN [AU_TFN], call [PHONE_NUMBER]",
  "findings": [
    { "entity_type": "EMAIL_ADDRESS", "start": 6, "end": 26, "score": 1.0 },
    { "entity_type": "AU_TFN", "start": 32, "end": 41, "score": 1.0 },
    { "entity_type": "PHONE_NUMBER", "start": 48, "end": 60, "score": 0.9 }
  ]
}

Offsets are byte offsets into the original text, with end exclusive. If PII_SVC_TOKEN is set, /redact requires Authorization: Bearer <token>. The full contract is in the PII API reference.

What it detects

Pass Finds
Patterns and checksums Emails, IPv4 and IPv6, phone numbers (AU, US and international), credit cards (Luhn), Australian TFN, ABN and Medicare; national IDs and IBANs such as US SSN, UK NHS and NINO, Spanish DNI and NIE, French NIR, Polish PESEL, Dutch BSN, German tax ID, Italian CF, Korean RRN, Singapore NRIC, Indian PAN and Aadhaar, Japanese My Number
Secrets PEM private keys, provider API keys, JWTs and credential assignments. Shape-only, with no live verification
Identity denylist Names, handles and hosts you supply through PII_IDENTITY_FILE or PII_IDENTITY_TERMS
Contextual model (optional) Names, addresses and other context-dependent entities, through a GLiNER model. Fails soft to the passes above
Resonance detector (opt-in) A single fast detector that flags spans an ordinary-prose model has never produced. Enabled with PII_RESONANCE

Overlaps resolve to the longest span.

Caveats

See the PII detection benchmark for the project’s own speed and accuracy figures and their limits.

Hosted API

If you would rather not run the service, call POST /v1/pii/redact (redacted text and findings) or POST /v1/pii/scan (findings and a clean flag, for gating) on the Webfold API. Findings carry byte offsets into your text. Up to 1 MiB of text per request. The text is sent to the API, so self-host the service if it must stay on your network.

curl -s https://api.webfold.ai/v1/pii/scan \
  -H "Authorization: Bearer $WEBFOLD_API_KEY" -H "Content-Type: application/json" \
  -d '{"text":"Call 0400 000 000 or write to someone@example.com"}'