The previous articles in this series covered why LLM redaction can fail and how adversaries can exploit those failures. This article addresses the engineering question: how do you build a redaction pipeline that leverages LLM capabilities while maintaining the reliability that compliance and legal contexts demand?
Design Principles
TCI's redaction architecture follows three principles:
- ▶Deterministic where possible, probabilistic where necessary: Pattern-based detection handles structured PII; LLMs handle contextual PII
- ▶Fail closed: If the system is uncertain whether something is PII, it's flagged for review rather than passed through
- ▶Verify independently: The verification layer has no access to the original document or the redaction model's internal state
Pipeline Architecture
Stage 1: Document Normalization
Before any PII detection, documents are normalized:
- ▶Text extraction through format-specific parsers (not LLMs)
- ▶Unicode normalization to canonical form
- ▶Metadata extraction and separate processing
- ▶Structural analysis (headers, footers, tables, embedded objects)
This stage produces clean, structured text and a separate metadata payload.
Stage 2: Deterministic Detection
Pattern-based detectors run first:
- ▶Regex patterns: SSN, credit card, phone number, email, IP address — formats with rigid structure
- ▶NER models: Microsoft Presidio with custom entity recognizers for domain-specific patterns
- ▶Dictionary matching: Known entity lists (employee names, client names, project codes)
Every detection is tagged with a confidence score and detection method.
Stage 3: LLM Contextual Analysis
The LLM processes pre-redacted text (with Stage 2 detections already marked) and identifies:
- ▶PII that's identifiable from context but has no structural pattern
- ▶Quasi-identifiers that could re-identify individuals in combination
- ▶Sensitive information that doesn't match standard PII categories
The LLM's detections are tagged separately from deterministic detections.
Stage 4: Ensemble Decision
A rules engine evaluates all detections:
- ▶High confidence from any single method (>0.95): Auto-redact
- ▶Agreement between deterministic and LLM methods: Auto-redact
- ▶LLM-only detection with moderate confidence: Flag for human review
- ▶Conflicting signals: Flag for human review with context
Stage 5: Redaction Execution
Redaction is applied deterministically — no LLM in this step. The redaction engine:
- ▶Replaces identified PII with consistent pseudonyms or category markers
- ▶Ensures redaction is complete (no partial masking that leaves identifiable fragments)
- ▶Preserves document structure and readability
- ▶Generates a redaction map linking each redaction to its detection source
Stage 6: Independent Verification
A separate verification pipeline processes the redacted output:
- ▶Full PII scan of the redacted document (same detectors, fresh context)
- ▶Semantic analysis for information that could identify individuals through inference
- ▶Format-level check for residual metadata
- ▶Comparison against expected redaction count (significant discrepancies trigger review)
The verification model is a different LLM from the detection model, reducing the risk that a shared blind spot persists through both stages.
Handling Edge Cases
Multi-document Correlation
PII that's safe in a single document may be identifying when combined with other documents in the same release. The pipeline maintains a cross-document entity registry and applies correlation-aware redaction rules.
Inconsistent Redaction
The same entity must be redacted consistently across all occurrences. The pipeline tracks entity identities across the document and applies uniform treatment.
Redaction of Redaction References
If a document discusses redaction methodology or references specific PII categories, the pipeline distinguishes between PII and meta-discussion about PII.
Compliance Documentation
Every redaction run produces:
- ▶Input document hash (for integrity verification)
- ▶Output document hash
- ▶Complete detection log (what was found, where, by which method, at what confidence)
- ▶Redaction map (what was redacted, what it was replaced with, why)
- ▶Verification results
- ▶Processing timestamps and system versions
This documentation satisfies audit requirements under GDPR, CCPA, and HIPAA Safe Harbor.
The Economics
A well-architected pipeline with LLM components costs more per document than regex-only redaction. But it catches significantly more PII, reduces human review time, and produces auditable documentation automatically. For organizations processing hundreds or thousands of documents, the ROI is clear.
TCI deploys this pipeline architecture for clients handling legal discovery, regulatory compliance, and source protection workloads.