Rule-based detection · measured limits
What OptinRouter detects today
Measured on synthetic examples. Coverage varies by kind, and missed details remain possible.
Read what is not detected ↓Detector quality
Detector local-normalized-1.1.1 · policy local-policy-1.0.1 · frozen labelling policy demo-gold-v1.
530 synthetic records, including 150 hard negatives.
Precision measures how many detections match a labelled span. Recall measures how many labelled spans are found. F1 balances precision and recall. TP = true positives; FP = false positives; FN = false negatives.
Exact requires matching kind and boundaries. Overlap gives full credit for a same-kind byte intersection, with one-to-one matching. Overall pools span counts across kinds. Percentages are rounded to a tenth of a percentage point.
| Kind | PRECISION | RECALL | F1 | TP | FP | FN |
|---|---|---|---|---|---|---|
| Overall (micro pooled) | 77.2% | 86.2% | 81.4% | 362 | 107 | 58 |
| Canadian social insurance numbers | 80.0% | 100.0% | 88.9% | 40 | 10 | 0 |
| Credentials | 35.1% | 82.5% | 49.3% | 33 | 61 | 7 |
| Email addresses | 69.4% | 73.8% | 71.5% | 59 | 26 | 21 |
| IBAN account numbers | 100.0% | 100.0% | 100.0% | 40 | 0 | 0 |
| Payment cards | 100.0% | 100.0% | 100.0% | 40 | 0 | 0 |
| Phone numbers | 83.3% | 83.3% | 83.3% | 50 | 10 | 10 |
| Private keys | 100.0% | 100.0% | 100.0% | 40 | 0 | 0 |
| Restriction markers | 100.0% | 75.0% | 85.7% | 30 | 0 | 10 |
| US social security numbers | 100.0% | 75.0% | 85.7% | 30 | 0 | 10 |
| Kind | PRECISION | RECALL | F1 | TP | FP | FN |
|---|---|---|---|---|---|---|
| Overall (micro pooled) | 82.1% | 91.7% | 86.6% | 385 | 84 | 35 |
| Canadian social insurance numbers | 80.0% | 100.0% | 88.9% | 40 | 10 | 0 |
| Credentials | 37.2% | 87.5% | 52.2% | 35 | 59 | 5 |
| Email addresses | 94.1% | 100.0% | 97.0% | 80 | 5 | 0 |
| IBAN account numbers | 100.0% | 100.0% | 100.0% | 40 | 0 | 0 |
| Payment cards | 100.0% | 100.0% | 100.0% | 40 | 0 | 0 |
| Phone numbers | 83.3% | 83.3% | 83.3% | 50 | 10 | 10 |
| Private keys | 100.0% | 100.0% | 100.0% | 40 | 0 | 0 |
| Restriction markers | 100.0% | 75.0% | 85.7% | 30 | 0 | 10 |
| US social security numbers | 100.0% | 75.0% | 85.7% | 30 | 0 | 10 |
Fully-clean rate in this evaluation
90.8%Positive records: 345 / 380
93.4%All records: 495 / 530
Negative records have no labelled sensitive bytes, so they are covered automatically. Here, fully clean means all labelled bytes would be covered by hypothetical replace-all de-identification. This does not establish anonymity, retained usefulness or permission to export.
Not detected in V1
Names and street addresses are not detected. Model-based detection is post-V1.
Non-ASCII text, images, and obfuscated or encoded content are not covered by these V1 rules.
Automated cleaning is not a guarantee. The contributor reviews and approves what is shared.
How these numbers were measured
Synthetic coding fragments were labelled under a frozen policy before evaluation. The examples were authored with knowledge of the detectors; this is not a population estimate.
Limitations, as recorded with the measurements
- Synthetic set authored with knowledge of the detector families; not a population estimate or independent held-out evaluation; repeated templates are correlated.
- Names and addresses are not covered: ML detector not admitted, contextual useful gate 0/16.
- ASCII only; no multilingual, Unicode, image, obfuscation or general encoded-content coverage claim.
- Fully clean means every gold byte covered by detected spans under hypothetical replace-all de-identification; not production Clean acceptance, utility, anonymity or export permission.
- Overlap uses same-kind binary full credit with one-to-one matching, not SemEval half-credit partial scoring; exact is boundary-and-kind matching.
Measurement provenance
Source: scanning/testdata/demo-eval/METRICS.json
Source SHA-256: 5a3eb6f8d30775f1d1c08a1315c380416873dbce97bff8537c5dc8da85c1099a
Effective policy: local-policy-1.0.1:75a6b50f2f4d2d2c16bc885d3f73c05f7f1bb1fe6e411ce0bba818e2c93119c6