# DupeHell Bug Tracker
Generated by Wave 14 — Massive Retest (2026-07-21). All previous bugs (C1–C11) confirmed fixed.
---
## Bug Status Overview
| **C12** — Phone alpha character injection in fuzzy_dup records | 🟠 HIGH | Open | Multi-domain (17) | 14.4 |
| **C13** — Datetime alpha character injection in fuzzy_dup records | 🟠 HIGH | Open | Multi-domain (6) | 14.4 |
| **C14** — Hash collision: `--singleton-master-fraction` flag | 🟡 MEDIUM | Open | CLI | 14.1 |
| **C15** — Hash collision: `--locale` flag | 🟡 MEDIUM | Open | CLI | 14.1 |
| **C16** — No overwrite warning on hash collision | 🟠 HIGH | Open | CLI | 14.1 |
| **C17** — Missing `__version__` attribute in Python package | 🟢 LOW | Open | Python | 14.2 |
| **C18** — `singleton_master_fraction` parameter undocumented | 🟢 LOW | Open | Python | 14.2 |
| **C19** — Null rate inflation: 71 columns exceed schema + 10pp | 🔴 CRITICAL | Open | Multi-domain | 14.4 |
| **C20** — IP address alpha injection in fuzzy_dup records | 🟡 MEDIUM | Open | cybersecurity | 14.5 |
| **C21** — Datetime format inconsistency (DD-MM-YYYY in YYYY-MM-DD cols) | 🟡 MEDIUM | Open | Multi-domain (7) | 14.4 |
| **C22** — Negative `current_balance` in fintech/banking entities | 🟢 LOW | Open | fintech, banking | 14.4 |
| **C23** — PAN watermark digit count: docs say 2, code uses 3 | 🟢 LOW | Open | docs/WATERMARK.md | 14.9 |
| **C24** — Passport watermark digit count: docs say 2, code uses 3 | 🟢 LOW | Open | docs/WATERMARK.md | 14.9 |
| **C25** — Unverified locality HMAC in fk_remap | 🟡 MEDIUM | Investigating | fk_remap.rs | 14.7 |
---
## Bug Details
### C12 — Phone alpha character injection in fuzzy_dup records
| **Severity** | 🟠 HIGH |
| **File** | `src/noise/typos.rs` or `src/noise/visual.rs` |
| **Domain(s)** | kyc, healthcare, fintech, publishing, ecommerce, education, realestate, energy, telecom, logistics, nonprofit, insurance, banking, hospitality, travel, retail |
| **Affected columns** | 19 phone columns across 17 domains |
**Description**: The noise pipeline applies visual homoglyph/typo substitutions to all string fields in fuzzy_dup records, including phone numbers. Letters substitute digits in phone strings, producing values no real-world dataset would contain.
**Evidence** (samples from kyc, seed=42, size=1000):
- `+1f-67z-282-3633`
- `+1-337k500-4620`
- `+1-915-h82-8521`
**Rate**: 5-90% of non-null phone values affected across domains. Worst in `education/office_phone` (90%, though some are "x" extensions — see below).
**Expected behavior**: Phone numbers should contain only digits, spaces, `+`, `-`, `(`, `)`. Typo noise should use digit-to-digit substitutions (e.g., `5→6`, `0→9`) rather than letter-to-digit.
**Root cause**: Noise pipeline applies the same OCR/typo noise functions to all string columns without considering column semantics. Phone numbers, IPs, and dates should use digit-safe noise.
**Note on education/office_phone**: 90% of non-null values contain "x" extension format (`+1-XXX-XXX-XXXX xNNNN`). This is likely by-design (office phones with extensions) but the strict validation regex rejects "x" as non-digit.
---
### C13 — Datetime alpha character injection in fuzzy_dup records
| **Severity** | 🟠 HIGH |
| **File** | `src/noise/typos.rs` or `src/noise/visual.rs` |
| **Domain(s)** | blockchain, gaming, energy, manufacturing, logistics, nonprofit |
| **Affected columns** | last_active, last_login, reading_date, start_date, delivery_date, donation_date |
**Description**: Letters substitute digits in datetime strings (year/month/day/hour positions), producing values like `2025-x06-15 14z:00:00`.
**Evidence** (samples):
- `2025-x06-15 14z:00:00` (blockchain/last_active)
- `2021-0n-10 2c:00:00` (gaming/last_login)
- `2022-08-2z4 c0:00:00` (energy/reading_date)
**Rate**: Up to 40/203 (20%) of non-null datetime values affected.
**Root cause**: Same as C12 — noise pipeline applies character-level noise to all string fields without semantic awareness.
---
### C14 — Hash collision: `--singleton-master-fraction` flag
| **Severity** | 🟡 MEDIUM |
| **File** | `src/main.rs` (hash computation) |
| **Domain(s)** | CLI |
**Description**: Running with `--singleton-master-fraction 0.5` produces a different dataset (60 masters vs 73 default) but the **same output hash** as the default configuration. This causes file overwrite.
**Evidence**:
- Default: `--domain kyc --size 200 --seed 42` → `kyc_51d06d6bdff895e8.parquet` (73 masters)
- With flag: `--domain kyc --size 200 --seed 42 --singleton-master-fraction 0.5` → `kyc_51d06d6bdff895e8.parquet` (60 masters)
- **Same filename, different data** → Run2 silently overwrites Run1.
**Expected behavior**: Output hash should include ALL CLI flags that affect data generation. The hash is stable across runs with identical params but must differ when params differ.
**Root cause**: `singleton_master_fraction` is not included in the hash input computation.
---
### C15 — Hash collision: `--locale` flag
| **Severity** | 🟡 MEDIUM |
| **File** | `src/main.rs` (hash computation) |
| **Domain(s)** | CLI |
**Description**: Running with `--locale fr`, `--locale de`, or `--locale es` produces different names (e.g., FR: `Aaron, Anis` vs EN: `Abigail, Alexander`) but the **same output hash** as `--locale en`.
**Evidence**:
- `--locale en` → `kyc_<same_hash>.parquet`
- `--locale fr` → `kyc_<same_hash>.parquet` but with French names
- **Same filename, different (localized) data** → overwrite
**Expected behavior**: Locale should be part of the hash input, just like domain and seed.
**Root cause**: `locale` is not included in the hash input computation.
---
### C16 — No overwrite warning on hash collision
| **Severity** | 🟠 HIGH |
| **File** | `src/main.rs` |
| **Domain(s)** | CLI |
**Description**: When `--output-dir` already contains a file from a prior run (or when hash collision occurs as in C14/C15), the CLI silently overwrites without warning. Combined with C14/C15, this causes silent data loss.
**Evidence**: Running the same output-dir twice with different locale/fraction produces no warning.
**Expected behavior**: At minimum, warn when overwriting an existing file. Ideally, refuse to overwrite unless `--force` is specified.
---
### C17 — Missing `__version__` attribute in Python package
| **Severity** | 🟢 LOW |
| **File** | `python/dupehell/__init__.py` |
| **Domain(s)** | Python API |
**Description**: `dupehell.__version__` is not defined. PEP 396 / `importlib.metadata` convention is absent — no programmatic way to check package version at runtime.
**Evidence**:
```python
>>> import dupehell
>>> dupehell.__version__
AttributeError: module 'dupehell' has no attribute '__version__'
```
**Fix**: Add `__version__ = version("dupehell")` using `importlib.metadata`.
---
### C18 — `singleton_master_fraction` parameter undocumented
| Field | Value |
|-------|-------|
| **Severity** | 🟢 LOW |
| **File** | `python/dupehell/__init__.py` |
| **Domain(s)** | Python API |
**Description**: The `singleton_master_fraction` parameter exists in the `generate()` signature but is missing from its docstring. All other 12 parameters are documented.
---
### C19 — Null rate inflation: 71 columns exceed schema + 10pp
| Field | Value |
|-------|-------|
| **Severity** | 🔴 CRITICAL |
| **File** | `src/pipeline.rs` / `src/column_gen.rs` |
| **Domain(s)** | Multi-domain (multi-entity schemas) |
**Description**: 71 columns across 17 domains where the schema specifies a non-zero `null_rate_default` have actual null rates exceeding schema + 10pp. This is a regression of the same pattern as C5/C10.
**Evidence** (worst offenders, size=1000, seed=42):
| Domain | Column | Schema null_rate | Actual null_rate | Delta |
|--------|--------|:---------------:|:----------------:|:-----:|
| realestate | parcel_number | 0.030 | 0.802 | +77pp |
| publishing | isbn | 0.050 | 0.801 | +75pp |
| insurance | ssn | 0.080 | 0.776 | +70pp |
| education | gpa | 0.100 | 0.782 | +68pp |
| energy | phone | 0.100 | 0.826 | +73pp |
**Expected behavior**: For a column with `null_rate_default = 0.03`, roughly 3% of values should be null (within binomial tolerance). Actual rates of 77-80% null indicate the schema null_rate is being applied additively on top of cross-entity structural nulls instead of being the final null probability.
**Root cause hypothesis**: In multi-entity domains, a column exists only for one entity type. When the pipeline computes "apply null_rate 0.03", it applies 0.03 null probability to ALL rows, but since 80% of rows belong to other entities, the effective null rate becomes `1 - (1-0.80)*(1-0.03) ≈ 0.806`. The schema null_rate should be applied only to rows of the owning entity, or the effective null rate should be calibrated.
---
### C20 — IP address alpha injection in fuzzy_dup records
| Field | Value |
|-------|-------|
| **Severity** | 🟡 MEDIUM |
| **File** | `src/noise/typos.rs` or `src/noise/visual.rs` |
| **Domain(s)** | cybersecurity |
| **Affected column** | ip_address |
**Description**: Same root cause as C12/C13. IP addresses in fuzzy_dup records get alphabetic characters injected into octets.
**Evidence**: 13/257 (5.1%) of non-null IP values affected:
- `191.2k.199.64`
- `37.78.13f2.231`
- `235.107.15e6.112`
---
### C21 — Datetime format inconsistency
| Field | Value |
|-------|-------|
| **Severity** | 🟡 MEDIUM |
| **File** | `src/noise/dates.rs` (date format swap) |
| **Domain(s)** | healthcare, fintech, blockchain, telecom, manufacturing, energy, logistics, gaming, nonprofit, banking |
| **Affected columns** | datetime columns |
**Description**: ~4-8% of datetime values use `DD-MM-YYYY HH:MM:SS` format instead of ISO 8601 `YYYY-MM-DD HH:MM:SS`. All digits are valid but day/month ordering is swapped.
**Evidence**:
- `02-02-2024 12:00:00` (ambiguous — could be Feb 2)
- `27-02-2025 06:00:00` (clearly DD-MM — 27 is not a month)
- `31-12-2020 04:00:00` (clearly DD-MM — 31 is not a month)
**Expected behavior**: While date format swapping is valid noise, DD-MM-YYYY format in a dataset that is otherwise YYYY-MM-DD can cause parsing errors. Consider using a consistent format per column or documenting the date-swap behavior clearly.
---
### C22 — Negative `current_balance` in fintech/banking
| Field | Value |
|-------|-------|
| **Severity** | 🟢 LOW |
| **File** | `src/column_gen.rs` (float generator) |
| **Domain(s)** | fintech, banking |
**Description**: `current_balance` column has negative values like `-3486.10` and `-5089.99`. This may be intentional (overdrawn accounts) but there's no schema annotation indicating signed vs non-negative semantics.
**Status**: Ambiguous — may be by-design. Flagged for maintainer review.
---
### C23 — PAN watermark digit count: docs say 2, code uses 3
| Field | Value |
|-------|-------|
| **Severity** | 🟢 LOW |
| **File** | `docs/WATERMARK.md` vs `src/buf_gen.rs:184-203` |
**Description**: Documentation states watermark encodes in "last 2 digits" of PAN, but `buf_pan` calls `watermark_buf(buf, 3, wm)` — actually using 3 digits.
---
### C24 — Passport watermark digit count: docs say 2, code uses 3
| Field | Value |
|-------|-------|
| **Severity** | 🟢 LOW |
| **File** | `docs/WATERMARK.md` vs `src/buf_gen.rs:263-278` |
**Description**: Same as C23 for passport — doc says 2, code uses 3.
---
## Previously Fixed Bugs (C1–C11)
All confirmed resolved in v0.5.0:
| Bug | Verification |
|-----|-------------|
| C8 — hard_neg_ratio validation | Agent 14.2: hard_neg_ratio=-0.1 correctly raises ValueError |
| C9 — blockchain owner_email | Agent 14.1: 16/16 valid emails with @ |
| C10 — Null rates (C5 regression) | ⚠️ Re-emerged as C19 (different manifestation) |
| C11 — difficulty column in GT | Agent 14.6: GT has correct columns only |
---
## Fixed During Wave 14
| Bug | Resolution |
|-----|-----------|
| C19 (wave) — CLI/Python cross-API divergence | FIXED by rebuilding Python package from latest source (was stale wheel from July 18) |
---
## Stats
- **Total bugs found (Wave 14)**: 14 (1 CRITICAL, 3 HIGH, 5 MEDIUM, 5 LOW)
- **Tests run**: ~290 tests across 11 agents
- **Domains tested**: All 40
- **Clean passes**: GT integrity (10/10), FK integrity (16/16 rules), IPC/Parquet (28/28), Watermark (10/10), Edge cases (30/30)