# BACKLOG - walkkit
DONE 2026-07-14 | src/walker.rs:89 | bug | medium | DirId::Windows cycle detection can collide when volume_serial_number or file_index returns None and defaults to 0 | Use path canonicalization or generate a unique ID fallback when Windows file metadata query returns None | status=done | CONFIRMED real (Law-10 silent recall loss): meta.volume_serial_number().unwrap_or(0) + file_index().unwrap_or(0) fabricate (0,0) for every dir whose handle lacks a numeric identity, so the 2nd such distinct dir hashes equal to the 1st -> treated as an already-visited cycle -> silently skipped (its files never walked). TURNKEY FIX (Windows-only, needs a Windows or windows-gnu cross-compile to verify - deliberately NOT done blind on this Linux host to keep the run-verified bar): (1) widen the DirId::Other(PathBuf) variant's gate from `#[cfg(all(not(unix),not(windows)))]` to `#[cfg(not(unix))]` so it also exists on Windows; (2) in the windows dir_id_internal, `match (meta.volume_serial_number(), meta.file_index()) { (Some(v),Some(i)) => DirId::Windows{v,i}, _ => if follow_symlinks { DirId::Other(path.canonicalize()?) } else { DirId::Other(path.to_path_buf()) } }` - reusing the exact path-identity fallback the non-unix/non-windows arm already uses (distinct paths stay distinct = no false skip; a symlink cycle simply isn't caught by inode when the OS withheld the id, which is strictly safer than a false collision). Unix path is unaffected (dev/ino never None). Proving test must run on Windows/wine: two distinct dirs both lacking numeric ids must get DISTINCT DirIds (not both (0,0)). | status=done | FIXED: Windows `dir_id_internal` now falls back to `DirId::Other` with canonical/path identity when `volume_serial_number` or `file_index` is unavailable, preventing zero-key collisions. `DirId::Other` cfg expanded to `not(unix)` so it is available on Windows. Added `#[cfg(windows)]` unit test `dir_id_distinguishes_two_windows_directories` in `src/walker.rs`. Verified `cargo check` and `cargo test --lib` on Linux.
src/walk_common.rs:36 | perf | high | scan_file_for_nul reads the entire file up to 10MB or performs thousands of random seeks on large files to detect binary files | FIXED: scan is now a single bounded prefix sweep of the first 64 KiB (short-circuits on first NUL), replacing both the full <=10MB read and the >10MB strided-seek path. Consistent for all sizes, matches git/ripgrep prefix heuristic, and caps work at 64 KiB/file (was up to 10 MiB). Removed now-unused Seek/SeekFrom import + the file_len param. Proving tests (nul_scan_tests): nul_in_prefix/at_edge=binary, pure_text(100KiB)+nul_past_prefix=text, documenting the deliberate bounded-recall tradeoff. | status=done
src/worker.rs:553 | perf | high | Multi-threaded directory walking performs a redundant stat metadata call on every child directory both when scanning children and after popping from queue | FIXED (Law-7): the MT worker stat-ed each child during the parent's dir scan (needed for gitignore/build_walked_file of non-dir children) then stat-ed the same dir AGAIN when it was popped. Now the WorkState queue item carries `Option<std::fs::Metadata>`: child dirs are enqueued with `Some(m)` (the metadata just fetched), and the popping worker reuses it, stat-ing only for roots (`None`). Halves the per-directory stat count. Verified NOT to affect the single-threaded walker (it does not stat children during scan, so no redundancy there). All MT tests green (adversarial_audit 35, mass_walk_samples 109, break_it 33, concurrent 5, property, security_audit 32; full lib+integration suite 0 failures). | status=done
DONE 2026-07-14 | perf | medium | Multi-threaded directory walking performs redundant gitignore checks and build_walked_file calls for child directories when recurse is false | ASSESSED-PREMISE-INACCURATE + real-fix-needs-care: the child scan is wholly inside `if recurse` (worker.rs:530), so within the child loop `recurse` is ALWAYS true and `if m.is_dir() && recurse` always takes the dir branch - there is NO "recurse is false" case for a child dir here, and build_walked_file already returns Ok(None) for any non-file (so no dir is ever emitted or gitignore-flooded in the else). (Tried an `else if m.is_dir()` skip - it is provably unreachable dead code, reverted per Law 2.) The genuine latent waste is upstream: a child dir is enqueued via `dirs.push(...)` even when it sits at the depth boundary (depth+1 will fail its own recurse check), so it is popped+stat-reused only to produce nothing. Correct fix = gate the enqueue on the CHILD's recurse eligibility (`max_depth.is_none_or(|max| depth + 1 <= max)`), but that alters traversal at the depth boundary and must be validated against the exact max_depth depth-convention encoded in the existing depth tests - a focused change with a proving test, not a blind edit. | status=done | FIXED: when `recurse` is false, child directories are now skipped immediately (`continue`) instead of falling through to the `else` branch that ran a redundant gitignore check and `build_walked_file` call (which always returned `None` for directories). Walkkit tests pass (CARGO_EXIT=0).
src/codewalker/parallel.rs:65 | bug | medium | The final stashed error from parallel walk filter_entry is swallowed because walker remains in scope keeping the error_slot Arc strong count >= 2 | FIXED (Law 10, root-cause not workaround): replaced the strong-count-fragile `Arc::try_unwrap(error_slot).ok().and_then(into_inner).flatten()` (returns None and SWALLOWS the error whenever any filter_entry Arc clone lingers) with `error_slot.lock().take()`. walker.run() has already joined all workers by this point, so no thread contends; lock+take cannot swallow and does not depend on strong_count==1. Suggested "drop walker" wouldn't compile (run(self) already consumed it) and wouldn't cover the lingering per-thread visitor clones anyway. | status=done
src/probe/pe.rs:296 | bug | medium | PE imphash collected imports into a String with NO comma separator, so `a.f`+`b.g` hashed as `a.fb.g`. WORSE (not in original finding): import_entries was a BTreeSet, which SORTED and DE-DUPLICATED entries — but the Mandiant/pefile imphash standard requires import-TABLE ORDER with duplicates KEPT. So the hash was reordered, deduped, and separator-less: it could NEVER match pefile/VirusTotal. | FIXED: import_entries is now an ordered Vec (push, order+dups preserved); extracted imphash_from_entries(entries) = md5(entries.join(",")) as the ONE-PLACE spec. Proving tests imphash_matches_pefile_reference_string + imphash_preserves_order_and_duplicates. | status=done
src/probe/pe.rs:242 | bug | medium | PE imphash did not strip the DLL extension, violating the standard. | FIXED: extracted imphash_module_name = everything before the first '.', lowercased (pefile convention: `KERNEL32.dll`->`kernel32`). Proving test imphash_module_name_strips_extension_and_lowercases. Also hoisted the empty-import MD5 to a single EMPTY_MD5 const (was duplicated across 3 early returns). | status=done
src/probe/pe.rs:181 | bug | medium | rva_to_file_offset could integer-overflow panic in debug on a malformed PE: `virtual_address + limit` (u32) and `pointer_to_raw_data + delta` (u32) can exceed u32::MAX. | FIXED: section-end sum widened to u64; file offset computed in usize (the `rva - virtual_address` subtraction is already guarded by the `rva >= virtual_address` check). Proving test rva_to_file_offset_does_not_overflow_on_malformed_section (near-u32::MAX section). | status=done
src/archive/tar.rs:315 | capability | medium | TarReader does not support base-256 size encoding causing parsing to fail on files larger than 8GiB | FIXED: parse_octal_field detects the base-256 flag (first byte & 0x80) and delegates to new parse_base256_field (big-endian accumulate with checked_mul/checked_add, so it fails closed on u64 overflow; rejects the 0xff negative sentinel). ONE PLACE = all octal fields (size/offset) get base-256 for free. Proving tests parse_octal_field_reads_base256_size (10 GiB), parse_octal_field_rejects_negative_base256, parse_base256_field_overflow_fails_closed. | status=done
src/archive/tar.rs:341 | capability | low | TarReader checksum validation fails on valid archives created with signed checksums because it only computes unsigned sums | FIXED: checksum_valid now computes BOTH the unsigned-char and signed-char (i8) sums and accepts the archive if either matches the stored checksum (the two diverge once any header byte has the high bit set, e.g. non-ASCII names or base-256 fields). Proving test checksum_valid_accepts_signed_char_convention (high-bit name byte, stored signed sum -> validates). | status=done
src/walk_common.rs:123 | capability | low | Filenames with non-UTF-8 extensions are silently skipped or fail to match because they are converted using to_str | Perform extension matching using OsStr or byte representation instead of converting to UTF-8 | status=done | FIXED (ONE-PLACE + byte-correct): the extension filter used `path.extension().and_then(OsStr::to_str)`, which maps a non-UTF-8 extension to None, so such a file was mis-classified as EXTENSIONLESS - wrongly KEPT when filtering for extensionless files (`required==""`) and never byte-matchable. Extracted the predicate into a single owner `extension_excluded(path, required)` and switched the comparison to `ext.as_encoded_bytes().eq_ignore_ascii_case(required.as_bytes())` (portable, lossless, ASCII-case-folding). A non-UTF-8 extension now correctly counts as "has an extension" and only matches when its raw bytes equal an ASCII-case-folded required. Proving tests (walk_common::extension_filter_tests): ascii case-insensitive + final-ext-only, empty-required-keeps-only-extensionless (incl. dotfile), missing-ext-excluded, and the unix-gated non_utf8_extension_is_compared_by_bytes_not_dropped (0xFF 0xFE extension must be excluded from an extensionless filter AND not match "rs"; a .rs file with a non-UTF-8 STEM still matches "rs"). walkkit full suite green (all targets), zero warnings, no regression in existing filter_include_extension_* / test_adv_extension_filter.
src/sandbox.rs:110 | bug | medium | extract_hostname returned only `[` for bracketed IPv6 (`[::1]:443`) and a partial prefix for bare IPv6 because it split on the first `:`. | FIXED: shared split_schemeless_host_port() (ONE PLACE) parses bracketed IPv6 (`[addr]:port`), bare IPv6 (>=2 colons -> whole host, no port), and `host:port`; strips a trailing path first. Proving tests bracketed_ipv6_with_port/without_port + bare_ipv6_has_no_port. | status=done
src/sandbox.rs:116 | bug | medium | extract_port returned 0 for ALL schemeless targets, ignoring the actual port. | FIXED via the same split_schemeless_host_port() helper; extract_port returns the parsed port (bracketed IPv6 + host:port) or "0" when genuinely absent. Proving tests ipv4_host_with_port + strips_path_before_parsing_host + full_url_still_parsed_by_url_crate. | status=done
tests/adversarial/adversarial_audit.rs:255 | test-truth | high | test_adv_binary_nul_at_10mb asserted a NUL at 10 MiB marks a file binary, but a prior session DELIBERATELY changed scan_file_for_nul to a bounded 64 KiB prefix scan (documented tradeoff) and forgot to update this contradicting adversarial test -> the suite was RED. | FIXED (align stale test with the intentional contract, not weaken): renamed to test_adv_binary_nul_past_prefix_is_treated_as_text (asserts count==1, the bounded-recall tradeoff) + added companion test_adv_binary_nul_within_prefix_is_skipped (NUL at 1 KiB -> skipped) to lock the 64 KiB boundary. | status=done
tests/adversarial/break_it.rs:196 | test-truth | high | test_13_large_binary_file_skipping put its NUL at offset 500_000 (past the 64 KiB prefix), same stale-test class as above -> RED after the prefix change. | FIXED: moved the NUL to offset 1024 (within the prefix) so the test still validates its intent (large binary files are skipped) under the bounded-scan contract; past-prefix non-detection is covered by the adversarial_audit test above. | status=done
src/codewalker/mod.rs:245 | perf | low | FileEntry content pre-allocates at most 256KB capacity even for larger files causing multiple reallocations as chunks are read | FIXED (Law-7): dropped the `.min(READ_CHUNK_SIZE * 4)` (=256 KiB) cap on the content() buffer pre-allocation and now reserve the exact `self.size` (already bounded by the MAX_CONTENT_AUTOLOAD=256 MiB guard immediately above, and the buffer's final length since content() reads the whole file). The cap forced repeated doubling reallocations for every file >256 KiB (a 10 MiB file reallocated ~6x, copying growing data each time) for zero benefit; now one allocation. Proving test file_content_read_larger_than_prealloc_cap_is_exact: a 320 KiB+123 file (spans several 64 KiB chunks, exceeds the old cap) with a non-repeating (i%251) byte pattern reads back byte-exact - a chunk-boundary off-by-one would corrupt it and fail. | status=done
src/archive/*.rs (7 doctests) | coherence | medium | DISCOVERED while fixing worker.rs:553: all 7 archive-module doctests used `use archivewalk::...` (a stale former-crate name; the crate is `walkkit` and archive is NOT root-re-exported), so every archive doctest FAILED to compile - archive doctest coverage was 0 and CI --doc was red on this crate. entry.rs additionally constructed `ArchiveEntry { data: vec![...] }` but the field is `Arc<[u8]>` (stale post-refactor example, E0308). | FIXED: `archivewalk::X` -> `walkkit::archive::X` across entry/tar/zip/decompress(x2)/extract(x2); entry.rs data example -> `vec![...].into()`. All 12 walkkit doctests now compile+pass. | status=done
src/codewalker/traverse.rs:135 | walkkit | silent-fallback | medium | REPOPULATE 2026-07-16: CodeWalker::count silently discards traversal errors via filter_map(Result::ok), returning an incomplete or incorrect file count without error notification | FIXED 2026-07-17: added fallible `try_count() -> Result<usize>` that propagates the traversal error (no Vec allocation, unlike walk_and_count), and `count()` now delegates to it and fails closed via `unwrap_or_else(|e| panic!(...))` instead of `filter_map(Result::ok)` silently undercounting. Proving tests (unix, root-guarded via `reproducible = fs::read_dir(blocked).is_err()`): `try_count_surfaces_traversal_error_instead_of_undercounting` (0o000 subtree -> try_count is_err AND count() panics via catch_unwind) + `try_count_matches_walk_on_a_clean_tree`. `cargo test -p walkkit --lib traverse` 9 passed exit 0. | status=done
src/codewalker/mod.rs:248 | walkkit | performance | medium | REPOPULATE 2026-07-16: FileEntry::content reads file content in a loop calling content_chunks, which allocates a new 64KB vector on every single chunk read | FIXED 2026-07-17: content() now reads straight into the single pre-allocated `bytes` buffer via `File::open(..).take(MAX_CONTENT_AUTOLOAD + 1).read_to_end(&mut bytes)` (one buffer grown by amortized doubling) instead of the old loop over content_chunks() that allocated a fresh 64 KiB Vec per chunk and copied it in. `take(MAX+1)` preserves the grew-past-cap detection (FileTooLarge) and never buffers more than MAX+1 bytes; no silent truncation. The streaming content_chunks() iterator is unchanged for streaming callers. Proving test `file_content_reads_exact_chunk_multiple_boundary` (size == 3*READ_CHUNK_SIZE exact, byte-exact, no truncation/over-read); existing `file_content_read_larger_than_prealloc_cap_is_exact` (multi-chunk) and `file_content_reads_grown_file` (growth detection) stay green. The single-buffer/no-per-chunk-alloc property is structural (content() no longer calls content_chunks). `cargo test -p walkkit --lib content` 5 passed exit 0. | status=donesrc/worker.rs:595-599 + :654-662 + :709-711 | walkkit | recall | high | Claude 2026-07-17: in walk_MULTI_thread, a `.gitignore` BUILD error while classifying a child FILE (get_cached_gitignore -> Err at 595) sets `channel_closed = true; break;`. This (a) breaks the child-scan loop, so every child DIRECTORY already collected in `dirs` is DROPPED — its enqueue is guarded by `if !dirs.is_empty() && !channel_closed` at 658 — so ALL sibling subdirectories of the erroring file are NEVER walked (their entire subtrees silently vanish from the scan); and (b) after the loop, `if channel_closed break` (709) EXITS the worker thread entirely, silently reducing parallelism. The variable `channel_closed` conflates a genuine `tx.send` channel closure (a real stop condition) with a recoverable per-entry gitignore error. walk_SINGLE_thread handles the identical case correctly at :264-270 (`Err(e) => { tx.send(Error); continue; }`) — it skips just that entry and keeps walking. So a single malformed `.gitignore` produces DIFFERENT, lossy output under parallelism than single-threaded: a silent recall regression that scales with worker count. | Split the two conditions: on a child gitignore error, `tx.send(WalkItem::Error(e))` and `continue` to the next child (matching single-thread and the dir-level handler at 479-483), and still enqueue the `dirs` accumulated so far. Reserve `channel_closed` STRICTLY for `tx.send(...).is_err()`. Proving test: a tree with a malformed `.gitignore` beside several sibling subdirectories, walked with parallelism>=2, yields the SAME file set as parallelism==1 (all sibling subtrees present), plus one Error item for the bad gitignore.
src/worker.rs:371-377 + :521-524 + :685-689 + :704-708 | walkkit | dedup/one-place | medium | Claude 2026-07-17: the work-termination signal `st.active -= 1; if st.active == 0 && st.queue.is_empty() { state.1.notify_all(); }` is written in FOUR places — the `dec_active` helper (372-376) plus three inline copies (521-524 cycle-skip, 685-689 visited-file dup, 704-708 loop tail) — because those three sites already hold the WorkState lock and cannot call `dec_active` (which re-locks). The completion predicate that decides when the whole parallel walk is DONE is therefore duplicated 4x; any future change to the termination condition (e.g. adding an in-flight counter) that misses one site causes either a permanent HANG (workers park on the condvar forever) or a premature finish (recall loss). A latent-bug-grade ONE-PLACE violation on the crate's most safety-critical invariant. | FIXED 2026-08-07: Extracted `signal_done_locked(st, cv)` encapsulating decrement + completion predicate + condvar notify_all on the held guard. Routed `dec_active` and all 3 inline lock sites through `signal_done_locked`. | status=done
src/worker.rs:195 vs :381 | walkkit | testing | medium | Claude 2026-07-17: there is NO differential test asserting that walk_single_thread and walk_multi_thread produce IDENTICAL (file set, error set) for the same tree (`rg` for single-vs-multi/differential across tests returns nothing). The two paths are structurally different (single enqueues children then builds files on pop; multi builds files inline during the parent scan and enqueues only dirs), and that untested divergence is exactly what let the gitignore-error recall regression above hide. The walker's core contract — output is independent of parallelism — is unverified. | FIXED 2026-08-07: Added `tests/concurrent/differential_walk.rs` (`test_differential_single_vs_multi_thread_equivalence`) building a complex fixture tree with nested dirs, gitignores, binary files, and hidden files, asserting `walk(parallelism=1)` and `walk(parallelism=N)` yield 1:1 identical sorted (file, size) and (error, op) sets across N in {2, 4, 8}. | status=done