# FFF benchmark history
Maintainer-only historical benchmark notes for the removed FFF content-search experiment and current `fffind` path-search comparisons. Current shipped behavior: `ffgrep` uses `rg` first with bounded Rust fallback; `fffind` uses local bounded Rust path search. This page is benchmark history, not shipped tool-contract truth.
## Current status
| `fffind` path search | Retained as local Rust path discovery | Third-party focused path runs beat substring-walk baseline in cold, warm, filtered, and repeated scenarios. | Useful for path discovery; not content search. |
| FFF content-search experiment | Removed from production `ffgrep` | Historical runs showed promising content-search latency, but the adapter/dependency path was removed. | Do not treat FFF as a shipped `ffgrep` backend. |
| `ffgrep` content search | `rg` first, Rust fallback | Current implementation no longer has an FFF adapter. | README, prompt, and tool docs should describe `rg`/Rust only. |
## Accepted historical contract
ADR-0034 recorded a prior accepted contract for a possible FFF-backed `ffgrep` backend. That contract is historical after removal of the adapter. Current `ffgrep` keeps the same tool name/schema and safety guarantees through `rg` plus bounded Rust fallback.
Current visible contract:
- Output lines stay `path:line:content`.
- Paths are cwd-relative and slash-normalized when results are under runtime cwd.
- `limit` remains a hard visible-line cap after context expansion.
- Metadata includes `engine`; current success reports `rg` or `rust`.
- `fffind` remains separate path discovery; `ffgrep` remains content search.
## `fffind` path benchmark
### Default repo run
```sh
FFFIND_BENCH_QUERIES='lib,tools,profile,readme,fff' \
FFFIND_BENCH_PATHS='src,docs' \
FFFIND_BENCH_REPETITIONS=30 \
cargo test fffind_benchmark -- --ignored --nocapture
```
### Focused third-party run
```sh
FFFIND_BENCH_ROOT=/path/to/third-party-corpus \
FFFIND_BENCH_PATHS='Lumen,ai-sdk,opencode,zed' \
FFFIND_BENCH_REPETITIONS=50 \
cargo test fffind_benchmark -- --ignored --nocapture
```
### Longer stability run
```sh
FFFIND_BENCH_ROOT=/path/to/large/repo \
FFFIND_BENCH_REPETITIONS=200 \
FFFIND_BENCH_CWD_CHURN=10 \
cargo test fffind_benchmark -- --ignored --nocapture
```
### Latest path results
| Cold unfiltered | `160.166ms` | `293.787ms` | `fffind` faster |
| Warm unfiltered | `2.312ms` | `249.743ms` | `fffind` much faster |
| Warm filtered | `10.406ms` | `20.088ms` | `fffind` faster |
| Repeated query | `1.670ms` | `230.580ms` | `fffind` much faster |
Local small-repo path run: baseline won overall, but `fffind` won warm unfiltered. Interpretation: large/focused corpora show stronger `fffind` value; small repos can favor current simple walk baseline.
## Historical content-search benchmark
These commands and environment variables remain diagnostic harness history. They do not imply a production FFF-backed `ffgrep` backend.
### Local repo run
```sh
FFF_CONTENT_BENCH_PATTERNS_PLAIN='ToolRuntime,ffgrep,serde_json,ProfileResult,content_search' \
FFF_CONTENT_BENCH_PATTERNS_REGEX='fn [a-zA-Z0-9_]+,pub\(crate\) struct,impl ToolRuntime' \
FFF_CONTENT_BENCH_PATHS='src,docs' \
FFF_CONTENT_BENCH_REPETITIONS=25 \
FFF_CONTENT_BENCH_LIMITS='20,100' \
FFF_CONTENT_BENCH_CONTEXTS='0,2' \
cargo test content_search_benchmark -- --ignored --nocapture
```
### Focused third-party rerun used for historical gate
```sh
FFF_CONTENT_BENCH_ROOT=/path/to/third-party-corpus \
FFF_CONTENT_BENCH_PATHS='Lumen,ai-sdk,opencode,zed' \
FFF_CONTENT_BENCH_REPETITIONS=2 \
FFF_CONTENT_BENCH_CONTEXTS='0' \
FFF_CONTENT_BENCH_BASELINE=both \
cargo test content_search_benchmark -- --ignored --nocapture
```
### Historical content results
| Plain | `50/50` | `3.544x` | `0` |
| Regex | `30/30` | `5.306x` | `0` |
Interpretation at time of run: FFF content search passed performance gates. Current implementation later removed that backend, so these results remain historical only.
## Environment knobs
### Path benchmark
| `FFFIND_BENCH_ROOT` | `CARGO_MANIFEST_DIR` | Repo root to benchmark. Output uses basename plus root hash, not absolute path. |
| `FFFIND_BENCH_QUERIES` | `lib,tools,profile,readme,fff` | Comma-separated fuzzy path queries. |
| `FFFIND_BENCH_PATHS` | `src,docs` | Comma-separated directory filters relative to root. Missing paths warn to stderr and skip. |
| `FFFIND_BENCH_REPETITIONS` | `25` | Warm repetitions per scenario; minimum `1`. |
| `FFFIND_BENCH_LIMIT` | `50` | `fffind` result limit; clamped to tool max `200`. |
| `FFFIND_BENCH_CWD_CHURN` | `0` | Optional child-cwd runtime churn count for resource growth checks. |
| `FFFIND_BENCH_BASELINE` | `ignore_walk` | `ignore_walk` or `none`. Baseline is substring file-path walk, not fuzzy ranking. |
| `FFFIND_BENCH_SHOW_RESULTS` | unset | Set `1` to include top 10 cold result paths in JSON output. |
### Historical content benchmark
| `FFF_CONTENT_BENCH_ROOT` | `CARGO_MANIFEST_DIR` | Repo root to benchmark. Output uses basename plus root hash. |
| `FFF_CONTENT_BENCH_PATHS` | `src,docs` | Comma-separated relative path scopes. Absolute and parent paths are rejected. |
| `FFF_CONTENT_BENCH_PATTERNS_PLAIN` | repo-safe code tokens | Comma-separated literal patterns. Hidden from output unless explicitly shown. |
| `FFF_CONTENT_BENCH_PATTERNS_REGEX` | repo-safe Rust regexes | Comma-separated regex patterns. Hidden from output unless explicitly shown. |
| `FFF_CONTENT_BENCH_REPETITIONS` | `25` | Warm iterations per scenario; minimum `1`. |
| `FFF_CONTENT_BENCH_LIMITS` | `20,100` | Comma-separated page limits. |
| `FFF_CONTENT_BENCH_CONTEXTS` | `0,2` | Comma-separated before/after context counts. |
| `FFF_CONTENT_BENCH_BASELINE` | `both` | `ffgrep`, candidate engine, or `both`. |
| `FFF_CONTENT_BENCH_CONTENT_INDEXING` | `1` | Historical candidate content-indexing flag. |
| `FFF_CONTENT_BENCH_CWD_CHURN` | `0` | Optional child-scope churn count for resource growth checks. |
| `FFF_CONTENT_BENCH_SHOW_PATTERNS` | unset | Set `1` to print pattern text; default output uses hashes/labels only. |
## Output schema
Stdout lines start with a prefix followed by JSON:
```text
fffind_benchmark {"schema":1,"scenario":"warm_unfiltered_files","root_label":"magi-code","root_hash":"...","query":"tools","kind":"files","path":null,"iterations":30,"elapsed_ms":{"min":4.1,"p50":6.2,"p95":11.8,"max":14.0},"matches_returned":50,"total_matched":134,"truncated":true,"index_ready":true,"rss_kb_delta":18432,"fd_delta":4,"thread_delta":3,"baseline":{"engine":"ignore_walk","p50_ms":41.0,"p95_ms":62.0,"matches_returned":50,"total_matched":134}}
content_search_benchmark {"schema":1,"scenario":"warm_scoped_plain","root_label":"_third-party","root_hash":"...","path_label":"Lumen","pattern_label":"plain_01","pattern_hash":"...","mode":"plain","engine":"candidate_plain","iterations":2,"limit":100,"context":0,"elapsed_ms":{"min":3.2,"p50":5.8,"p95":11.4,"max":18.1},"successes":2,"failures":0,"matches_returned":100,"truncated":true,"timed_out":false,"index_ready":true,"rss_kb_delta":32768,"fd_delta":3,"thread_delta":4}
```
Rules:
- Output omits matched content by design.
- `root_label` is basename; `root_hash` hashes canonical root.
- Content benchmark pattern text is hidden by default; use `pattern_label` and `pattern_hash` for comparisons.
- Diagnostics such as skipped paths go to stderr and are not benchmark data.
- Final `summary` line includes speedups, failure counts, compatibility/resource booleans, and `step4_exploration.recommended`.
## Interpretation gates
- `ffgrep` FFF backend exploration: historical and no longer production truth.
- Production `ffgrep`: current implementation is `rg` first with bounded Rust fallback.
- TUI autocomplete exploration: use path-search results only. Continue only if warm filtered/mixed p95 and repeated same-query p95 remain low enough for interactive use and cwd-churn resources stay stable.