1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
//! Env-gated per-query vector-search profiling + open-time vector-base
//! build profiling.
//!
//! Activation is opt-in via the `VALISE_QUERY_PROFILE` env var, captured
//! **once per process** into an `OnceLock<bool>` — the disabled hot-path
//! cost is a single branch on a cached bool, no `Instant::now()` calls
//! and no env lookups per query. This mirrors the repo's profiling
//! idiom (`VALISE_INGEST_PROFILE` → [`super::profile::IngestProfile`],
//! `VALISE_OPEN_PROFILE` → open-phase timings) but reports **structured
//! values** through a thread-local slot instead of stderr lines, so a
//! bench harness reads numbers instead of parsing logs.
//!
//! When enabled, each `vector_search` call that goes through a
//! sign-sketch path (QAM `sketch_then_rerank_impl` in
//! `vector_search.rs`, UPQ `upq_sketch_then_rerank` in `upq_search.rs`)
//! records a [`QueryProfile`] for the calling thread; the caller drains
//! it with [`last_query_profile`]. The open-time vector-base build
//! (`build_vector_base_ptrs` — pointer walk + per-family caches +
//! sign-sketch derivation) records a [`VectorOpenProfile`] the same way
//! ([`last_vector_open_profile`]); the open-side recording also honors
//! `VALISE_OPEN_PROFILE` so it composes with the existing open profiling.
use Cell;
use OnceLock;
use ;
/// Per-stage wall-clock + candidate-count breakdown of one
/// `vector_search` call through a sign-sketch path. Stage boundaries
/// match the existing delimitation in `vector_search.rs` /
/// `upq_search.rs`:
///
/// - `prep`: space/codec resolution, admission, query rotation and
/// sign-sketch packing — everything before the Hamming scan.
/// - `stage1_sketch`: the sign-sketch Hamming scan + counting-sort
/// candidate selection over all active rows of the space.
/// - `stage2_rerank`: candidate resolution + integer rerank (QAM
/// sliding i8 / UPQ decoded-i8 cache dot) + bounded top-pool
/// selection.
/// - `stage3_exact`: the exact rescoring pass over the survivors.
/// `Duration::ZERO` unless the query asked for
/// [`super::VectorFidelity::Full`].
///
/// `prep + stage1 + stage2 + stage3` ≈ the whole call (the residue is
/// dispatch + final top-k descriptor resolution, both `O(k)`).
/// Wall-clock breakdown of one `build_vector_base_ptrs` run — the
/// open-time (or post-commit) derivation of everything `vector_search`
/// reads from memory instead of disk.
/// `VALISE_QUERY_PROFILE` presence, resolved once per process.
pub
/// Open-side gate: the structured open profile is recorded when either
/// profiling env var is present. Called only from the (rare) vector-base
/// build, so the extra env lookup is not on a hot path.
pub
thread_local!
pub
pub
/// Take (and clear) the profile recorded by the calling thread's most
/// recent sketch-path `vector_search` call. `None` when
/// `VALISE_QUERY_PROFILE` is unset, when the last call on this thread took
/// the brute-force path, or when it was already taken.
/// Take (and clear) the profile recorded by the calling thread's most
/// recent vector-base build (lazy first-search build or post-commit
/// rebuild). `None` unless `VALISE_QUERY_PROFILE` or `VALISE_OPEN_PROFILE`
/// was set when the build ran on this thread.
/// Boundary lap timer: returns the elapsed time since `*mark` and
/// advances the mark to now. A `None` mark (profiling disabled) costs
/// one branch and returns `Duration::ZERO` — no clock reads.
pub