Crate keyhog_profile

Expand description

Record causal performance evidence for one KeyHog run.

Start a Session at the beginning of the production operation. Record macro state changes with Session::transition. Wrap measured work in span, then call Session::finish to produce a versioned RunProfile.

use keyhog_profile::{RunIdentity, RunState, Session, Stage, span};

let identity = RunIdentity::new(
    "0.5.49",
    "detector-digest",
    "config-digest",
    "filesystem",
    "small-text",
    "auto",
);
let mut session = Session::start(identity).expect("start profile");
session.transition(RunState::Scanning);
{
    let _read = span(Stage::SourceRead);
    std::hint::black_box(42);
}
let profile = session.finish(RunState::Completed);

assert_eq!(profile.status, RunState::Completed);
assert_eq!(profile.stages[0].stage, Stage::SourceRead);
assert_eq!(profile.stages[0].calls, 1);

§Runtime ownership

A session owns an isolated Runtime. The session enters that runtime on the calling thread. Propagate a clone explicitly when work crosses a thread boundary. This keeps concurrent runs isolated.

use keyhog_profile::{RunIdentity, RunState, Session, Stage, span};

let identity = RunIdentity::new("0.5.49", "d", "c", "stdin", "stream", "auto");
let session = Session::start(identity).expect("start profile");
let runtime = session.runtime();
std::thread::spawn(move || runtime.scope(|| {
    let _scan = span(Stage::BackendDispatch);
}))
.join()
.expect("join worker");
let profile = session.finish(RunState::Completed);
assert_eq!(profile.stages[0].stage, Stage::BackendDispatch);

§Recording cost

The disabled span path checks one relaxed atomic and does not read the clock. Enabled spans update fixed atomic counters indexed by MetricId. They do not allocate, hash metric names, or format text. Vector construction, JSON serialization, and report analysis run only when counters are drained or a session is finished.

cargo bench -p keyhog-profile --bench overhead_budget enforces absolute median budgets for disabled checks, aggregate spans, and causal spans. The regular CI workflow runs this gate with an optimized benchmark build.

§Metrics and collectors

METRICS is the static registry for metric names, kinds, and units. A collector implements SnapshotCollector and reports a CollectorCapability before sampling. The default process-metrics feature samples process CPU time, resident memory, virtual memory, and thread count. Disable default features when you need stage timing without platform process sampling. The profile then reports the collector as disabled instead of silently emitting unavailable measurements.

§Persisted records

PROFILE_SCHEMA identifies the profile envelope. Every persisted component also carries its own numeric version. Missing component versions decode as version one for compatibility with early records. Compare identity fields, collector capabilities, workload state, and metric units before comparing measurements from two profiles.

§Privacy

The profiler records counts, durations, run identity, execution choices, and process resources. Do not use source content, credentials, raw URLs, or sensitive paths as identity labels. RunProfile::render_text and RunProfile::to_json_pretty serialize the labels supplied by the caller.

Re-exports§

pub use insight::BackendAttributionV2;
pub use insight::BottleneckKindV2;
pub use insight::FindingV2;
pub use insight::InsightCoverageV2;
pub use insight::MemoryInsightV2;
pub use insight::ParallelismInsightV2;
pub use insight::PhaseInsightV2;
pub use insight::RunInsightV2;
pub use insight::SerialRegionV2;
pub use insight::SerialScopeV2;
pub use insight::StageAttributionV2;
pub use insight::StageMemoryV2;
pub use insight::ThroughputInsightV2;
pub use insight::RUN_INSIGHT_V2_VERSION;

Modules§

insight: Turn one recorded profile into the answers an operator asked for.

Structs§

AllocationEvidenceV2: Allocator evidence: exact counts when a crate::TrackingAllocator is installed, an explicit capability gap otherwise.
AllocationSlotV2: Per-slot allocation counters at one instant.
AllocationSnapshotV2: Process-wide allocation counters at one instant, split by owning stage.
AllocationTotalsV2: Session-window allocation totals with live and peak levels at the end.
AnnotationV2: One bounded typed numeric annotation on the run timeline.
ArtifactIntegrityV2: Digest protecting the canonical profile artifact.
BatchRouteV2: One actual route completed for a measured batch.
BlockedWaitRecordV2: Blocked wait time attributed separately from runnable execution for one stage.
BuildIdentityInput: Build-specific values supplied by the final binary crate.
BuildIdentityV2: Exact executable and toolchain identity used by one run.
CacheEffectivenessV2: Hit and miss counts for one reuse cache.
CacheLayerV2: State and generation identity for one cache layer.
CausalParent: Portable causal parent captured from a runtime’s current span context.
CausalProfileV2: Versioned causal profile envelope and measurements.
CausalRunIdentityV2: Comparison identity joining every timing-relevant dimension.
CollectorCapability: Host-specific availability report for one collector.
ComparisonDifference: One identity or workload field that prevents a valid performance comparison.
ConfigIdentityInput: Canonical resolved configuration values supplied by the final operator.
ConfigIdentityV2: Canonical resolved configuration and policy identity.
ContextGuard: Thread context guard returned by Runtime::enter.
CounterSpan: Time a sub-stage region into a crate::CounterId instead of a stage.
CpuFrequencySampleV2: Aggregate CPU frequency across all CPUs at one sample instant.
DaemonIdentityV2: Daemon mode and request linkage for one run.
DecisionTimer: Time a region whose measurement drives a decision, profiled or not.
DecodeRetentionEvidenceV2: Decode expansion and retained-buffer evidence.
DetectorIdentityInput: Detector-specific values supplied after the scanner compiles its effective corpus.
DetectorIdentityV2: Detector corpus and compiled execution-plan identity.
DistributionBucketV2: One exact logarithmic bucket of a caller-recorded value distribution.
EventLossCounts: Exact reasons typed timeline records were not retained.
EventStreamV2: Bounded event stream with explicit availability and loss accounting.
FaultEvidenceV2: Page-fault deltas across one run from proc stat.
HardwareCounterCollector: Linux perf, Windows cycle-time, or stub collector for hardware counters.
HardwareCounterSampleV2: One absolute hardware-counter reading taken by HardwareCounterCollector.
HardwareCounterSetV2: Counter deltas and derived ratios across one run (session-thread scope).
HardwareRunEvidenceV2: Complete CPU hardware evidence for one run.
HostIdentityV2: Exact host and operating environment used by one run.
IndexedCounterRecordV2: One indexed counter family, summed per slot across every worker.
IoEvidenceV2: Process IO deltas across one run from /proc/self/io.
LatencyBucketV2: One exact logarithmic latency bucket.
LatencyDistributionV2: Allocation-free hot-path call latency distribution for one micro-function.
MemoryEvidenceV2: Memory levels at the end of one run, including the kernel high water.
MetricDescriptor: Static metric metadata. Every string is embedded in the binary.
MetricDistributionV2: Caller-recorded logarithmic distribution for one typed metric.
NetworkEvidenceV2: Network evidence: process-level counters or an explicit gap, plus retry activity aggregated from caller annotations.
NetworkProcessCountersV2: Per-process network counters where the host exposes them.
OutcomeIdentityV2: Terminal outcome and result identity for one run.
PointEventV2: One bounded instantaneous event with a typed numeric payload.
PressureEvidenceV2: Kernel pressure-stall averages at the end of one run.
PressureThermalCollector: Pressure-stall and thermal collector backed by /proc/pressure and sysfs thermal zones on Linux.
PressureThermalSampleV2: One absolute pressure and thermal reading.
ProducerIdentityV2: Producer identity for the code that emitted an artifact.
ProfileComparison: Deterministic comparison of two profile records.
ProfileEnvelopeV2: Self-describing envelope for a v2 causal profile.
QueueDepthV2: Current depth and high-water mark for one bounded queue slot.
QueueLinkLossCounts: Exact reasons queue causality records were not retained.
QueueLinkV2: One matched producer enqueue and consumer dequeue through a bounded queue.
ResourceSample: Process resource observation associated with a run-state boundary.
ResourceSnapshot: Process resource observation at a macro boundary.
ResourceUsage: Resource change across a completed profile session.
RetryRecordV2: Retry attempts recorded for one cause.
RouteIdentityV2: Requested, selected, completed, and recovered backend identity.
RunIdentity: Identity and execution choices required to compare two run records honestly.
RunProfile: Complete replayable profile record.
RunSpanHardwareV2: Run-level cycle and instruction totals joined from span records.
Runtime: Owned fixed-stage metric storage that can be propagated across worker boundaries.
SamplingPolicy: Deterministic bounded policy for retaining expensive detail events.
SchedulerCollector: Context-switch, migration, and runqueue-delay collector.
SchedulerEvidenceV2: Scheduler activity deltas across one run with an explicit source per field.
SchedulerSampleV2: One absolute scheduler-activity reading from procfs and perf software events.
SchemaVersionV2: Independent major and minor version for one schema family.
Session: One causal profiling session with isolated owned metric storage.
SessionActive: Reserved error type for profile-session initialization failures.
SourceIdentityInput: Safe source adapter names and hashed target values supplied by the operator.
SourceIdentityV2: Safe source adapter and target identity.
SourcedEvidenceV2: One measured field plus the facility that produced or was asked for it.
Span: Allocation-free stage guard. It contains no start timestamp while disabled.
SpanHardwareAggregationV2: Cold-path CPI aggregation over one drained span set.
SpanHardwareV2: Raw per-span cycle and instruction readings attached at span begin and end.
SpanRecordV2: One nested or linked causal interval.
StageAllocationV2: Per-stage allocation ownership; metric_id is None for the root slot that owns allocations made outside any recorded span.
StageComparison: Exact aggregate difference for one stage.
StageConcurrencyV2: Wall-clock occupancy of one micro-function across every worker.
StageHardwareV2: Per-stage cycle and instruction totals joined from span records.
StageMeasurement: One aggregate fixed-stage measurement.
StateMeasurement: One completed macro state with its wall time and boundary resource deltas.
StateTransition: One run-state transition relative to session start.
SystemIoCollector: Faults and process-IO collector backed by /proc/self/stat and /proc/self/io on Linux.
SystemIoSampleV2: One absolute faults-and-IO reading from procfs.
SystemRunEvidenceV2: Complete memory, IO, and system evidence for one run.
ThermalEvidenceV2: Thermal state at the end of one run.
ThreadCpuV2: One thread’s cumulative CPU consumption at a sample instant.
ThreadHardwareV2: Per-thread cycle and instruction totals joined from span records.
ThreadUtilizationCollector: Per-thread CPU utilization and frequency sampler.
ThreadUtilizationSampleV2: Per-thread CPU census at one instant, bounded with explicit loss.
ThreadUtilizationV2: Per-thread CPU consumption and utilization across one run.
TopologyCollector: Static CPU topology, affinity, NUMA, and cgroup limit collector.
TopologyEvidenceV2: Static CPU topology, affinity, NUMA, and cgroup CPU limits for one run.
TrackingAllocator: Global allocator that counts allocations, bytes, and live memory with per-stage ownership. Install with #[global_allocator]. Without the allocation-tracking feature every method inlines to the system allocator.
TypedMetricRecordV2: One typed counter or gauge materialized from fixed runtime storage.
UtilizationEvidenceV2: Per-thread utilization, effective parallelism, and frequency series.
WorkerImbalanceV2: Work-stealing imbalance evidence merged from every worker shard.
WorkerLoadV2: Per-worker load observed from one counter shard.
WorkerOccupancyRowV2: Busy, blocked, and idle time for one worker across the whole session.
WorkerOccupancyV2: Pool-wide busy versus idle accounting merged from every worker shard.
WorkloadIdentityInput: Measured byte and unit totals used to classify comparable workload shapes.
WorkloadIdentityV2: Measured workload shape used to classify comparable runs.
WorkloadMeasurements: Optional byte domains whose totals distinguish source, expansion, decode, and dispatch work.

Enums§

AnnotationId: Stable identifier for a numeric annotation attached to the run timeline.
Attribution: Optional attribution for work performed inside a derived input.
CacheId: A reuse cache the profiler reports hit and miss counts for.
CacheLayerKindV2: Cache families whose preparation state changes run cost.
CacheState: Cache state that materially changes run cost.
CollectorAvailability: Whether a collector can produce measurements on this host.
CollectorId: Stable identity of a profiling data collector.
CounterId: Type-safe identifier for an additive monotonic metric.
CoverageStateV2: Whether scanner coverage was complete, partial, or unknown.
DaemonState: Daemon state that materially changes startup and resident work.
Detail: How much performance measurement this process performs.
EventId: Stable identifier for an instantaneous causal event.
Evidence: A measured value or an explicit reason why no value exists.
EvidenceGap: Why a v2 evidence field has no measured value.
GaugeId: Type-safe identifier for a latest-value metric.
HardwareFieldSourceV2: Exact host facility that produced (or was asked for) one hardware field.
IndexedCounterId: An additive counter that exists once per caller-owned slot.
IoCacheStateV2: Explicitly observed page-cache state for one source of IO work.
MacroStageId: Stable identifier for a top-level production pipeline stage.
MetricId: Stable wire identifier for a metric recorded by keyhog-profile.
MetricKind: Measurement behavior associated with a metric.
MetricUnit: Stable unit associated with a metric value.
QueueId: Bounded fixed set of queue slots for causality links and depth gauges.
RetryCause: Why one operation was attempted again.
RunState: Coarse causal state of a profiling run.
Stage: Stable micro-function identifier shared by scanner, source, verifier, and reporter paths.
WorkOrigin: Causal origin of the work measured by one span.

Constants§

CACHE_EFFECTIVENESS_V2_VERSION
CAUSAL_IDENTITY_V2_VERSION
CAUSAL_PROFILE_V2_VERSION
COLLECTOR_CAPABILITY_VERSION
COMPARISON_DIFFERENCE_VERSION
EVENT_SCHEMA_VERSION
EXPORTER_VERSION
HARDWARE_EVIDENCE_V2_VERSION
INDEXED_COUNTER_SLOTS: Number of slots in every indexed counter family.
INDEXED_COUNTER_V2_VERSION
MAX_ANNOTATIONS
MAX_BATCH_ROUTES: Hard cap on retained batch-route records; further routes count as drops.
MAX_POINT_EVENTS
MAX_QUEUE_LINKS: Maximum pending enqueues and completed links retained per runtime.
MAX_RECORDED_SPANS: Maximum number of causal span records retained by one profiling runtime.
MAX_SAMPLE_THREADS: Maximum retained threads per utilization sample; excess is counted.
MAX_UTILIZATION_SAMPLES: Maximum retained utilization samples per session; excess is counted, never stored.
METRIC_REGISTRY_VERSION
PROFILE_COMPARISON_VERSION
PROFILE_ENVELOPE_V2_VERSION
PROFILE_SCHEMA: Stable wire schema for persisted profiling records.
PROFILE_SCHEMA_V2
PROFILE_SCHEMA_V2_MAJOR
PROFILE_SCHEMA_V2_MINOR
RESOURCE_SAMPLE_VERSION
RESOURCE_SNAPSHOT_VERSION
RESOURCE_USAGE_VERSION
RETRY_RECORD_V2_VERSION
ROOT_SLOT: Slot index for allocations made outside any recorded span.
RUN_IDENTITY_VERSION
RUN_PROFILE_VERSION
SPAN_HARDWARE_V2_VERSION
STAGE_COMPARISON_VERSION
STAGE_CONCURRENCY_V2_VERSION
STAGE_MEASUREMENT_VERSION
STAGE_SLOTS: Stage attribution slots: one per crate::Stage plus one root slot for allocations made outside any recorded span.
STATE_MEASUREMENT_VERSION
STATE_TRANSITION_VERSION
SYSTEM_EVIDENCE_V2_VERSION
WORKER_OCCUPANCY_V2_VERSION
WORKLOAD_MEASUREMENTS_VERSION

Statics§

METRICS: Allocation-free registry in numeric MetricId order.

Traits§

SnapshotCollector: Portable lifecycle for a collector that snapshots one metric family.

Functions§

add_backend_dispatched_bytes: Add bytes submitted once to the completed backend route in the current profile.
add_counter: Increment one typed monotonic counter in the current profiling runtime.
add_derived_decoder_bytes: Add bytes produced by accepted decode-through work in the current profile.
add_indexed_counter: Add to one slot of an indexed counter family.
add_input_bytes: Add source bytes processed by the current profile.
add_input_units: Add source units such as files, objects, responses, or chunks.
add_stage_bytes: Attribute bytes to one micro-function so its throughput can be reported.
aggregate_span_hardware: Join span-attached counter readings into per-stage, per-thread, and run CPI.
allocation_snapshot: Snapshot the process-wide allocation counters; all zeros when the allocation-tracking feature is disabled.
allocation_tracking_installed: Whether any tracked allocation has run through a TrackingAllocator.
blocked: Record one blocked wait interval separately from runnable execution.
compare_profiles: Compare two runs only after checking every identity field that changes timing.
counter_span: Start a sub-stage measurement that accumulates into one counter.
current_causal_parent: Capture a portable token naming the current runtime’s causal parent.
current_runtime: Return a clone of the runtime current on this thread.
current_task_id: Current thread’s caller-assigned task identity, or zero when unset.
current_work_origin: Current thread’s causal work origin.
decision_timer: Start a decision-driving measurement of one micro-function.
detail: Return the measurement level requested for this process.
enabled: Return whether fixed-stage profiling is active on the calling thread.
instrument_future: Propagate the current runtime and causal parent while polling one future.
instrument_future_with_parent: Propagate the current runtime with an explicit portable causal parent.
milli_ratio: Exact integer ratio in thousandths; None when the denominator is zero.
record_annotation: Record one typed numeric annotation on the current run timeline.
record_batch_route: Record the requested, selected, and completed route for one completed batch.
record_cache_hit: Count one consultation of a reuse cache that was served from the cache.
record_cache_miss: Count one consultation of a reuse cache that had to recompute or refetch.
record_distribution: Record one observed value into a metric’s bounded logarithmic distribution.
record_event: Record one typed instantaneous event with a numeric payload.
record_fs_metadata_latency_ns: Record one filesystem metadata (stat/readdir) latency inside a Stage::SourceWalk instrumented path.
record_fs_open_latency_ns: Record one filesystem open latency inside a Stage::SourceWalk or Stage::SourceRead instrumented path.
record_fs_read_latency_ns: Record one filesystem read latency inside a Stage::SourceRead instrumented path.
record_io_cache_state: Record one explicitly observed page-cache state for IO work.
record_network_bytes: Add network bytes a caller read and wrote; process-level counters are not visible to the profiler on every host, so callers report their own IO.
record_network_latency_ns: Record one network request latency observed by a caller.
record_network_request: Count one completed network request.
record_queue_depth_dequeue: Decrement one queue’s depth gauge, saturating at zero.
record_queue_depth_enqueue: Increment one queue’s depth gauge and refresh its high-water mark.
record_queue_dequeue: Record the consumer dequeue matching one earlier record_queue_enqueue.
record_queue_enqueue: Record one producer enqueue for later matching by record_queue_dequeue.
record_retained_buffer_bytes: Record the current retained-buffer level in bytes; the runtime keeps the running high water alongside the latest value.
record_retry: Count one retry attempt, whether or not the retry eventually succeeded.
record_sampled_event: Record one expensive detail event under a deterministic bounded sampling policy.
reset: Discard fixed-stage counters and input totals in the current runtime.
reset_allocation_peaks: Restart peak-live tracking from the current live levels.
serial_span: Declare that this region runs with the worker pool idle.
set_attribution: Replace this thread’s attribution and return its previous value.
set_detail: Set the measurement level for this process and enable or disable the calling thread’s standalone profiling runtime to match.
set_enabled: Enable or disable the calling thread’s standalone profiling runtime.
set_gauge: Replace one typed latest-value gauge in the current profiling runtime.
set_queue_depth: Replace one queue’s depth gauge and refresh its high-water mark.
set_task_id: Replace this thread’s caller-assigned task identity and return the previous.
set_work_origin: Replace this thread’s causal work origin and return its previous value.
span: Start one fixed-stage measurement.
span_with_parent: Start one fixed-stage measurement with an explicit portable causal parent.
take_input_totals: Atomically read and clear aggregate input bytes and units.
take_metric_distributions: Drain caller-recorded value distributions from the current runtime.
take_stage_measurements: Atomically drain fixed counters and materialize stable stage records.
take_typed_metrics: Drain typed counters from the current session or standalone runtime.