Expand description
asd probe run — golden benchmark harness for ASD ranking/classification.
Reads .asd/probes.toml (relative to the DB), executes each probe as a
subprocess call to asd --db <path> <command> <args...>, parses the JSON
output, and evaluates structural assertions. On failure, attaches the full
debug payload so an agent can read boost_debug/classification_debug and
make a single targeted heuristic change per iteration.
asd probe add appends a new [[probe]] entry to probes.toml.
Filtering flags (may combine):
–name
Output flags:
–json emit results as a JSON object (includes duration_ms per probe,
wall_time_ms, worker_count, total/passed/failed, slowest top-5,
slow_violations list)
–fail-slow
Assertion kinds:
file_not_in_key — no item in JSON array key has field containing value
file_in_key — at least one item in array key has field containing value
qname_rank_lte — result whose qname contains fragment appears at rank ≤ max_rank
qname_rank_eq — result whose qname contains fragment appears at EXACTLY exact_rank (precision-mode; Plan J t-019)
result_count_lte — results array length ≤ max
cluster_winner_kind_not — cluster_debug entry matching doc_stem winner kind ≠ kind_not
cluster_winner_qname_contains — cluster_debug entry matching doc_stem winner qname contains fragment
no_duplicate_summaries — no two suggested_entries share the same summary per symbol
qname_not_in_results — no result’s qname contains fragment (feedback suppression check)
boosted_outranked_contains — boosted_outranked has an entry containing fragment
ambiguous_terms_nonempty — ambiguous_terms array is non-empty (broad query uncertainty check)
scoped_suggestions_nonempty — scoped_suggestions array is non-empty
scoped_suggestions_contains — scoped_suggestions contains an entry matching fragment
uncertainty_level_lte — uncertainty.level ≤ max_level (low/medium/high/critical)
uncertainty_reason_contains — uncertainty.reasons[*].code contains code
uncertainty_action_eq — uncertainty.recommended_action equals action
recovery_suggestions_nonempty — uncertainty.recovery_suggestions is non-empty
recovery_suggestion_estimated — at least one recovery suggestion has estimated_recovery = strength
feedback_summary_gte — feedback_summary[field] ≥ min_value (e.g. suppressed ≥ 1)
feedback_summary_eq — feedback_summary[field] == value
feedback_rules_contains — feedback_summary.rules_applied contains rule
field_gte — output[dot.path] ≥ min_value (numeric)
field_eq — output[dot.path] equals expected string
array_field_count_lte — length of dot-path array field ≤ max_count
array_field_count_gte — length of dot-path array field ≥ min_count
workflow_steps_contains — workflow.steps_detected contains step
evidence_score_gte — workflow.evidence_quality.evidence_quality_score ≥ min_value
data_quality_state_eq — trust.data_quality.state equals expected string
feedback_state_eq — feedback_state[field] == value (bool)
feedback_state_field_eq — feedback_state[field] equals expected string
feedback_coverage_eq — feedback_summary.coverage equals expected string
array_field_contains — dot-path array field contains value string
array_field_excludes — dot-path array field does NOT contain value string
uncertainty_exact_symbol_match — uncertainty.exact_symbol_match == expected (bool)
uncertainty_primary_source_eq — uncertainty.sources.primary equals expected string
uncertainty_source_gte — uncertainty.sources[source] >= min_value
all_items_have_field — every object in a dot-path array has the named field set
file_field_contains — specific array item (by file_fragment) has field containing substring
Special probe command (no assert block needed): hydrate-roundtrip — write sentinel, sync, hydrate into fresh DB, verify survival