Expand description
admin evals command group: judge runs over production AI traffic.
Exposes EvalsCommands — launch an auto-improve judge run, list and
inspect runs, replay a run’s failures, and promote a sampled request into
the golden case set. All inference goes through the process-local
AiService built from the active profile.
Copyright (c) systemprompt.io — Business Source License 1.1. See https://systemprompt.io for licensing details.
Modules§
- list
admin evals list— list evaluation runs.- promote
admin evals promote— promote an AI request into the golden case set.- replay
admin evals replay— replay a run’s failures against the provider.- run
admin evals run— start an evaluation run.- show
admin evals show— inspect a run and its results.