pub struct RubricReport {
pub rubric_nodes: usize,
pub leaf_count: usize,
pub satisfied_leaves: usize,
pub coverage: f64,
pub score: f64,
pub score_std: f64,
pub generation_calls: usize,
pub judge_model: String,
pub abstain_leaves: usize,
pub abstain_rate: f64,
pub leaf_verdict_repeats: usize,
pub aggregation_level: String,
}Expand description
维度 7:Rubric 层级完整性打分(v14 C 组,CodeWikiBench 协议,–judge 启用时)
协议(arXiv:2510.24428):从被测仓库 README + docs/ 提取 docs_tree → LLM 独立生成 3 份层级 rubrics(requirement + weight 1-3 + 递归 sub_tasks) → 第 4 次调用语义合并(名称相似度 >70% 的节点合并,权重取均值)→ 裁判对每个叶子以产物为证据做 0/1 满足判定 → 加权自底向上聚合: S(n) = Σw(c)·S(c)/Σw(c);叶子 σ 按二项近似 sqrt(p(1-p)),非叶子 σ² = Σ(w²σ²)/Σw² 传播;Coverage = 满足叶子数 / 总叶子数。 与 TQS 并存:TQS 是相对对比(新旧文档),rubric 是绝对质量(对 仓库意图的覆盖度)。mock/无 key 时 None(“失败只告警“策略)。
Fields§
§rubric_nodes: usize合并后的 rubric 节点总数(含非叶子)
leaf_count: usize叶子(可判定项)数
satisfied_leaves: usize判定为满足(1)的叶子数
coverage: f64覆盖率 = satisfied / 有效判定叶子(leaf − abstain;0 时为 1.0 空集约定)——t04 起 abstain 叶子排除出分母(2606.00093 item 8: exclude 模式报 covered-subset 性能)
score: f64加权总分 S(0-1;×10 可与 TQS 0-10 口径对比)
score_std: f64加权标准差 σ_R(二项近似 + 权重传播)
generation_calls: usize生成轮次(独立生成 3 次 + 合并 1 次 = 4 次 LLM 调用)
judge_model: String裁判模型(config.llm.model)
abstain_leaves: usizet04:abstain 叶子数(判定调用/解析失败——不再 recode 为 false; 2606.00093 item 6:recode 改变 estimand,排除需显式报告)
abstain_rate: f64t04:abstain 率 = abstain / 总叶子(2606.00093 item 7: abstention/tie/invalid 率本身作为指标报告)
leaf_verdict_repeats: usizet04:叶子判定轮数(3 次多数投票;1:2 争议升级 5 次——2606.13685: 单次判定保真仅 86.6%,3 trials 达约 90%)
aggregation_level: Stringt04:聚合层级声明(叶子级多数投票 → 权重自底向上;2606.00093 item 10 要求声明 micro/macro/item-level)
Trait Implementations§
Source§impl Clone for RubricReport
impl Clone for RubricReport
Source§fn clone(&self) -> RubricReport
fn clone(&self) -> RubricReport
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for RubricReport
impl Debug for RubricReport
Auto Trait Implementations§
impl Freeze for RubricReport
impl RefUnwindSafe for RubricReport
impl Send for RubricReport
impl Sync for RubricReport
impl Unpin for RubricReport
impl UnsafeUnpin for RubricReport
impl UnwindSafe for RubricReport
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more