Skip to main content

RubricReport

Struct RubricReport 

Source
pub struct RubricReport {
    pub rubric_nodes: usize,
    pub leaf_count: usize,
    pub satisfied_leaves: usize,
    pub coverage: f64,
    pub score: f64,
    pub score_std: f64,
    pub generation_calls: usize,
    pub judge_model: String,
    pub abstain_leaves: usize,
    pub abstain_rate: f64,
    pub leaf_verdict_repeats: usize,
    pub aggregation_level: String,
}
Expand description

维度 7:Rubric 层级完整性打分(v14 C 组,CodeWikiBench 协议,–judge 启用时)

协议(arXiv:2510.24428):从被测仓库 README + docs/ 提取 docs_tree → LLM 独立生成 3 份层级 rubrics(requirement + weight 1-3 + 递归 sub_tasks) → 第 4 次调用语义合并(名称相似度 >70% 的节点合并,权重取均值)→ 裁判对每个叶子以产物为证据做 0/1 满足判定 → 加权自底向上聚合: S(n) = Σw(c)·S(c)/Σw(c);叶子 σ 按二项近似 sqrt(p(1-p)),非叶子 σ² = Σ(w²σ²)/Σw² 传播;Coverage = 满足叶子数 / 总叶子数。 与 TQS 并存:TQS 是相对对比(新旧文档),rubric 是绝对质量(对 仓库意图的覆盖度)。mock/无 key 时 None(“失败只告警“策略)。

Fields§

§rubric_nodes: usize

合并后的 rubric 节点总数(含非叶子)

§leaf_count: usize

叶子(可判定项)数

§satisfied_leaves: usize

判定为满足(1)的叶子数

§coverage: f64

覆盖率 = satisfied / 有效判定叶子(leaf − abstain;0 时为 1.0 空集约定)——t04 起 abstain 叶子排除出分母(2606.00093 item 8: exclude 模式报 covered-subset 性能)

§score: f64

加权总分 S(0-1;×10 可与 TQS 0-10 口径对比)

§score_std: f64

加权标准差 σ_R(二项近似 + 权重传播)

§generation_calls: usize

生成轮次(独立生成 3 次 + 合并 1 次 = 4 次 LLM 调用)

§judge_model: String

裁判模型(config.llm.model)

§abstain_leaves: usize

t04:abstain 叶子数(判定调用/解析失败——不再 recode 为 false; 2606.00093 item 6:recode 改变 estimand,排除需显式报告)

§abstain_rate: f64

t04:abstain 率 = abstain / 总叶子(2606.00093 item 7: abstention/tie/invalid 率本身作为指标报告)

§leaf_verdict_repeats: usize

t04:叶子判定轮数(3 次多数投票;1:2 争议升级 5 次——2606.13685: 单次判定保真仅 86.6%,3 trials 达约 90%)

§aggregation_level: String

t04:聚合层级声明(叶子级多数投票 → 权重自底向上;2606.00093 item 10 要求声明 micro/macro/item-level)

Trait Implementations§

Source§

impl Clone for RubricReport

Source§

fn clone(&self) -> RubricReport

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for RubricReport

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Serialize for RubricReport

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more