Skip to main content

TqsReport

Struct TqsReport 

Source
pub struct TqsReport {
Show 25 fields pub judged_modules: usize, pub avg_clarity: f64, pub avg_readability: f64, pub avg_conciseness: f64, pub avg_richness: f64, pub avg_structure: f64, pub avg_total: f64, pub repeats: usize, pub kappa_like: f64, pub kappa: f64, pub kappa_cohen: f64, pub flip_rate: f64, pub position_flip_rate: f64, pub delta_kappa: f64, pub eligible_modules: usize, pub parse_success_rate: f64, pub judgment_scale: String, pub aggregation_level: String, pub tie_handling: String, pub position_bias: f64, pub low_confidence_modules: Vec<String>, pub avg_std: f64, pub judge_model: String, pub tie_rate: f64, pub agreement_breakdown: [usize; 3],
}
Expand description

RepoDocBench 协议:对同一模块的旧文档(导出快照)与当前产物, 裁判按五维 0-10 打分(Clarity/Readability/Conciseness/Richness/ Structure),交换文档顺序两轮取平均消除位置偏差(position bias)。 裁判模型与温度由 config.llm 决定(默认配置 mock/未配置 key 时 本维度被跳过,report.tqs = None)。

Fields§

§judged_modules: usize

完成打分的模块数(旧文档与当前产物都存在的模块)

§avg_clarity: f64

五维平均分(0-10,顺序消偏 + 复测平均后取平均)

§avg_readability: f64§avg_conciseness: f64§avg_richness: f64§avg_structure: f64§avg_total: f64

五维总分平均(0-10)

§repeats: usize

t05/MVVP:每模块的复测次数(AB/BA 各 repeats 轮)

§kappa_like: f64

t05/MVVP:复测一致性(κ 近似)——同一模块任意两轮、同一维度 分数绝对差 ≤1 的比例。1.0 = 完全稳定;低值 = 裁判不稳定或 文档差异导致敏感(2606.19544 指出高 test-retest 与低位置偏差 可并存,一致性是可靠性下限)。

§kappa: f64

v14 C 组(MVVP 缺口):机会校正 κ——把“同一维度分数差 ≤1“ 视为二分类判定(一致/不一致),一致率经机会一致性 p²+(1-p)² 校正: κ = (P_o − P_e)/(1 − P_e),负值截断为 0。注意:机会基线用 p_obs 自身近似,非标准 Cohen’s κ(标准 κ 需两个独立 rater 的判定与 边际表,见 kappa_cohen);保留字段名与语义兼容既有消费方。

§kappa_cohen: f64

t04:标准 Cohen’s κ(2606.19544 通缩诊断的对照口径)——rater1 = AB 顺序调用、rater2 = BA 顺序调用,item = (模块, 维度, 轮), 类别 = {A 胜, B 胜}(平局按 B 胜计入,连续分数相等概率近零)。 与 kappa_like/kappa 的自定义稳定率口径不同,独立报告。

§flip_rate: f64

t04:判定翻转率(模块级平均)——单次调用的 A 胜/平/B 胜判定与 模块内多数判定不一致的比例(2606.13685 单次 flip rate 13.6%; 2606.19544 self-consistency 的互补面)。

§position_flip_rate: f64

t04:位置翻转率(模块级平均)——同一轮内 AB 顺序与 BA 顺序的 判定相异比例(2606.19544 item 级定义:交换位置后判定翻转的比例; 区别于 position_bias 的 AB/BA 组均值口径)。

§delta_kappa: f64

t04:κ 通缩诊断 = kappa_like − kappa——原始一致率被机会校正 削掉多少(2606.19544:Δκ 33-41pp 量级;本地无 human 参考时以 自稳定性口径替代)。

§eligible_modules: usize

t04:有效模块数 = 新旧文档都存在的模块总数(2606.00093 item 8 coverage;judged_modules 仅计判定成功数,failed 模块不计入)

§parse_success_rate: f64

t04:解析成功率 = 成功解析的裁判调用 / 全部调用(含失败模块; 2606.00093 item 7 要求 abstention/invalid 率单独作为指标报告)

§judgment_scale: String

t04:判定尺度声明(2606.00093 item 1:必须声明判定尺度)

§aggregation_level: String

t04:聚合层级声明(2606.00093 item 10)

§tie_handling: String

t04:tie/abstain 处理声明(2606.00093 item 6:exclude/recode/retain 是三种不同 estimand,必须显式声明)

§position_bias: f64

v14 C 组(MVVP 缺口):位置偏差 |P(A 胜) − 0.5|——对每模块每维度, AB 顺序(A 先)轮与 BA 顺序(B 先)轮的分数均值比较出 A 胜/负判定, P(A 胜) = A 胜判定数 / 判定总数。接近 0 = 文档顺序不影响裁判; 接近 0.5 = 裁判对位置敏感(分数结论可能被顺序污染)。

§low_confidence_modules: Vec<String>

v14 C 组(MVVP 缺口):低置信模块清单——复测失败(裁判调用/解析 失败被跳过,不再静默)或复测标准差超过阈值的模块(分数波动大, 结论不可信;2606.19544 要求显式报告而非跳过)

§avg_std: f64

五维分数的平均标准差(跨复测轮次;量化波动幅度)

§judge_model: String

裁判模型(config.llm.model;style 消偏在多裁判轮转下才完整, 单裁判时报告模型便于人工判断偏差来源——2604.23178)

§tie_rate: f64

v32(6.1 FR-102/FR-103):三态判定中平局占比(模块级平均)—— 全部调用级判定(judgment()==0)中 tie 的比例。tie 率升高 = 裁判区分度不足的信号(旧文档与产物五维总分经常相等),

TQS_TIE_ESCALATION_THRESHOLD 时升级复测轮数。

§agreement_breakdown: [usize; 3]

v32(6.1 FR-102):三态判定明细 [A 胜, B 胜, 平局]——全部调用级 判定(judgment() 的 1/-1/0)的累计计数。tie 是独立类别而非静默 归入胜负(2606.00093 item 6 estimand 声明);报告暴露三态结构 供人工判断裁判区分度。

Trait Implementations§

Source§

impl Clone for TqsReport

Source§

fn clone(&self) -> TqsReport

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for TqsReport

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Serialize for TqsReport

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more