ucci 0.1.0

UCCI: calibrated uncertainty for cost-optimal LLM cascade routing (token-margin signal, isotonic calibration, constrained threshold selection)
Documentation
# ucci (Rust)

Rust implementation of **UCCI**, calibrated uncertainty for cost-optimal LLM
cascade routing, from Varun Kotte,
["UCCI: Calibrated Uncertainty for Cost-Optimal LLM Cascade Routing"](https://arxiv.org/abs/2605.18796),
arXiv:2605.18796 (2026).

A two-model cascade answers each query with a small model and escalates to a
large model only when the small model is likely wrong. UCCI makes that
decision in three steps (paper Section 4):

1. **Signal** (Section 4.1, Eq. 4): `u(x) = 1 - (1/T) sum_t (p_{t,1} - p_{t,2})`,
   the mean top-1 minus top-2 token-probability margin of the small model's
   greedy generation, subtracted from 1.
2. **Calibration** (Section 4.2): an isotonic map `g` from `u(x)` to the
   probability that the small model is wrong, `p_hat(x) = g(u(x))`.
3. **Threshold** (Section 4.3, Eqs. 6 and 7): escalate when `p_hat(x) > theta`,
   with `theta` the cheapest threshold whose validation accuracy reaches a
   target `tau`. Under the assumptions of Theorem 1 (Section 5) this
   threshold policy is cost-optimal among policies that depend only on `u(x)`.

This crate is a port of the Python reference package in the same repository
(`src/ucci`). It computes the same numbers: every function is tested against
golden vectors generated by the Python package, and router files move freely
between the two.

## Install

```toml
[dependencies]
ucci = { git = "https://github.com/varunkotte6/ucci" }
```

The default feature `json` adds router files and serde support (dependencies:
`serde`, `serde_json`). With `default-features = false` the crate has no
dependencies.

Minimum supported Rust version: 1.70. The crate itself needs nothing newer;
recent `serde_derive` releases need Rust 1.71, so on 1.70 pin serde with
`cargo update -p serde --precise 1.0.228`.

## Route with a router fitted in Python

Fit and save the router with the Python package:

```python
from ucci import UCCIRouter

router = UCCIRouter().calibrate(u_cal, e_cal)          # Section 4.2
router.choose_threshold(u_val, small_val, large_val, tau=0.91)   # Eq. 7
router.save("router.json")
```

Load it in Rust and route each query from the small model's top-2
log-probabilities:

```rust,no_run
use ucci::signal::uncertainty_from_top_logprobs;
use ucci::Router;

fn main() -> Result<(), ucci::UcciError> {
    let router = Router::load("router.json")?;

    // Candidate log-probabilities per generated content token, as returned by
    // an OpenAI-compatible server with top_logprobs = 2 (any order).
    let top_logprobs = vec![vec![-0.01, -4.7], vec![-0.9, -0.6], vec![-0.05, -3.2]];
    let u = uncertainty_from_top_logprobs(&top_logprobs)?;

    let route = router.route(u)?;
    println!("p_hat = {:.3}, escalate = {}", route.p_hat, route.escalate);
    Ok(())
}
```

`cargo run --example route` loads `examples/router.json`, a router saved by
the Python package, and routes a few values of `u(x)`; pass a path and values
to route your own (`cargo run --example route -- router.json 0.1 0.4`).

With the `json` feature, `ucci::signal::from_openai_logprobs` takes the
`choice.logprobs.content` array of a chat completion directly, and
`ucci::signal::MarginAccumulator` computes `u(x)` token by token while a
response streams in.

## Fit a router in Rust

The evaluation protocol of Section 6.1 (fit `g` on the calibration split,
choose `theta` on the validation split, route the test split end to end with
actual outputs and costs):

```rust
use ucci::{Costs, RouterBuilder};

fn main() -> Result<(), ucci::UcciError> {
    // Calibration split: u(x) and whether the small model was wrong.
    let u_cal = [0.05, 0.10, 0.20, 0.30, 0.40, 0.55, 0.70, 0.90];
    let e_cal = [0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0];
    // Validation split: both models were run; per-query scores of their outputs.
    let u_val = [0.05, 0.25, 0.35, 0.60, 0.80, 0.95];
    let small_val = [1.0, 1.0, 0.0, 0.0, 1.0, 0.0];
    let large_val = [1.0, 1.0, 1.0, 1.0, 1.0, 0.0];
    // Test split.
    let u_test = [0.02, 0.33, 0.51, 0.88];
    let small_test = [1.0, 1.0, 0.0, 0.0];
    let large_test = [1.0, 1.0, 1.0, 1.0];

    let router = RouterBuilder::new(Costs::paper())? // c_small = 1.0, c_large = 3.02
        .calibrate(&u_cal, &e_cal)?
        .choose_threshold(&u_val, &small_val, &large_val, 0.8)?;

    let test = router.evaluate(&u_test, &small_test, &large_test)?;
    println!(
        "theta = {}, test cost = {:.3}, test accuracy = {:.3}",
        router.theta(),
        test.cost,
        test.accuracy
    );
    Ok(())
}
```

`Costs::paper()` uses the paper's normalized costs, `c_small = 1.0` and
`c_large = 3.02` (the measured latency ratio, Section 6.1), under the routing
cost model (an escalated query costs `c_large`). `CostModel::Sequential`
charges `c_small + c_large` instead. `choose_threshold_for_budget` gives the
matched-budget form used for the bottom block of Table 2, and
`policy::pareto_frontier` returns the cost and accuracy of every grid
threshold. For a corpus-level metric such as micro-F1 over entities, pass
`metrics::RoutedMicroF1::metric()` to `policy::select_threshold_with_metric`.

## Modules

| Module | Paper | Contents |
|---|---|---|
| `signal` | Section 4.1 | `u(x)` from probability pairs, log-probabilities, OpenAI-style `top_logprobs`, streaming |
| `calibration` | Section 4.2 | weighted pool-adjacent-violators, `IsotonicCalibrator` (scikit-learn semantics, clipped ends) |
| `policy` | Section 4.3, Theorem 1, Tables 2 and 3 | `escalate`, `policy_cost`, `select_threshold`, budget form, Pareto frontier, `evaluate` |
| `metrics` | Section 6 | ECE (uniform or quantile bins), reliability table, Brier score, micro-F1 |
| `router` | Sections 4 and 6.1 | `Router`, `RouterBuilder`, the `ucci-router` JSON format |

Every fallible function returns `Result<_, UcciError>`; inputs are validated
exactly like in the Python package (NaN and infinities are rejected,
probabilities must lie in `[0, 1]`, costs must be positive).

## Parity with Python

Each computation follows the Python package operation by operation, down to
numpy's summation order, `numpy.interp` and scikit-learn's tie pooling, and
router files are parsed with exact float round-tripping. The golden test
replays every vector in `../tests/golden` (written by
`tools/make_golden.py`), requires agreement to `1e-12` and reports how many
values match bit for bit; with the current vectors all of them do:

```sh
cargo test --test golden -- --nocapture
```

Routers written by this crate are read back by the Python test suite
(`tests/test_golden.py`). After an intended change of the writer, refresh
that file with `UCCI_BLESS=1 cargo test --test golden`.

Bootstrap confidence intervals, the baselines of Section 6.1, the command-line
tool and the benchmark scripts live in the Python package.

## Citation

```bibtex
@article{kotte2026ucci,
  title   = {{UCCI}: Calibrated Uncertainty for Cost-Optimal {LLM} Cascade Routing},
  author  = {Kotte, Varun},
  journal = {arXiv preprint arXiv:2605.18796},
  year    = {2026}
}
```

## License

MIT, like the Python package. See `LICENSE`.