aprender 0.31.2

Next-generation ML framework in pure Rust — `cargo install aprender` for the `apr` CLI
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<!-- PCU: examples-tokenizer-surgery | contract: contracts/apr-page-examples-tokenizer-surgery-v1.yaml -->
<!-- Example: cargo run -p aprender-core --example tokenizer_surgery -->
<!-- Status: enforced -->

# Case Study: Tokenizer Surgery

**Ticket**: GH-447
**Module**: `aprender::online::tokenizer_surgery`

## Overview

Transplants embedding rows when adapting a pre-trained model to a new tokenizer. Supports direct copy, nearest-neighbor, and average-pool strategies for unmatched tokens.

## Key Components

- **`TokenizerSurgeryConfig`** — Vocab sizes, overlap threshold, surgery method
- **`SurgeryMethod`** — DirectCopy, NearestNeighbor, AveragePool
- **`VocabMapping`** — Bidirectional source/target index mapping
- **`compute_vocab_overlap`** — O(n+m) vocabulary intersection
- **`transplant_embeddings`** — Row-by-row embedding transfer
- **`validate_surgery`** — Quality gate on overlap ratio

## Run

```bash
cargo run --example tokenizer_surgery
```

## Falsification Tests

| ID | Property | Status |
|----|----------|--------|
| FALSIFY-SURGERY-001 | Overlap ratio in [0, 1] | Falsified (holds) |
| FALSIFY-SURGERY-002 | Transplant preserves dimensions | Falsified (holds) |
| FALSIFY-SURGERY-003 | Identical vocabs yield identity | Falsified (holds) |