Skip to main content

Module sampler_order

Module sampler_order 

Source
Expand description

The ORDER the sampler chain runs in, as llama.cpp’s --samplers spells it, and the refusal for every sampler ferrox does not have.

§Why the order is a parameter and not a constant

llama.cpp lets a caller reorder its chain (--samplers, --sampler-seq, and the server’s samplers request field). ferrox ran one fixed order, so a command that worked upstream either could not be expressed here or – worse – was accepted with the field dropped and answered under a different chain.

The order is not cosmetic. sampler_chain models the SHRINKING candidate list llama.cpp passes down the chain, and each filter renormalises over the survivors, so moving one step changes which candidates exist for the next. ferrox has already shipped the bug: temperature used to run FIRST, and top-p then summed probabilities temperature had already reshaped, keeping a different candidate set for identical flags.

§The one table

A list of names in the CLI and a second list in the server is this repo’s dominant defect shape: two structures that must agree with nothing enforcing it. So there is exactly one list, in the sampler_names! invocation below, and the enum, the canonical spellings, the alias table and the parser are all generated from it. Adding a name is one row.

SamplerName::implemented is the second half of the guarantee: an EXHAUSTIVE match, no .., that must say for every name whether ferrox runs it or why it does not. A name added to the table without a verdict does not compile.

§Partial support, stated

ferrox implements five of llama.cpp’s samplers. The rest are named in the table purely so that asking for one is a refusal that says WHICH sampler is missing, rather than an unknown-name error or, far worse, a chain quietly built without it. A caller who asked for xtc and was served a chain with no XTC in it got a different sampler than they requested and no way to tell.

Structs§

SamplerOrder
A validated sampler chain: the steps ferrox will run, in the order it will run them.

Enums§

ChainStep
A step the ferrox chain can actually run.
SamplerName
Every sampler name llama.cpp’s --samplers accepts.
SamplerOrderError
Why a caller-supplied sampler order was refused.