Module distill 

Local-LLM distillation trait and the optional distill-http implementation. Distillation: turning a raw conversation transcript into durable memory.

This module defines the Distiller trait and the data types it produces. The trait itself is always available (not feature-gated) so that callers can implement their own distillers — for example, a remote API client, a different local model runtime, or a test stub.

The OpenAiCompatDistiller implementation in the openai_compat submodule (behind the distill-http feature) talks to an OpenAI-compatible chat completions endpoint such as Ollama or llama-server. It is the only place in this crate that performs HTTP, and only when that feature is enabled.

Structs

ChunkingDistiller

A Distiller decorator that bounds the size of any single prompt sent to inner.

DistilledMemory

The result of distilling a conversation transcript into durable memory.

Fact

A single distilled fact extracted from a transcript.

Enums

FactKind

The category of a distilled Fact.

ReduceStrategy

Strategy used to reduce several partial DistilledMemory results — one per transcript chunk — into a single result.

Constants

MAX_FACTS

Maximum number of facts persisted from a single distillation. Excess facts (untrusted model output) are dropped.

MAX_FACT_CHARS

Maximum character length of a single distilled fact’s text. Longer text is truncated on a char boundary.

MAX_SUMMARY_CHARS

Maximum character length of a distilled summary. Longer text is truncated on a char boundary.

Traits

Distiller

Produces DistilledMemory from a raw conversation transcript.

Functions

merge_distilled

Reduces several DistilledMemory partial results — one per transcript chunk — into a single DistilledMemory.

split_on_budget

Splits transcript into chunks that each fit within max_chars.