Crate devboy_format_pipeline 

Format pipeline for tool output transformation.

Formats tool responses into an optimal format for LLM:

• TOON (default): Token-Oriented Object Notation – saves 39-90% of tokens
• JSON: for programmatic processing
• Budget trimming: smart strategy-based trimming when output exceeds budget

Example

use devboy_format_pipeline::{Pipeline, PipelineConfig, OutputFormat};
use devboy_core::Issue;

let pipeline = Pipeline::with_config(PipelineConfig {
    format: OutputFormat::Toon,
    max_chars: 100_000,
    ..Default::default()
});

let output = pipeline.transform_issues(issues)?;
println!("{}", output.to_string_with_hints());

Re-exports

pub use token_counter::Tokenizer;

pub use token_counter::estimate_tokens;

pub use token_counter::tokens_to_chars;

pub use truncation::TruncationPlugin;

Modules

adaptive_config

Adaptive configuration — TOML-backed tuning knobs for the layered pipeline.

budget

Iterative budget pipeline: encode → check → trim → re-encode → verify.

dedup

Hint-based deduplication cache for tool responses in multi-turn agents.

enrichment

Paper 3 — Enrichment Planner.

layered_pipeline

LayeredPipeline — generic tool-response compressor.

mckp_router

L2 generic MCKP router.

near_ref

Type-2 (near-duplicate) reference hints — Paper 2 §Near-reference.

page_index

Page index generation for large results.

pagination

Cursor-based pagination for budget-trimmed responses.

projection

Paper 3 — argument projection for speculative pre-fetch.

round_trip

Round-trip correctness gate for L1 / L2 encoders.

shape

Structural shape classifier for tool responses.

strategy

Trimming strategies assign information value to tree nodes.

telemetry

Pipeline telemetry — per-response event capture for adaptive tuning.

templates

L1 per-endpoint templates.

token_counter

Token counting for budget pipeline and savings telemetry.

tool_defaults

Paper 3 — built-in default ToolValueModels for selected common tools by corpus volume.

toon

TOON (Token-Oriented Object Notation) encoding for tool output.

tree

Trim tree (TrimTree) for budget trimming.

trim

Tree trimming algorithms by budget.

truncation

Truncation utilities for limiting output size.

Structs

Pipeline

Pipeline for chaining output transformations.

PipelineConfig

Configuration for pipeline transformations.

TransformOutput

Output from a pipeline transformation.

Enums

OutputFormat

Output format for transformations.