brainwires-datasets 0.8.0

Training data pipelines for the Brainwires Agent Framework — JSONL I/O, tokenization, deduplication, format conversion
Documentation
1
2
3
4
5
6
7
/// Streaming JSONL reader.
pub mod reader;
/// Buffered JSONL writer.
pub mod writer;

pub use reader::{JsonlReader, read_jsonl, read_jsonl_preferences};
pub use writer::{JsonlWriter, write_jsonl, write_jsonl_preferences};