newt-data — the Phase 21 Centaur data-science engine

A headless Rust library: the SQLite data engine behind the Centaur Data Scientist capability. No MCP, no JSON-RPC, no PyO3, no Jupyter kernel — those arrive in later steps (21.2 MCP server, 21.3+ kernel, 21.6 PyO3 submodule). This crate is only the engine where ~all the coverage is earned: pure functions over an in-memory or on-disk SQLite database.

The shape

• [DataStore] is the trait the whole capability is built on — the DuckDB-later seam (step 21.8 swaps the backend without touching a signature). It exposes CSV ingest, ad-hoc SQL query, schema/statistics summary, and table listing.
• [SqliteBackend] is the first implementation: bundled rusqlite 0.31 (MSRV 1.75), WAL-with-DELETE-fallback like the conversation store, and a separate database at <workspace>/.newt-data/data.db.

Design contracts honored here (Phase 21)

• Identifier safety — every table/column name reaches SQL only through [ingest::quote_ident]; table names are additionally validated. A CSV header like weird "name round-trips as a literal column name; it cannot inject SQL.
• Honest query results — [QueryResult::truncated] is set by reading one row past the cap, never by fabricating a total.
• pandas-faithful statistics — [NumericDescribe] reproduces .describe() exactly: sample std (ddof = 1), linear-interpolation quartiles.

Example

use std::path::Path;
use newt_data::{DataStore, SqliteBackend};

# fn main() -> anyhow::Result<()> {
let store = SqliteBackend::open_in_memory()?;
let report = store.ingest_csv(Path::new("sales.csv"), "sales")?;
println!("loaded {} rows into {}", report.row_count, report.table);

let result = store.query("SELECT * FROM sales", 100)?;
if result.truncated {
    println!("(showing first {} rows; more exist)", result.returned);
}

let summary = store.summarize("sales")?;
for col in &summary.columns {
    println!("{}: {} ({} nulls)", col.name, col.dtype, col.null_count);
}
# Ok(())
# }