Crate newt_data 

newt-data — the Phase 21 Centaur data-science engine

A headless Rust library: the SQLite data engine behind the Centaur Data Scientist capability. No MCP, no JSON-RPC, no PyO3, no Jupyter kernel — those arrive in later steps (21.2 MCP server, 21.3+ kernel, 21.6 PyO3 submodule). This crate is only the engine where ~all the coverage is earned: pure functions over an in-memory or on-disk SQLite database.

The shape

• DataStore is the trait the whole capability is built on — the DuckDB-later seam (step 21.8 swaps the backend without touching a signature). It exposes CSV ingest, ad-hoc SQL query, schema/statistics summary, and table listing.
• SqliteBackend is the first implementation: bundled rusqlite 0.31 (MSRV 1.75), WAL-with-DELETE-fallback like the conversation store, and a separate database at <workspace>/.newt-data/data.db.

Design contracts honored here (Phase 21)

• Identifier safety — every table/column name reaches SQL only through ingest::quote_ident; table names are additionally validated. A CSV header like weird "name round-trips as a literal column name; it cannot inject SQL.
• Honest query results — QueryResult::truncated is set by reading one row past the cap, never by fabricating a total.
• pandas-faithful statistics — NumericDescribe reproduces .describe() exactly: sample std (ddof = 1), linear-interpolation quartiles.

Example

use std::path::Path;
use newt_data::{DataStore, SqliteBackend};

let store = SqliteBackend::open_in_memory()?;
let report = store.ingest_csv(Path::new("sales.csv"), "sales")?;
println!("loaded {} rows into {}", report.row_count, report.table);

let result = store.query("SELECT * FROM sales", 100)?;
if result.truncated {
    println!("(showing first {} rows; more exist)", result.returned);
}

let summary = store.summarize("sales")?;
for col in &summary.columns {
    println!("{}: {} ({} nulls)", col.name, col.dtype, col.null_count);
}

Re-exports

pub use error::DataError;

pub use error::Result;

pub use store::sqlite::SqliteBackend;

pub use store::ColumnDtype;

pub use store::ColumnInfo;

pub use store::ColumnSummary;

pub use store::DataStore;

pub use store::IngestReport;

pub use store::NumericDescribe;

pub use store::QueryResult;

pub use store::TableInfo;

pub use store::TableSummary;

Modules

error

Error type for the Phase 21 newt-data engine.

ingest

CSV → SQLite-table ingest (Phase 21).

notebook

nbformat .ipynb notebook persistence (Phase 21.4): pure serde_json read/insert/persist over the human’s reviewable, git-diffable notebook artifact. A normal module (no kernel/HTTP deps) — it manipulates JSON, it does not run cells. run_cell(persist_to=…) calls notebook::persist_cell to append each executed cell. See docs/design/centaur-data-scientist.md §4.1, the notebook-artifact bullet. nbformat .ipynb notebook persistence (Phase 21.4).

store

The DataStore trait and its result types (Phase 21).

summary

describe / summarize logic (Phase 21).