Crate fp_io

Expand description

IO layer for frankenpandas: round-trips between DataFrame and the fifteen supported on-disk / wire formats — CSV, JSON, JSONL, Parquet, ORC, HDF5, Excel (XLSX), Feather (Arrow IPC v2), SQL, Markdown, LaTeX, HTML, XML, Pickle, and Stata.

§Format readers / writers

CSV: read_csv, read_csv_with_options, write_csv, write_csv_string
JSON / JSONL: read_json, read_jsonl, write_json, write_jsonl
Parquet: read_parquet, write_parquet
ORC: read_orc, write_orc
HDF5: read_hdf, write_hdf for the keyed DataFrame snapshot surface.
Excel: read_excel, write_excel
Feather / Arrow IPC: read_feather, write_feather, read_ipc_stream_bytes, write_ipc_stream_bytes
SQL: read_sql, read_sql_table, write_sql, write_sql_with_options, plus the chunked variants (read_sql_chunks, SqlChunkIterator).
Markdown / LaTeX / HTML / XML: write_markdown_string, write_latex_string, write_html_string, read_html_str, write_xml_string, read_xml_str.
Pickle: write_pickle_bytes, read_pickle_bytes for the fail-closed FrankenPandas DataFrame snapshot envelope.
Stata: write_stata_bytes, read_stata_bytes for the bounded DTA V118 DataFrame round-trip surface.

Each format has a per-call options struct (CsvReadOptions, ExcelReadOptions, SqlReadOptions, SqlWriteOptions, …) so pandas-shaped keyword arguments thread cleanly through the Rust API. The DataFrameIoExt extension trait adds df.to_csv(path) / df.to_parquet(path) / etc. methods on DataFrame for ergonomic method-chain use.

§SQL backend abstraction

SQL IO is built around the SqlConnection trait — a backend-neutral contract that mirrors the supported subset of pandas / SQLAlchemy.Inspector. Concrete backends (today: rusqlite via the sql-sqlite feature) implement the trait and inherit:

Mutation primitives: SqlConnection::query, SqlConnection::execute_batch, SqlConnection::insert_rows, SqlConnection::truncate_table, SqlConnection::with_transaction.
Capability probes: SqlConnection::dialect_name, SqlConnection::server_version, SqlConnection::max_param_count, SqlConnection::max_identifier_length, SqlConnection::supports_returning, SqlConnection::supports_schemas.
Identifier / parameter shape hooks: SqlConnection::quote_identifier, SqlConnection::parameter_marker.
Introspection (matching SQLAlchemy.Inspector shape): SqlConnection::list_tables, SqlConnection::list_views, SqlConnection::list_schemas, SqlConnection::table_schema, SqlConnection::list_indexes, SqlConnection::list_foreign_keys, SqlConnection::list_unique_constraints, SqlConnection::primary_key_columns, SqlConnection::table_comment.

The SqlInspector facade wraps a &C: SqlConnection and exposes the whole introspection API as methods on a single bundle:

let inspector = SqlInspector::new(&conn);
let bundle = inspector
    .reflect_table("users", None)?
    .expect("table exists");
for col in &bundle.columns {
    println!("{}: {:?}", col.name, col.declared_type);
}

SqlReflectedTable is the bundled metadata returned by SqlInspector::reflect_table / SqlInspector::reflect_all_tables / SqlInspector::reflect_all_views — columns, primary key, indexes, foreign keys, unique constraints, and table-level comment, with per-column lookup helpers.

§Cargo features

sql-sqlite (default): bind SqlConnection for rusqlite::Connection.
sql-postgresql, sql-mysql: placeholder feature flags for the fd90 Phase 2 backend integrations (no concrete bindings yet).

Use default-features = false to drop the rusqlite dep when only the non-SQL formats are needed.

Structs§

CsvReadOptions
CsvWriteOptions: Options controlling CSV serialization.
ExcelReadOptions: Options for reading Excel files.
ExcelWriteOptions: Options for serializing a DataFrame to Excel.
FwfReadOptions: Options for read_fwf_str and read_fwf.
HdfReadOptions: Options controlling HDF5 path reads.
HdfWriteOptions: Options controlling HDF5 path writes.
HtmlReadOptions: Options controlling HTML table parsing.
HtmlWriteOptions: Options controlling HTML table serialization.
LatexWriteOptions: Options controlling LaTeX table serialization.
MarkdownWriteOptions: Options controlling Markdown table serialization.
MysqlConnection: Wrapper around mysql::Conn providing interior mutability for the SqlConnection trait (which requires &self).
PickleReadOptions: Options controlling Pickle deserialization.
PickleWriteOptions: Options controlling Pickle serialization.
PostgresConnection: Wrapper around postgres::Client providing interior mutability for the SqlConnection trait (which requires &self).
SqlBackendCaps: Backend capability summary exposed through SqlInspector.
SqlChunkIterator: Iterator over DataFrame chunks produced by a SQL query.
SqlColumnSchema: Backend-neutral SQL column metadata.
SqlForeignKeySchema: Backend-neutral SQL foreign-key constraint metadata.
SqlIndexSchema: Backend-neutral SQL index metadata.
SqlIndexedChunkIterator: Iterator over SQL DataFrame chunks with optional per-chunk index promotion.
SqlInspector: Backend-agnostic introspection facade matching the SQLAlchemy.Inspector shape.
SqlQueryResult: Backend-agnostic in-memory representation of a SQL query result.
SqlReadOptions: Options for reading SQL query results into a DataFrame.
SqlReflectedTable: Bundle of all introspection metadata for a single SQL table.
SqlTableSchema: Backend-neutral SQL table metadata.
SqlUniqueConstraintSchema: Backend-neutral SQL unique-constraint metadata.
SqlWriteOptions: Options for writing a DataFrame to SQL.
StataWriteOptions: Options controlling Stata DTA serialization.
XmlReadOptions: Options controlling XML parsing.
XmlWriteOptions: Options controlling XML serialization.

Enums§

CsvOnBadLines
IoError
JsonOrient
PickleProtocol: Pickle protocol used by write_pickle_bytes_with_options.
SqlIfExists: Options for writing a DataFrame to SQL.
SqlInsertMethod: Strategy for emitting INSERT statements during write_sql.

Constants§

DEFAULT_HDF5_KEY: Default HDF5 group key used by read_hdf and write_hdf.

Traits§

DataFrameIoExt: Extension trait that adds IO convenience methods to DataFrame.
SeriesIoExt: Extension trait that adds IO convenience methods to Series.
SqlConnection: Minimal SQL connection surface needed by FrankenPandas SQL IO.

Functions§

inspect: Convenience constructor for SqlInspector.
list_sql_foreign_keys: List foreign-key constraints declared on a SQL table, optionally schema-scoped.
list_sql_indexes: List user-defined indexes on a SQL table, optionally schema-scoped.
list_sql_schemas: List user-visible schemas exposed by the SQL backend.
list_sql_tables: List user-visible table names known to the SQL backend.
list_sql_unique_constraints: List UNIQUE constraints declared on a SQL table.
list_sql_views: List user-visible view names known to the SQL backend.
read_clipboard: Reject-closed clipboard reader, matching pd.read_clipboard() shape.
read_csv
read_csv_str
read_csv_with_index_cols: Read CSV and promote the named columns into a row index / row MultiIndex.
read_csv_with_index_cols_path
read_csv_with_options
read_csv_with_options_path
read_excel: Read an Excel (.xlsx/.xls/.xlsb/.ods) file into a DataFrame.
read_excel_bytes: Read Excel from in-memory bytes.
read_excel_bytes_with_index_cols
read_excel_sheets
read_excel_sheets_bytes: Read multiple sheets from Excel bytes.
read_excel_sheets_ordered: Read multiple sheets from an Excel file.
read_excel_sheets_ordered_bytes: Byte-based counterpart to read_excel_sheets_ordered.
read_excel_with_index_cols
read_feather: Read a DataFrame from an Arrow IPC (Feather v2) file.
read_feather_bytes: Read a DataFrame from Arrow IPC (Feather v2) bytes in memory.
read_fwf: Read a fixed-width file from disk, matching pd.read_fwf(path, ...).
read_fwf_str: Parse a fixed-width string, matching pd.read_fwf(io.StringIO(s), ...).
read_gbq: Reject-closed BigQuery reader, matching pd.read_gbq(query, project_id).
read_hdf: Read a DataFrame from the default HDF5 key.
read_hdf_key: Read a DataFrame from an explicit HDF5 key.
read_hdf_with_options: Read a DataFrame from an HDF5 file with options.
read_html
read_html_str: Parse a DataFrame from the first HTML table in a document string.
read_html_str_with_options: Parse a DataFrame from an HTML document string with options.
read_html_with_options
read_ipc_stream_bytes: Read a DataFrame from Arrow IPC stream bytes (streaming format).
read_json
read_json_str
read_jsonl: Read a DataFrame from a JSONL file.
read_jsonl_str
read_orc: Read a DataFrame from an ORC file.
read_orc_bytes: Read a DataFrame from in-memory ORC bytes.
read_parquet: Read a DataFrame from a Parquet file.
read_parquet_bytes: Read a DataFrame from in-memory Parquet bytes.
read_pickle: Read a DataFrame from a Pickle file.
read_pickle_bytes: Deserialize a DataFrame from Pickle bytes.
read_pickle_bytes_with_options: Deserialize a DataFrame from Pickle bytes with options.
read_pickle_with_options: Read a DataFrame from a Pickle file with options.
read_sas: Reject-closed SAS reader, matching pd.read_sas(path).
read_spss: Reject-closed SPSS reader, matching pd.read_spss(path).
read_sql: Read the result of a SQL query into a DataFrame.
read_sql_chunks: Read a SQL query result as an iterator of DataFrame chunks.
read_sql_chunks_with_index_col: Read a SQL query result as DataFrame chunks with optional index promotion.
read_sql_chunks_with_options: Read a SQL query result as DataFrame chunks with read-time options.
read_sql_chunks_with_options_and_index_col: Read a SQL query result as DataFrame chunks with read-time options and optional index promotion.
read_sql_query: Read the result of a SQL query into a DataFrame.
read_sql_query_chunks: Read the result of a SQL query as an iterator of DataFrame chunks.
read_sql_query_chunks_with_index_col: Read a SQL query result as chunks with one column promoted to each chunk’s index.
read_sql_query_chunks_with_options: Read the result of a SQL query as chunks with read-time options.
read_sql_query_chunks_with_options_and_index_col: Read a SQL query result as chunks with read-time options and index promotion.
read_sql_query_with_index_col: Read a SQL query result with one column promoted to the index.
read_sql_query_with_options: Read the result of a SQL query into a DataFrame with read-time options.
read_sql_query_with_options_and_index_col: Read a SQL query result with read-time options and optional index promotion.
read_sql_table: Read an entire SQL table into a DataFrame.
read_sql_table_chunks: Read an entire SQL table as an iterator of DataFrame chunks.
read_sql_table_chunks_with_index_col: Read an entire SQL table as chunks with one column promoted to each chunk’s index.
read_sql_table_chunks_with_options: Read an entire SQL table as DataFrame chunks with read-time options.
read_sql_table_chunks_with_options_and_index_col: Read an entire SQL table as chunks with read-time options and optional index promotion.
read_sql_table_columns: Read a subset of columns from a SQL table.
read_sql_table_columns_chunks: Read a subset of columns from a SQL table as DataFrame chunks.
read_sql_table_columns_chunks_with_index_col: Read a subset of columns from a SQL table as chunks with optional index promotion.
read_sql_table_columns_with_index_col: Read a subset of columns from a SQL table with optional index promotion.
read_sql_table_with_index_col: Read an entire SQL table with one column promoted to the index.
read_sql_table_with_options: Read an entire SQL table into a DataFrame with read-time options.
read_sql_table_with_options_and_index_col: Read an entire SQL table with read-time options and optional index promotion.
read_sql_with_index_col: Read a SQL query result with one column promoted to the index.
read_sql_with_options: Read the result of a SQL query into a DataFrame with read-time options.
read_stata: Read a DataFrame from a Stata DTA file.
read_stata_bytes: Read a DataFrame from Stata DTA bytes.
read_table: Read a tab-separated file from disk, matching pd.read_table(path).
read_table_str: Parse a tab-separated string, matching pd.read_table(io.StringIO(s)).
read_table_with_options: Parse a tab-separated string with explicit options. The caller-supplied options.delimiter is preserved when it differs from the comma default to allow override; otherwise it is forced to b'\t' so that the pandas read_table semantics survive CsvReadOptions::default().
read_table_with_options_path: Read a tab-separated file from disk with explicit options. The caller-supplied delimiter is honored when it has been overridden from the comma default; otherwise it is forced to b'\t'.
read_xml
read_xml_str: Parse a DataFrame from a row-oriented XML document string.
read_xml_str_with_options: Parse a DataFrame from a row-oriented XML document string with options.
read_xml_with_options
series_from_arrow_array: Rebuild a Series from an Arrow array and explicit dtype metadata.
series_to_arrow_array: Convert a Series to its Arrow data type plus backing array.
sql_backend_caps: Gather the backend capability probes into one typed bundle.
sql_max_identifier_length: Maximum identifier length supported by the SQL backend, or None when no documented limit exists.
sql_max_insert_rows: Maximum INSERT rows for column_count, derived from the backend’s bound-parameter ceiling.
sql_max_param_count: Maximum bound parameters supported by the SQL backend, if known.
sql_primary_key_columns: Return the primary-key column names for a SQL table, ordered by the table’s primary-key ordinal.
sql_server_version: Probe the SQL backend’s server version string.
sql_supports_returning: Whether the SQL backend supports native INSERT ... RETURNING.
sql_supports_schemas: Whether the SQL backend exposes schema-qualified namespaces.
sql_table_comment: Probe the table-level comment for a SQL table, optionally schema-scoped.
sql_table_schema: Introspect a SQL table’s column metadata, optionally schema-scoped.
truncate_sql_table: Reset a SQL table to empty without dropping its definition.
write_csv
write_csv_string
write_csv_string_with_options: Serialize a DataFrame to CSV with explicit options.
write_excel: Write a DataFrame to an Excel (.xlsx) file.
write_excel_bytes: Write a DataFrame to Excel (.xlsx) bytes in memory.
write_excel_bytes_with_options: Serialize a DataFrame to Excel bytes with explicit options.
write_excel_with_options: File-based counterpart to write_excel_bytes_with_options.
write_feather: Write a DataFrame to an Arrow IPC (Feather v2) file.
write_feather_bytes: Write a DataFrame to Arrow IPC (Feather v2) bytes in memory.
write_hdf: Write a DataFrame to the default HDF5 key.
write_hdf_key: Write a DataFrame to an explicit HDF5 key.
write_hdf_series: Write a Series to the default HDF5 key.
write_hdf_series_key: Write a Series to an explicit HDF5 key.
write_hdf_with_options: Write a DataFrame to an HDF5 file with options.
write_html
write_html_string
write_html_string_with_options: Serialize a DataFrame to an HTML table string.
write_html_with_options
write_ipc_stream_bytes: Write a DataFrame to Arrow IPC stream bytes (streaming format, no random access).
write_json
write_json_string
write_jsonl: Write a DataFrame to a JSONL file.
write_jsonl_string: Write a DataFrame to JSONL (JSON Lines) format.
write_latex: Write a DataFrame to a LaTeX tabular file.
write_latex_string
write_latex_string_with_options: Serialize a DataFrame to a booktabs-compatible LaTeX tabular block.
write_latex_with_options: Write a DataFrame to a LaTeX tabular file with explicit options.
write_markdown: Write a DataFrame to a Markdown table file.
write_markdown_string
write_markdown_string_with_options: Serialize a DataFrame to a GitHub-style Markdown table.
write_markdown_with_options: Write a DataFrame to a Markdown table file with explicit options.
write_orc: Write a DataFrame to an ORC file.
write_orc_bytes: Write a DataFrame to an in-memory ORC buffer.
write_parquet: Write a DataFrame to a Parquet file.
write_parquet_bytes: Write a DataFrame to an in-memory Parquet buffer.
write_pickle: Write a DataFrame to a Pickle file.
write_pickle_bytes
write_pickle_bytes_with_options: Serialize a DataFrame to Pickle bytes.
write_pickle_with_options: Write a DataFrame to a Pickle file with options.
write_sql: Write a DataFrame to a SQL table.
write_sql_with_options: Write a DataFrame to a SQLite table with pandas-style index options.
write_stata: Write a DataFrame to a Stata DTA file.
write_stata_bytes
write_stata_bytes_with_options: Serialize a DataFrame to Stata DTA bytes.
write_stata_with_options: Write a DataFrame to a Stata DTA file with explicit options.
write_xml
write_xml_string
write_xml_string_with_options: Serialize a DataFrame to an XML document string.
write_xml_with_options

Crate fp_io

Crate fp_io Copy item path

§Format readers / writers

§SQL backend abstraction

§Cargo features

Structs§

Enums§

Constants§

Traits§

Functions§

Crate fp_io