from_csv_reader_with_config

Function from_csv_reader_with_config 

Source
pub fn from_csv_reader_with_config<R: Read>(
    reader: R,
    type_name: &str,
    schema: &[&str],
    config: FromCsvConfig,
) -> Result<Document>
Expand description

Parse CSV from a reader into a HEDL document with custom configuration.

This is the most flexible CSV parsing function, supporting both custom I/O sources and custom parsing configuration.

§Arguments

  • reader - Any type implementing Read
  • type_name - The HEDL type name for rows
  • schema - Column names excluding the ‘id’ column
  • config - Configuration controlling all parsing behavior

§Examples

§Large File with Custom Limit

use hedl_csv::{from_csv_reader_with_config, FromCsvConfig};
use std::fs::File;

let file = File::open("large_dataset.csv").unwrap();
let config = FromCsvConfig {
    max_rows: 50_000_000, // 50M rows for high-memory server
    ..Default::default()
};
let doc = from_csv_reader_with_config(file, "Record", &["value"], config).unwrap();

§TSV from Network Stream

use hedl_csv::{from_csv_reader_with_config, FromCsvConfig};
use std::net::TcpStream;

let stream = TcpStream::connect("example.com:8080").unwrap();
let config = FromCsvConfig {
    delimiter: b'\t',
    ..Default::default()
};
let doc = from_csv_reader_with_config(stream, "Data", &["col1", "col2"], config).unwrap();

§Implementation Details

The function performs the following steps:

  1. Creates a CSV reader with the specified configuration
  2. Initializes a new HEDL document with version (1, 0)
  3. Constructs the full schema (ID column + provided columns)
  4. Registers the struct type in the document
  5. Iterates through CSV records:
    • Checks row count against max_rows security limit
    • Parses each field using type inference
    • Validates field count matches schema
    • Creates Node instances and adds to matrix list
  6. Inserts the completed matrix list into the document

§See Also

  • from_csv_with_config - For parsing CSV strings
  • FromCsvConfig - Configuration options documentation