Trait StreamingRowDecoder

Source

pub trait StreamingRowDecoder {
    // Required methods
    fn feed(&mut self, chunk: &[u8]) -> Result<(), Error>;
    fn drain(&mut self) -> Result<Vec<Vec<(String, LoraValue)>>, Error>;
    fn finish(&mut self) -> Result<Vec<Vec<(String, LoraValue)>>, Error>;
    fn header(&self) -> Option<&[String]>;
    fn bytes_fed(&self) -> u64;
    fn rows_emitted(&self) -> u64;

    // Provided methods
    fn set_permissive(&mut self, _on: bool) { ... }
    fn take_errors(&mut self) -> Vec<RowParseError> { ... }
}

Expand description

Push-based streaming row decoder. The caller feeds bytes one chunk at a time (e.g. from File.stream().getReader() in the browser); the decoder accumulates partial records internally and emits completed rows via Self::drain. Designed for the WASM streaming-import path where the engine and the source live on different sides of the worker boundary.

Memory bound: the decoder retains at most one in-progress record plus the bytes between the most-recently-completed record and the end of the most-recently-fed chunk.

Required Methods§

Source

fn feed(&mut self, chunk: &[u8]) -> Result<(), Error>

Append a chunk of bytes to the internal buffer and parse any records that became complete. Idempotent: feeding zero bytes is a no-op.

Source

fn drain(&mut self) -> Result<Vec<Vec<(String, LoraValue)>>, Error>

Pull all records completed since the previous drain / finish call. Returns an empty Vec when no full record has been parsed yet.

Source

fn finish(&mut self) -> Result<Vec<Vec<(String, LoraValue)>>, Error>

Signal that no more bytes will be fed. Returns any records emitted by handling the residual buffer — for JSONL/CSV this covers the case where the file ends without a trailing newline.

Column names declared in the file header. Populated after the first record arrives for CSV; always None for JSONL.

Source

fn bytes_fed(&self) -> u64

Total bytes accepted via feed since construction. Used for progress reporting.

Source

fn rows_emitted(&self) -> u64

Total records emitted via drain + finish so far.

Provided Methods§

Source

fn set_permissive(&mut self, _on: bool)

Switch the decoder into permissive mode. In permissive mode, per-record parse failures are accumulated for retrieval via Self::take_errors instead of bubbling out of feed / finish. Fatal errors (encoding issues that desync the byte stream itself) still bubble. Must be called before the first feed; later calls take effect at the next record boundary. Default impl is a no-op for decoders that don’t support it.

Source