pub fn stream_reader<R>(
reader: R,
format: &str,
f: &mut dyn FnMut((String, String, String, Option<String>)),
) -> Result<(), IngestError>where
R: BufRead,Expand description
Stream N-Triples ("nt") / N-Quads ("nq") from a reader, invoking f
once per parsed quad — like parse_reader but without collecting into a
Vec. Each quad’s term Strings are owned by f (and dropped when it returns
if it doesn’t retain them), so a caller that only needs to observe every
term — e.g. the two-pass assemble_dataset_streaming building its
dictionary — never materializes the whole quad multiset. The big-graph,
low-RAM ingest primitive. Blank/comment lines are skipped; a parse error stops
the stream and is returned.
Every format streams: the line-based ones a line at a time, Turtle / TriG /
RDF-XML through their oxttl/oxrdfxml reader parsers, which pull from the
reader incrementally. Nothing here holds the input text — so a 60 GB Turtle
file is as ingestible as a 60 GB .nt one, which is what lets the external
(memory-bounded) build accept them.