Skip to main content

JsonParserConfig

Struct JsonParserConfig 

Source
pub struct JsonParserConfig {
    pub format: Option<JsonFormat>,
    pub max_rows: Option<usize>,
    pub error_policy: JsonErrorPolicy,
}
Expand description

Configuration for JSON/JSONL parsing and scanning.

The default detects the grammar from the input and reads every record, skipping malformed ones. The two format constructors say which grammar to read with, which is what a caller who knows should do: detection cannot tell a JSON object document from a JSONL file, since both open with {.

§Examples

The same two records under each grammar:

use std::io::Cursor;

use dataprof_json::{JsonParserConfig, scan_json_from_reader};

let document = r#"[{"id": 1}, {"id": 2}]"#;
let lines = "{\"id\": 1}\n{\"id\": 2}\n";

let from_document = scan_json_from_reader(
    Cursor::new(document),
    &JsonParserConfig::json_document(),
    |_| {},
)?;
let from_lines =
    scan_json_from_reader(Cursor::new(lines), &JsonParserConfig::jsonl(), |_| {})?;

assert_eq!(from_document.rows_read, 2);
assert_eq!(from_lines.rows_read, 2);

Fields§

§format: Option<JsonFormat>

The grammar to read with. None detects it from the first byte, which resolves [ to JsonFormat::Json and everything else to JsonFormat::Jsonl — a leading { opens both a JSON object document and a JSONL file, so callers that know which they have (from the file extension or an explicit format=) should say so rather than rely on detection.

§max_rows: Option<usize>

Maximum rows to process (None = all rows).

§error_policy: JsonErrorPolicy

How to react to a malformed record (default: skip).

Implementations§

Source§

impl JsonParserConfig

Source

pub fn with_max_rows(self, max_rows: usize) -> Self

Set the maximum number of rows to process.

The cap counts profileable records, not lines: blank lines and records skipped as malformed do not consume it.

A cap is only truncation when a record still remained once it was reached, so a source holding exactly max_rows records reads as complete. JsonScanSummary::truncated carries that distinction, and analyze_json_file turns it into a TruncationReason::MaxRows on the report.

§Examples
use std::io::Cursor;

use dataprof_json::{JsonParserConfig, scan_json_from_reader};

let lines = "{\"id\": 1}\n{\"id\": 2}\n{\"id\": 3}\n";

// Two of three records: a third remained, so the scan was cut short.
let capped = JsonParserConfig::jsonl().with_max_rows(2);
let summary = scan_json_from_reader(Cursor::new(lines), &capped, |_| {})?;
assert_eq!(summary.rows_read, 2);
assert!(summary.truncated);

// A cap the source never reaches past is a complete read, not a
// truncated one.
let exact = JsonParserConfig::jsonl().with_max_rows(3);
let summary = scan_json_from_reader(Cursor::new(lines), &exact, |_| {})?;
assert_eq!(summary.rows_read, 3);
assert!(!summary.truncated);
Source

pub fn with_error_policy(self, policy: JsonErrorPolicy) -> Self

Set how malformed records are handled.

Under the default JsonErrorPolicy::Skip a malformed record is counted in JsonScanSummary::malformed_lines and scanning continues, so a partial profile never looks like a clean one. Under JsonErrorPolicy::Strict the first one fails the scan.

§Examples
use std::io::Cursor;

use dataprof_json::{JsonErrorPolicy, JsonParserConfig, scan_json_from_reader};

let lines = "{\"id\": 1}\nnot json\n{\"id\": 2}\n";

let skipping = JsonParserConfig::jsonl();
let summary = scan_json_from_reader(Cursor::new(lines), &skipping, |_| {})?;
assert_eq!(summary.rows_read, 2);
assert_eq!(summary.malformed_lines, 1);

let strict = JsonParserConfig::jsonl().with_error_policy(JsonErrorPolicy::Strict);
assert!(scan_json_from_reader(Cursor::new(lines), &strict, |_| {}).is_err());
Source

pub fn jsonl() -> Self

Read as JSON Lines: one record per physical line.

Source

pub fn json_document() -> Self

Read as a standard JSON document: an array of records, or a single object as one record.

Source

pub fn with_format(self, format: JsonFormat) -> Self

Set the grammar explicitly.

Trait Implementations§

Source§

impl Clone for JsonParserConfig

Source§

fn clone(&self) -> JsonParserConfig

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for JsonParserConfig

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for JsonParserConfig

Source§

fn default() -> JsonParserConfig

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V