Trait JsonHandler 

pub trait JsonHandler: AsAny {
    // Required methods
    fn parse_json(
        &self,
        json_strings: Box<dyn EngineData>,
        output_schema: SchemaRef,
    ) -> DeltaResult<Box<dyn EngineData>>;
    fn read_json_files(
        &self,
        files: &[FileMeta],
        physical_schema: SchemaRef,
        predicate: Option<PredicateRef>,
    ) -> DeltaResult<FileDataReadResultIterator>;
    fn write_json_file(
        &self,
        path: &Url,
        data: DeltaResultIterator<'_, FilteredEngineData>,
        overwrite: bool,
    ) -> DeltaResult<FileSize>;

    // Provided method
    fn read_json_files_with_cancellation(
        &self,
        files: &[FileMeta],
        physical_schema: SchemaRef,
        predicate: Option<PredicateRef>,
        cancellation_token: Option<CancellationTokenRef>,
    ) -> DeltaResult<FileDataReadResultIterator> { ... }
}

Provides JSON handling functionality to Delta Kernel.

Delta Kernel can use this handler to parse JSON strings into Row or read content from JSON files. Connectors can leverage this trait to provide their best implementation of the JSON parsing capability to Delta Kernel.

Required Methods

fn parse_json( &self, json_strings: Box<dyn EngineData>, output_schema: SchemaRef, ) -> DeltaResult<Box<dyn EngineData>>

Parse the given json strings and return the fields requested by output schema as columns in EngineData. json_strings MUST be a single column batch of engine data, and the column type must be string

fn read_json_files( &self, files: &[FileMeta], physical_schema: SchemaRef, predicate: Option<PredicateRef>, ) -> DeltaResult<FileDataReadResultIterator>

Read and parse the JSON format file at given locations and return the data as EngineData with the columns requested by physical schema. Note: The FileDataReadResultIterator must emit data from files in the order that files is given. For example if files [“a”, “b”] is provided, then the engine data iterator must first return all the engine data from file “a”, then all the engine data from file “b”. Moreover, for a given file, all of its EngineData and constituent rows must be in order that they occur in the file. Consider a file with rows (1, 2, 3). The following are legal iterator batches: iter: [EngineData(1, 2), EngineData(3)] iter: [EngineData(1), EngineData(2, 3)] iter: [EngineData(1, 2, 3)] The following are illegal batches: iter: [EngineData(3), EngineData(1, 2)] iter: [EngineData(1), EngineData(3, 2)] iter: [EngineData(2, 1, 3)]

Additionally, engines may not merge engine data across file boundaries.

Parameters

• files - File metadata for files to be read.
• physical_schema - Select list of columns to read from the JSON file.
• predicate - Optional conservative push-down predicate. Implementations may ignore it. If applied, a file or row may be omitted only when the predicate cannot evaluate to true for any row in that unit. CAST conversion, failure, and NULL semantics must match final filtering over exact row values. Unsupported or missing references remain unknown without failing the read, including references synthesized as NULL columns during schema reconciliation. Returned data is not guaranteed to satisfy the predicate.

fn write_json_file( &self, path: &Url, data: DeltaResultIterator<'_, FilteredEngineData>, overwrite: bool, ) -> DeltaResult<FileSize>

Atomically (!) write a single JSON file. Each selected row of the input data must be written as a new JSON object appended to the file; rows not selected by a batch’s selection vector (see FilteredEngineData) must not be written. FilteredEngineData::apply_selection_vector produces the selected-rows view for implementations that do not filter during serialization. This write must: (1) serialize the selected rows to newline-delimited json (each row is a json object literal) (2) write the data to storage atomically (i.e. if the file already exists, fail unless the overwrite flag is set)

For example, the JSON data should be written as { “column1”: “val1”, “column2”: “val2”, .. } with each row on a new line.

NOTE: Null columns should not be written to the JSON file. For example, if a row has columns [“a”, “b”] and the value of “b” is null, the JSON object should be written as { “a”: “…” }. Note that including nulls is technically valid JSON, but would bloat the log, therefore we recommend omitting them.

Parameters

• path - URL specifying the location to write the JSON file
• data - Iterator of FilteredEngineData to write to the JSON file
• overwrite - If true, overwrite the file if it exists. If false, the call must fail if the file exists.

Returns

The exact number of serialized bytes written to the file.

Errors

Returns Error::FileAlreadyExists when overwrite is false and the destination exists, or another error when serialization or storage fails.

Provided Methods

fn read_json_files_with_cancellation( &self, files: &[FileMeta], physical_schema: SchemaRef, predicate: Option<PredicateRef>, cancellation_token: Option<CancellationTokenRef>, ) -> DeltaResult<FileDataReadResultIterator>

Cancellation-aware variant of read_json_files.

When cancellation_token is Some, an engine may race its I/O against the token and terminate the returned iterator with Error::Cancelled once cancellation is observed, rather than reading every file to completion.

The default implementation returns Error::Cancelled if the token is already cancelled and otherwise delegates to read_json_files, ignoring the token for the rest of the read. So an engine that does not override this stays source-compatible while still honoring an up-front cancellation; kernel additionally polls the token at action-batch boundaries. An engine that overrides this takes over the up-front check and should also fast-path an already-cancelled token before starting I/O.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl JsonHandler for MeteredJsonHandler

impl JsonHandler for PlanBasedJsonHandler

Available on crate feature declarative-plans and crate feature default-engine-base and (crate features arrow-conversion or declarative-plans or default-engine-base) only.