Module jsonl_proc 

NDJSON / JSON Lines structured processor.

Processes files where each non-empty line is an independent JSON object (Newline-Delimited JSON, also called JSON Lines).

The CLI uses process_to_edits: it runs the JSON span walker on each line and offsets the resulting edits by the line’s byte position, so each line’s exact formatting is preserved and values escaped in the source never leak. Files within --max-structured-size are processed this way; larger files fall back to the bounded-memory streaming path (process_stream), which parses, walks, and re-serializes each line independently — keeping per-line memory constant for GB-scale NDJSON.

Options

Key	Values	Default	Description
skip_invalid	"true" / "false"	"false"	Pass malformed lines through unchanged instead of returning an error. Useful for mixed log files that interleave plain-text lines with JSON.
compact	"true" / "false"	"true"	Serialise each output line as compact JSON. Set to "false" only for debugging — pretty-printed NDJSON is non-standard.

Example profile entry

- processor: jsonl
  extensions: [".jsonl", ".ndjson", ".log"]
  options:
    skip_invalid: "true"
  fields:
    - pattern: "*.email"
      category: email
    - pattern: "*.password"
      category: "custom:password"

Structs

JsonLinesProcessor

Structured processor for NDJSON / JSON Lines files.