Skip to main content

Module export

Module export 

Source
Expand description

Writing the view to an export file. run takes the plan to a committed file: uncompressed CSV and Parquet stream through Polars’ sink into the OutputFile; everything else (compressed CSV, JSON, NDJSON, IPC, Avro, or no streaming engine) collects and encodes. The destination changes only at commit, after the last byte, encoder finish and flush succeed. Streaming bounds the export, not the plan: a sort, group-by or join still gathers its input first.

Modules§

avro_types
Columns cast to types Polars’ Avro writer can hold, for Avro export. The writer handles booleans, 32/64-bit ints and floats, strings, binary, dates, naive ms/µs datetimes, and lists and structs of those; other types fail the export, so they are strictly cast to the nearest (a value that does not fit fails by column name). Decimals are written as exact text: the writer drops a positive value’s sign byte when its leading byte is ≥ 0x80 (327.68 reads -327.68). Avro names must be [A-Za-z_][A-Za-z0-9_]*, so write() names the record and gives columns and struct fields valid names (originals in doc), writes the header once (Polars repeats it per chunk) and cuts blocks by size.
export_modal
Export modal state and focus management.
nested_json
List, array and struct cells as JSON text for one-value-per-field destinations (CSV export, clipboard copies): JSON keeps struct field names and reads back anywhere (str.json_decode). The text is Polars’ JSON writer’s, as in NDJSON export.
output_file
A file written whole or not at all. OutputFile writes a private temp file beside the destination and moves it into place in OutputFile::commit; until then a failure leaves the old file (bytes and permissions) untouched and a new export leaves nothing; dropping uncommitted (even in a panic) removes the temp file. The file is synced before the rename (some errors surface only then); the rename is not, so a power cut right after can lose it. Every user-facing file goes through here: exports, the Data Quality report, chart images.
python_script
Copy as Python: the view as a Python Polars script, built from datui’s record of how the view was made (the open’s reader and options, then each Step), never by printing a plan, so it reads like handwritten code. A step Python cannot repeat is said in a comment and everything after is commented out with it: the script never computes something other than the screen.

Structs§

ExportOptions
ExportRequest
One export: the file, its form, and whether it may replace a file there.

Enums§

Route
How an export’s rows reach its file.

Functions§

encode
Encode df into sink and finish (drain buffers, write the last block and trailer, flush), so failures surface here rather than at drop.
run
Run an export: plan lf for the format (ExportFormat::prepare), write by its Route, commit. written hears bytes so far (0 at start, then a few times a second). The destination changes only if everything was written.