Skip to main content

detect_file_format

Function detect_file_format 

Source
pub fn detect_file_format(path: &Path) -> InferredFileFormat
Expand description

Decide how to dispatch an ingest / inspect call for path.

Extension first (zero-IO, cheap): binary formats (.parquet, .pq, .arrow, .ipc, .feather) always win by extension because the file is a binary container whose magic bytes we’d have to unpack anyway. Known text extensions (.json, .jsonl, .ndjson) map straight to JSON without needing to read the file.

Otherwise — for unknown, ambiguous, or missing extensions (.log, .txt, no extension at all) — peek at the first 4 KiB and return InferredFileFormat::Json if the first non-whitespace byte is [ or {, else InferredFileFormat::Csv. This is the path that lets hyperd’s raw .log files load without renaming, since JSONL lines always begin with {.

Returns InferredFileFormat::Csv if the file can’t be opened; the subsequent CSV ingest will surface a clearer error than a format-detection failure would.