liboxen 0.53.0

Oxen is a fast data version control system, built with machine learning training data in mind. Designed to handle terabytes of data with ease, using a workflow similar to git. Version both structured and unstructured data of any modality: text, images, video, audio, CSV, Parquet, JSONL, model checkpoints, and more. liboxen is the embeddable core library behind the oxen CLI and server, which power fine tuning and inference pipelines for multimodal LLMs, image models, and video models on Oxen.ai.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

//! One-shot DuckDB extension installation.

use crate::error::OxenError;

/// Install the JSON extension into DuckDB's user extensions directory so subsequent
/// connections load it without triggering the autoload flow.
///
/// Call once at startup before any parallel DuckDB use. Concurrent first-use autoloads
/// race on the extension file's temp→final rename (`Could not move file: Access is
/// denied` on Windows, comparable timing hazards elsewhere); preinstalling closes that
/// window.
pub fn preload_extensions() -> Result<(), OxenError> {
    let conn = duckdb::Connection::open_in_memory().map_err(|e| {
        OxenError::basic_str(format!(
            "failed to open in-memory DuckDB connection for extension preload: {e}"
        ))
    })?;
    conn.execute_batch("INSTALL json;")
        .map_err(|e| OxenError::basic_str(format!("failed to install DuckDB json extension: {e}")))
}