Skip to main content

Crate product_os_models

Crate product_os_models 

Source
Expand description

§product-os-models

A unified machine learning library providing LLM inference, classical ML training, deep learning (Burn), custom computational graph building, and ML tooling – all behind feature flags for minimal dependency footprint.

§Architecture

This crate uses a unified trait hierarchy where all model types implement the base ModelKind trait, with extension traits for specific capabilities:

  • Trainable – models that can be trained on data
  • LlmCapable – large language models with generation capabilities (behind llm)
  • Educatable – models with educational content
  • PretrainedLoadable – models that can load pre-trained weights

§Feature Flags

§Core

FeatureDescription
coreLightweight traits, types, and errors (no heavy dependencies)
stdStandard library support (enabled by default)

§LLM Inference

FeatureDescription
llmLocal LLM inference via Candle (enables candle-core, candle-nn, etc.)
llm-apiRemote LLM backends shared plumbing (requires product-os-request)
llm-ollamaOllama API backend (implies llm-api)
llm-vllmvLLM API backend (implies llm-api)
llm-sglangSGLang API backend (implies llm-api)
llm-maxMAX API backend (implies llm-api)

§Hardware Acceleration (require llm)

FeatureDescription
metalApple Metal GPU backend for Candle (macOS / Apple Silicon)
accelerateApple Accelerate framework for BLAS on macOS
cudaNVIDIA CUDA GPU backend for Candle

§Deep Learning & Burn

FeatureDescription
dl-coreDeep learning foundation via Burn
dl-mlpMulti-layer perceptron architecture (implies dl-core)
dl-cnnConvolutional neural network architecture (implies dl-core)
dl-rnnRecurrent neural network architecture (implies dl-core)
dl-transformerTransformer architecture (implies dl-core)
burn-backendBurn with Candle compute backend (implies llm)

§ML Toolkit Modules

FeatureDescription
classicalClassical ML (linear regression, SVM, k-means, decision trees, etc.)
dataData loading, schema inference, and dataset management
preprocessingFeature engineering and data transformation pipelines
experimentsExperiment tracking, run comparison, and metric logging
tuningHyperparameter optimisation (grid, random, Bayesian)
custom-graphCustom computational graph builder for research models
inferenceInference runtime for trained models
onnxONNX model export and import (implies inference)

Moved to product-os-ml-tooling: eda, interpretability, registry

§Other

FeatureDescription
remoteDownload models from Hugging Face Hub (hf-hub)

§Bundles

FeatureDescription
ml-fullAll ML toolkit modules (classical + DL + data + preprocessing + …)
allEverything: ml-full + llm + all API backends

§Quick Start

§Using the unified trait hierarchy (core only)

use product_os_models::core::traits::{ModelKind, ModelCategory};
// Implement ModelKind for your own model kind...

§Using an Ollama backend (requires llm + llm-api features)

ⓘ
use product_os_models::llm::parameters::Parameters;
use product_os_models::llm::model::ModelDefinition;
use product_os_models::llm::backends::ollama::Ollama;

let model_def = ModelDefinition {
    name: "llama3".to_string(),
    external_model_name: "llama3:8b".to_string(),
    ..Default::default()
};

let mut params = Parameters::new(model_def);
params.root_url = Some("http://localhost:11434".to_string());

let backend = Ollama::new(params);

Re-exports§

pub use llm::model;
pub use llm::parameters;
pub use llm::model_response;
pub use llm::model_runner;
pub use llm::ExecuteChatExtras;
pub use llm::predefined_models;
pub use llm::backends;
pub use llm::local;
pub use llm::models;
pub use core::LlmCapable;
pub use core::ArtifactFormat;
pub use core::DetectedObject;
pub use core::Educatable;
pub use core::MetricLogger;
pub use core::ModelCapabilities;
pub use core::ModelCategory;
pub use core::ModelError;
pub use core::ModelInput;
pub use core::ModelKind;
pub use core::ModelOutput;
pub use core::ModelTensor;
pub use core::PretrainedLoadable;
pub use core::Trainable;

Modules§

core
Core types, traits, and error definitions for the unified ML platform. Core types, traits, and abstractions for the unified ML platform.
llm
LLM inference module: local Candle execution and remote API backends. LLM inference capabilities (existing product-os-models functionality).