opf-mlx 0.2.1

Thin Rust client for the native MLX OpenAI Privacy Filter server
Documentation

opf-mlx

opf-mlx is a small Rust client for a persistent native MLX implementation of the OpenAI Privacy Filter. It tokenizes text in Rust, exchanges bounded binary messages with the helper process, applies the checkpoint's Viterbi calibration, and returns UTF-8 byte spans for detected privacy categories.

The crate does not compile or link MLX, Metal, C++, model weights, or native runtime code. Applications provide the separately built helper executable and model artifact when creating a client.

Platform status

The client targets Unix process and pipe APIs. The accompanying native helper currently targets Apple silicon and is distributed separately from this crate.

Example

use std::path::PathBuf;

use opf_mlx::{FixedTraceShape, PrivacyFilter, ServerOptions};

let options = ServerOptions {
    max_batch_tokens: 16_384,
    ..ServerOptions::default()
};

let mut filter = PrivacyFilter::spawn_fixed(
    PathBuf::from("/path/to/opf-mlx-server"),
    PathBuf::from("/path/to/model.mlxfn"),
    options,
    FixedTraceShape {
        batch: 4,
        tokens: 4_096,
    },
)?;
let detection = filter.detect("Contact alice@example.com")?;

for span in detection.spans {
    println!("{}: {}..{}", span.label.as_str(), span.start, span.end);
}

# Ok::<(), opf_mlx::Error>(())

detect_batch accepts mixed document lengths. The native server automatically groups requests by exported token bucket under the configured padded-token budget and restores the input order. The budget applies to every Metal invocation, including a single document, so an unexpectedly long input cannot select an oversized trace. ServerOptions::default() uses a 32,768 padded-token budget and an independent 0.5 GiB MLX allocator-cache limit. Set log_memory_stats to report active, peak, and cached bytes on server stderr.

For memory-constrained deployments, call spawn_fixed with an exact shape listed in model.mlxfn.json. Every invocation then uses that one batch/token shape, including padded rows in the final batch. Inputs longer than the fixed token length, missing exported shapes, and shapes exceeding max_batch_tokens fail before inference. This avoids retaining compiled graphs for several shapes at the cost of additional padding. Use spawn for dynamic multi-shape execution only after it has been memory-qualified on the target machine. Fixed mode prewarms its selected trace before startup returns by default; set prewarm_fixed_trace to false when cold startup latency is preferable to a predictable first request. memory_limit_gb is an MLX allocator setting, not a hard process-memory or compiled-graph limit.

detect_chunked_batch is lossless: it processes inputs that already fit the stricter of the requested chunk size and the configured fixed-trace length, and returns LosslessChunkingUnavailable rather than silently splitting a longer input. Exact full-document equivalence is not available with the current two-input exported-model contract. If boundary-dependent output is acceptable, opt in explicitly with detect_chunked_batch_approximate(texts, 4_096, 2_048). The approximate method tokenizes each source once, maps findings back to original UTF-8 byte offsets, and deduplicates only identical spans; it does not stitch logits or run a global Viterbi decode. In fixed mode, its requested chunk size must not exceed the configured trace length.

Privacy boundary

Raw text remains in the caller. Only token IDs are sent to the local helper, and only BIOES labels are returned. The client validates protocol cardinality, size limits, timeouts, UTF-8 boundaries, and calibrated span decoding. Policy, overlap resolution, pseudonymization, and storage belong to the application.

License

Apache-2.0.