atomr-infer 0.8.0

Multi-runtime GPU + remote inference as a supervised actor system on atomr. Single-crate rollup that re-exports per-runtime backends behind feature flags.
Documentation

atomr-infer

There is currently very little information to present on this page because Docs.rs has only limited support for extracting structured feature metadata from Cargo crates. This issue is tracked in Rust RFC #3416. Check this library's main docs, readme, and Cargo.toml in case its authors have documentation for features available there instead.

This version has 30 feature flags, 0 of them enabled by default.

default

This feature flag does not enable additional features.

accel

accel-patterns

all-local

all-native

all-python

all-remote

all-runtimes

anthropic

candle

cudarc

default-prod

gemini

gemma-default

litellm

mistralrs

openai

ort

ort-cuda

ort-hf-hub

ort-load-dynamic

pipeline

remote-only

tensorrt

tensorrt-fp8

tensorrt-int8

tensorrt-onnx

tensorrt-plugin

testkit

vllm