atomr-infer 0.8.0

Multi-runtime GPU + remote inference as a supervised actor system on atomr. Single-crate rollup that re-exports per-runtime backends behind feature flags.
Documentation

atomr-infer

There is very little structured metadata to build this page from currently. You should check the main library docs, readme, or Cargo.toml in case the author documented the features in them.

This version has 30 feature flags, 0 of them enabled by default.

default

This feature flag does not enable additional features.

accel

accel-patterns

all-local

all-native

all-python

all-remote

all-runtimes

anthropic

candle

cudarc

default-prod

gemini

gemma-default

litellm

mistralrs

openai

ort

ort-cuda

ort-hf-hub

ort-load-dynamic

pipeline

remote-only

tensorrt

tensorrt-fp8

tensorrt-int8

tensorrt-onnx

tensorrt-plugin

testkit

vllm