Re-exports§
pub use crate::channels::mutex::MutexChannel;pub use crate::channels::passthrough::SingleThreadedChannel;pub use crate::client::InferenceClient;pub use crate::message::Message;pub use crate::message::MessageRole;pub use crate::plugin::InferencePlugin;pub use crate::server::InferenceServer;pub use crate::server::InferenceServerConfig;pub use crate::server::ServerConfigParsing;pub use crate::stats::StatEntry;pub use crate::stats::Stats;pub use crate::stats::STATS_MARKER;pub use backends::DTYPE_NAME;pub use clap;pub use serde_json;pub use crate::errors::*;pub use backends::burn_backend_types::*;
Modules§
Structs§
- Generated
Item Emitter - An emitter is responsible to send generated items to the inference job channel.
- Inference
Job - Defines a job to be run during inference.
- JobHandle
- The handle returned by [InferenceJob::start].
- StdOut
Listener - The stdout listener directly writes the intermediary generated item to std::io::stdout.
- Text
Generation Listener - The text generation listener accumulate the generated text in a string that can be obtained at the end of the job with the handle finished method.
Enums§
- Generated
Item - Defines all the potential items that can be generated by an inference job.
- Inference
Task - The potential tasks that can be executed by an inference job using the inference server.
Traits§
- Any
- A trait to emulate dynamic typing.
- Backend
- This trait defines all types and functions needed for a backend to be used with burn.
- Command
Factory - Create a
Commandrelevant for a user-defined container. - Deserialize
- A data structure that can be deserialized from any data format supported by Serde.
- From
ArgMatches - Converts an instance of
ArgMatchesto a user-defined container. - Inference
JobListener - An inference job listener receive events while the inference job is running.
- Parser
- Parse command-line arguments into
Self.
Type Aliases§
Attribute Macros§
- inference_
server_ config - This macro consumes the struct, extracts any
#[config(default = ...)]attributes and regenerates a brand-new struct with:
Derive Macros§
- Deserialize
- Inference
Server - Parser
- Generates the
Parserimplementation.