Crate inferlet

Re-exports§

anyhow: Construct an ad-hoc error from a string or existing non-anyhow error value.
bail: Return early with an error.
ensure: Return early with an error if a condition is not satisfied.
format_err: Construct an ad-hoc error from a string or existing non-anyhow error value.
main_async
main_sync

App
Args: An arguments parser.
Blob
Error: The Error type, a wrapper around a dynamic error type.
Model: Represents a specific model instance, providing access to its metadata and functionality.
Queue
Tokenizer

broadcast: Publishes a message to a topic, broadcasting it to all subscribers.
debug_query: Executes a debug command and returns the result as a string.
get_all_models: Get a list of all available model names.
get_all_models_with_traits: Get names of models that have all specified traits (e.g. “input_text”, “tokenize”).
get_arguments: Retrieves POSIX-style CLI arguments passed to the inferlet from the remote user client.
get_auto_model
get_instance_id: Returns a unique identifier for the running instance.
get_model: Retrieve a model by its name.
get_version: Returns the runtime version string.
receive: Receives an incoming message from the remote user client.
receive_blob: Receives an incoming message from the remote user client.
send: Sends a message to the remote user client.
send_blob: Sends a message to the remote user client.
set_return
store_delete: Deletes a key-value pair from the store.
store_exists: Checks if a key exists in the store.
store_get: Retrieves a value from the persistent store for a given key.
store_list_keys: Returns a list of all keys currently in the store.
store_set: Sets a value in the persistent store for a given key.
subscribe: Subscribes to a topic and returns a Subscription handle.