Module rust_bert::resources

Resource definitions for model weights, vocabularies and configuration files

This crate relies on the concept of Resources to access the data used by the models. This includes:

• model weights
• configuration files
• vocabularies
• (optional) merges files for BPE-based tokenizers

These are expected in the pipelines configurations or are used as utilities to reference to the resource location. Two types of resources are pre-defined:

• LocalResource: points to a local file
• RemoteResource: points to a remote file via a URL
• BufferResource: refers to a buffer that contains file contents for a resource (currently only usable for weights)

For LocalResource and RemoteResource, the local location of the file can be retrieved using get_local_path, allowing to reference the resource file location regardless if it is a remote or local resource. Default implementations for a number of RemoteResources are available as pre-trained models in each model module.

Structs

• BufferResource
  In-memory raw buffer resource
• LocalResource
  Local resource
• RemoteResource
  Remote resource that will be downloaded and cached locally on demand

Enums

• Resource

Traits

• ResourceProvider
  Resource Trait that can provide the location or data for the model, and location of

Functions

• load_weights
  Load the provided VarStore with model weights from the provided ResourceProvider