llm_interface 0.0.3

llm_interface: The backend for the llm_client crate
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<!-- cargo-rdme start -->

# llm_interface: The backend for the llm_client crate
[![API Documentation](https://docs.rs/llm_interface/badge.svg)](https://docs.rs/llm_interface)

The llm_interface crate is a workspace member of the [llm_client](https://github.com/ShelbyJenkins/llm_client) project.

This crate contains the build.rs, data types, and behaviors for LLMs.

## Features

* Integration with Llama.cpp (through llama-server)
    * Repo cloning and building
    * Managing Llama.cpp server
* Support for various LLM APIs including generic OpenAI format LLMs

This crate enables running local LLMs and making requests to LLMs, designed
for easy integration into other projects.

## Examples

See the various `Builders` implemented in the [integration tests](./tests/it/main.rs) for examples
of using this crate.

For a look at a higher level API and how it implements this crate, see the
[llm_client](https://github.com/ShelbyJenkins/llm_client) crate.

<!-- cargo-rdme end -->