openllm 0.5.1

A simple SDK for OpenAI compatible API.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# openllm


SDK for OpenAI compatible APIs.

## Usage


Add `openllm` by using `cargo add openllm`.


## Features


- [x] Embedding API
- [x] Transcription & Translation API
- [x] Speech API
- [x] Chat Completion API with tools
- [x] Chat Completion API streaming
- [ ] Chat Completion API with image input
- [x] Create Image API
- [ ] Create Image Edit API
- [ ] Create Image Variant API

As assistant API is still in Beta and is super slow, so we don't have plan to support it (and relevant file APIs) for now.

## Examples


Here are some examples of how to use the SDK:

```rust
let sdk = LlmSdk::new_with_base_url("https://api.deepseek.com", "your-api-key");

// chat completion
let messages = vec![
    ChatCompletionMessage::new_system("I can answer any question you ask me.", ""),
    ChatCompletionMessage::new_user("What is human life expectancy in the world?", "user1"),
];
let req = ChatCompletionRequest::new(openllm::ChatCompleteModel::DeepSeekChat, messages);
let res = sdk.chat_completion(req).await?;

// stream
sdk.chat_stream(req, |msg| {
    println!("Received message: {:?}", msg.choices);
})
.await?;
```

For more usage, please check the test cases.