Crate agentkit_provider_vllm 

vLLM model adapter for the agentkit agent loop.

This crate provides VllmAdapter and VllmConfig for connecting the agent loop to a vLLM server via its OpenAI-compatible chat completions endpoint. It is built on the generic agentkit_adapter_completions crate.

An API key is optional — vLLM servers can run with or without authentication.

Quick start

use agentkit_loop::{Agent, SessionConfig};
use agentkit_provider_vllm::{VllmAdapter, VllmConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = VllmConfig::new("meta-llama/Llama-3.1-8B-Instruct");
    let adapter = VllmAdapter::new(config)?;

    let agent = Agent::builder()
        .model(adapter)
        .build()?;

    let mut driver = agent
        .start(SessionConfig::new("demo"))
        .await?;
    Ok(())
}

Structs

VllmAdapter

Model adapter that connects the agentkit agent loop to a vLLM server.

VllmConfig

Configuration for connecting to a vLLM server.

VllmProvider

The vLLM provider, implementing CompletionsProvider.

VllmRequestConfig

Request parameters serialized into the vLLM request body.

Enums

VllmError

Errors produced by the vLLM adapter.

Type Aliases

VllmSession

An active session with a vLLM server.

VllmTurn

A completed turn from a vLLM server.