Crate anyllm_cloudflare_worker 

Cloudflare Workers AI provider for anyllm.

This crate implements anyllm::ChatProvider using the worker::Ai binding, enabling AI model calls directly from Cloudflare Workers without HTTP overhead. Construction is runtime-bound: unlike the HTTP providers, this adapter is not loaded from local environment variables and instead requires a live Worker Ai binding.

Usage

use anyllm::prelude::*;
use anyllm_cloudflare_worker::Provider;

async fn handler(env: &worker::Env) -> Result<String> {
    let ai = env.ai("AI").map_err(|e| anyllm::Error::Provider {
        status: None,
        message: e.to_string(),
        body: None,
        request_id: None,
    })?;

    let provider = Provider::new(ai);

    let response = provider.chat(
        &ChatRequest::new("@cf/meta/llama-3.1-8b-instruct")
            .message(Message::user("What is Rust?"))
    ).await?;

    Ok(response.text_or_empty())
}

Streaming

Streaming uses Ai::run_bytes() with stream: true, parsing SSE events from the raw byte stream:

use anyllm::prelude::*;
use anyllm_cloudflare_worker::Provider;
use futures_util::StreamExt;

async fn stream_handler(env: &worker::Env) -> Result<()> {
    let ai = env.ai("AI").map_err(|e| anyllm::Error::Provider {
        status: None,
        message: e.to_string(),
        body: None,
        request_id: None,
    })?;

    let provider = Provider::new(ai);

    let mut stream = provider.chat_stream(
        &ChatRequest::new("@cf/meta/llama-3.1-8b-instruct")
            .message(Message::user("Tell me a story"))
    ).await?;

    while let Some(event) = stream.next().await {
        let event = event?;
        if let StreamEvent::TextDelta { text, .. } = &event {
            // Process streaming text
        }
    }
    Ok(())
}

Structs

Provider

Cloudflare Workers AI provider implementing anyllm::ChatProvider.