Expand description
Unified LLM provider abstraction (ADR-010).
One trait covers both local Candle inference and cloud frontier LLMs. The
host picks a backend at session construction time — the rest of the SDK
sees only LlmProvider. This is the seam that lets mobile apps swap
local ↔ frontier without touching their UI code.
Design notes:
- All types are plain
std(no async runtime dep in this crate). chat_streamuses a callback so each binding surface wraps it in its own async/stream primitive (FRB → DartStream, uniffi → async callback, wasm-bindgen →ReadableStream).CredentialRefis a runtime value from the host — never embedded.
Structs§
- Chat
Message - One turn in the conversation.
- Chat
Request - A chat completion request.
modelis a routing hint: - Chat
Response - A completed chat response.
- Chat
Token - A single streamed token fragment.
- Credential
Ref - Runtime API credential. The host resolves this from platform keystore
(Android Keystore / iOS Keychain) before calling
start_session. The SDK never logs or persists the value.
Enums§
- Chat
Role - Which role a message belongs to.
Traits§
- LlmProvider
- The unified backend trait (ADR-010). Implemented by: