newt-inference

Newt-Agent inference layer: backend trait + local Ollama/vLLM + provider-plugin host.

• backend::InferenceBackend is the trait every backend implements.
• local::LocalOllamaBackend and local::LocalVllmBackend are the only backends compiled into the default Newt binary.
• provider_plugin::ProviderPluginBackend spawns a subprocess speaking the Newt-Provider JSON-RPC protocol — how OpenAI, Anthropic, etc. join via opt-in plugin installs.

Also provides the BackendRegistry, streaming chat types, and retry/backoff helpers used by the rest of the workspace.

Part of Newt-Agent, a small, fast, local-first agentic coder.

License

Apache-2.0