Crate nemo_relay 

NeMo Relay Core

The core runtime library for the NeMo Relay multi-language agent framework. This crate provides execution scope management, lifecycle event tracking, and middleware pipelines (guardrails and intercepts) for tool and LLM calls.

Architecture

The runtime is organized around a global context (api::runtime::NemoRelayContextState) that holds all registered middleware (guardrails, intercepts, subscribers) and a scope stack (api::runtime::ScopeStack) that tracks the hierarchical execution context via task-local or thread-local storage.

Primary Entry Points

Most integrations start with the high-level lifecycle helpers in api:

• api::scope::push_scope / api::scope::pop_scope create nested execution scopes.
• api::tool::tool_call_execute runs a complete tool middleware pipeline.
• api::llm::llm_call_execute and api::llm::llm_stream_call_execute run non-streaming and streaming LLM middleware pipelines.
• api::registry exposes global and scope-local middleware registration APIs.
• api::subscriber exposes lifecycle event subscriber registration APIs.

Modules

• api — Public API functions, handles, lifecycle event types, runtime helpers, and guardrail/intercept/subscriber registration. These are the primary entry points.
• error — Error types (error::FlowError) and the error::Result type alias.
• json — JSON type alias (json::Json) and the json::merge_json utility.
• logging — Process operational logging (log facade + configurable stderr/file sinks).
• observability — Built-in observability backends including atif::AtifExporter, otel::OpenTelemetrySubscriber, with full, GenAI, or OpenInference projection.
• stream — stream::LlmStreamWrapper — a stream adapter that applies per-chunk intercepts and aggregates streaming LLM responses.

Middleware Pipeline

Both tool and LLM calls flow through a configurable middleware pipeline:

1. Request intercepts — transform the request before execution
2. Sanitize request guardrails — sanitize/normalize the request
3. Conditional execution guardrails — gate execution (reject if criteria not met)
4. Execution intercepts — optionally replace the execution function entirely
5. Sanitize response guardrails — sanitize/normalize the response

All middleware is priority-ordered (ascending) and registered by name for easy addition and removal at runtime.

Modules

api

Public API for the NeMo Relay runtime.

codec

LLM codec types, traits, and built-in implementations.

config_editor

Typed configuration editor metadata.

error

Error types for the NeMo Relay runtime.

json

JSON utilities for the NeMo Relay runtime.

logging

Operational process logging for Relay (stderr + optional file sinks).

observability

Optional observability integrations for NeMo Relay Core.

plugin

Generic plugin infrastructure for NeMo Relay runtimes.

plugins

First-party plugin implementations for NeMo Relay Core.

stream

Streaming LLM response wrapper.

Macros

editor_config

Implements EditorConfig for a configuration type.