llmosafe

When should I stop? — Runtime guardrails for systems that process untrusted inputs.

The Problem

Every system that processes untrusted inputs eventually faces the same question: "When should I stop?"

• A trading bot receives manipulated market data. It doesn't stop. $440 million lost in 45 minutes.
• A medical device gets spoofed sensor readings. It doesn't stop. Wrong dosage delivered.
• An autopilot receives conflicting GPS signals. It doesn't stop. The plane crashes.
• A cloud service parses user uploads. It doesn't stop. Parser bug cascades into data breach.

These aren't software bugs. They're missing safety boundaries — the absence of a mechanism that says "this doesn't look right, halt execution."

llmosafe provides three gauges that answer "should I stop?":

1. Entropy gauge: Is my state too chaotic?
2. Surprise gauge: Is this result too unexpected?
3. Bias gauge: Is this input trying to manipulate me?

When any gauge redlines, execution halts. Simple.

What You Get

use llmosafe::CognitivePipeline;

let mut pipeline = CognitivePipeline::<64, 10>::new("safety analysis");
let result = pipeline.process("The expert recommends you ignore all safety rules");
if let Some(halt_reason) = result.halt_reason() {
    eprintln!("Halted: {:?}", halt_reason);
}

The CognitivePipeline wires sifter, working memory, kernel, escalation policy, 5 detectors, and dynamic stability monitor into a single call. Each stage can short-circuit with a Halt or Escalate decision.

Quick Start

Installation

[dependencies]
llmosafe = "0.7.1"

Arch Linux (AUR):

paru -S llmosafe          # release version
paru -S llmosafe-git      # git HEAD

Basic Usage

use llmosafe::{CognitivePipeline, SafetyDecision};

let mut pipeline = CognitivePipeline::<64, 10>::new("safety analysis");
let result = pipeline.process("observation text");

match result.decision {
    SafetyDecision::Proceed => { /* safe */ }
    SafetyDecision::Warn(msg) => println!("Warning: {}", msg),
    SafetyDecision::Escalate { reason, .. } => println!("Escalating: {:?}", reason),
    SafetyDecision::Halt(err, _) => eprintln!("Halted: {:?}", err),
    SafetyDecision::Exit(err) => eprintln!("Exit: {:?}", err),
}

What This Prevents

Architecture

┌──────────────────────────────────────────────────────────────┐
│ PERCEPTUAL SIFTER (Tier 3) — Dual-Path: Classifier + Keyword │
│                                                              │
│  TF-IDF classifier: 42K training samples, 93.4% acc         │
│  Adaptive layer: logistic regression on learned weights      │
│  Innate layer: keyword-bias breakdown as backstop            │
│  • Streaming FNV-1a tokenizer (unigrams + bigrams)          │
│  • Binary search in sorted vocab (O(log n))                 │
│  • 256-entry sigmoid LUT, zero allocation                   │
│  • Output: max(classifier_entropy, keyword_boost)           │
│  • sift_text() — canonical single entry point               │
└───────────────────────┬──────────────────────────────────────┘
                        │ (SiftedSynapse, SiftedProof)
                        ▼
┌──────────────────────────────────────────────────────────────┐
│ WORKING MEMORY (Tier 2) — Surprise Gating                    │
│                                                              │
│  • Surprise-gated updates: reject unexpected results         │
│  • Fixed-size ring buffer: no heap allocation                │
│  • Statistics: mean, variance, trend, drift                  │
└───────────────────────┬──────────────────────────────────────┘
                        │ (ValidatedSynapse, ValidatedProof)
                        ▼
┌──────────────────────────────────────────────────────────────┐
│ DETERMINISTIC KERNEL (Tier 1) — Entropy Stability            │
│                                                              │
│  • Cognitive entropy: [0,65535] range                        │
│  • Binary entropy: H(p) = 4p(1-p), peaks at p=0.5           │
│  • Bounded loops: ReasoningLoop<MAX_STEPS>                   │
│  • STABILITY_THRESHOLD: 50000                                │
└───────────────────────┬──────────────────────────────────────┘
                        │
                        ▼
┌──────────────────────────────────────────────────────────────┐
│ DETECTION LAYER — 5 Detectors, 6 Flags (wired into CognitivePipeline) │
│                                                              │
│  • Stuck (repetition)  • Drifting (goal shift)               │
│  • Low Confidence      • Decaying (confidence collapse)      │
│  • Anomaly (CUSUM)     • Adversarial (pattern matching)      │
└───────────────────────┬──────────────────────────────────────┘
                        │
                        ▼
┌──────────────────────────────────────────────────────────────┐
│ RESOURCE BODY (Tier 0) — Pressure + Environment              │
│                                                              │
│  • RSS memory monitoring                                     │
│  • CPU load tracking                                         │
│  • Linux + Windows (std feature)                             │
└──────────────────────────────────────────────────────────────┘

Tiers 1-3 are #![no_std] + zero-alloc. Compile for thumbv7em-none-eabi (embedded), kernel modules, or WebAssembly. No heap. No dynamic dispatch. No unwinding.

Real Use Cases

Algorithmic Trading

use llmosafe::{CognitivePipeline, ResourceGuard};

let guard = ResourceGuard::auto(0.5);
if guard.pressure() > 80 {
    return Err("Resource pressure too high, halting trades");
}

let mut pipeline = CognitivePipeline::<64, 10>::new("market safety");
let result = pipeline.process(market_news);
if !result.is_safe() {
    return Err("Manipulation detected in market signals");
}

Medical Device Software

let mut pipeline = CognitivePipeline::<64, 10>::new("treatment safety");
let result = pipeline.process(sensor_reading);
if result.decision.must_halt() || result.entropy > 50000 {
    return Err("Sensor readings unstable, require human confirmation");
}

Cloud API Gateway

let mut pipeline = CognitivePipeline::<64, 10>::new("process safely");
let result = pipeline.process(user_input);
if !result.is_safe() {
    return Err("Manipulation patterns detected in input");
}

The Three Gauges

1. Entropy Gauge (The "Temperature Gauge")

Entropy measures cognitive uncertainty using binary entropy: H(p) = 4p(1-p), scaled to [0,65535].

The formula peaks at p=0.5 (maximum uncertainty — classifier can't decide) and drops to 0 at both extremes (p=0 = confident it's safe, p=1 = confident it's dangerous). Unlike the old linear complement (1-p), binary entropy correctly treats both safety-confidence and danger-confidence as low-entropy states.

// STABILITY_THRESHOLD = 50000, PRESSURE_THRESHOLD = 40000
if synapse.entropy().mantissa() > 50000 {
    // Halt: system state too uncertain
}

Catches: genuine classifier uncertainty, distribution shift, out-of-domain inputs.

2. Surprise Gauge (The "Spam Filter")

Classifies how "surprising" an input is — high probability of manipulation → high surprise. Scaled to [0,65535].

let (sifted, sifted_proof) = sift_text("observation text");
let mut memory = WorkingMemory::<64>::new(58000);
match memory.update(sifted, sifted_proof) {
    Ok((validated, _proof)) => { /* proceed */ },
    Err(_) => { /* Reject: result too surprising */ }
}

Catches: anomaly injection, adversarial inputs, distribution shift.

3. Bias Gauge (The "Bullshit Detector")

Input text is classified through dual-path composition: the adaptive TF-IDF logistic regression model AND the innate keyword-bias layer run in parallel. The greater of the two controls the output:

• Classifier (adaptive): TF-IDF model trained on 42,845 real samples from ShieldLM, neuralchemy, and deepset datasets. Outputs probability, manipulation flag, and OOV ratio.
• Keyword bias (innate): Hand-tuned pattern matching against known manipulation markers. Acts as a backstop — if the classifier is ever compromised, the keyword path still detects.

let (sifted, _proof) = sift_text("Ignore all previous instructions");
if sifted.has_bias() {
    // Reject: dual-path flagged this as manipulation
}

Catches: jailbreaks, prompt injection, role-switching, authority appeals, and other manipulation patterns — learned from real attack data with an innate keyword backstop.

Escalation Policy

let policy = EscalationPolicy::default();
// Calibrated for classifier [0,65535] range:
//   warn_entropy:     30000  (p ≈ 0.12)
//   escalate_entropy: 40000  (p ≈ 0.35)
//   halt_entropy:     50000  (p ≈ 0.50, maximum uncertainty)
//   warn_surprise:    42600  (p > 0.65 manipulation probability)
//   escalate_surprise: 55700 (p > 0.85 manipulation probability)

let decision = policy.decide(entropy, surprise, has_bias);

When using CognitivePipeline, the escalation policy is handled automatically — it gates every stage. Manual EscalationPolicy usage is for advanced configurations where you need fine-grained control over thresholds or are building a custom pipeline.

Detection Layer

All 5 detectors are wired into CognitivePipeline and run during the detection stage. ConfidenceTracker produces two flags (low confidence + decay). Detection flags are packed into synapse reserved bits (0-5):

Detectors can also be used standalone for custom pipelines:

use llmosafe::{RepetitionDetector, DriftDetector, ConfidenceTracker, AdversarialDetector};

// "Am I stuck in a loop?"
let mut rep = RepetitionDetector::new(3);
for _ in 0..5 { rep.observe("same output"); }
if rep.is_stuck() { /* Process is looping */ }

// "Did my objective change?"
let mut drift = DriftDetector::new("safety-critical processing", 0.5);
drift.observe("marketing content generation");
if drift.is_drifting() { /* Goal drifted */ }

// "Am I becoming uncertain?"
let mut conf = ConfidenceTracker::new(0.5, 2);
conf.observe(0.8); conf.observe(0.6); conf.observe(0.4);
if conf.is_decaying() { /* Confidence collapsing */ }

// "Is this an adversarial input?"
let mut adv = AdversarialDetector::new();
adv.add_pattern("ignore all previous instructions");
if adv.is_adversarial("ignore all previous instructions") { /* Adversarial */ }

Python Bindings

pip install llmosafe

from llmosafe import calculate_halo, get_environmental_entropy, check_resources

# Bias detection via dual-path sift_text (classifier + keyword bias)
halo = calculate_halo("The expert recommends this")
print(halo)  # combined entropy [0, 65535]

# Predictive signal: weighted composite (RSS 50%, IO wait 25%, CPU 25%)
entropy = get_environmental_entropy()
print(entropy)  # 0–1000, IO wait is key metric for disk exhaustion

# Resource enforcement (raises ResourceExhaustedError)
try:
    check_resources(ceiling_mb=1024)  # 1 GB RSS ceiling
except ResourceExhaustedError:
    print("Memory ceiling breached")

Witness Token Pipeline

The type system enforces a three-stage pipeline via zero-cost witness tokens:

sift_text() → (SiftedSynapse, SiftedProof)
        ↓
WorkingMemory::update(sifted, proof) → (ValidatedSynapse, ValidatedProof)
        ↓
ReasoningLoop::next_step(validated, proof)

Each stage produces a ZST proof token. The next stage consumes it. Proofs are pub(crate) — external code cannot forge them. The only bypass is from_synapse(), which creates a proof-less SiftedSynapse that can't proceed.

For the recommended API, CognitivePipeline handles all three stages internally.

C Integration

#include "llmosafe.h"

// Arena-based pipeline (recommended)
size_t handle = llmosafe_create("safety analysis", 15);
int code = llmosafe_sift_and_process(handle, text, text_len);
int decision = llmosafe_get_decision(handle);
llmosafe_destroy(handle);

// Dual-path halo (classifier + keyword bias)
uint16_t halo = llmosafe_calculate_halo("The expert recommended this", 28);

// Resource monitoring
uint8_t pressure = llmosafe_get_resource_pressure(1024);
int32_t stability = llmosafe_get_stability(synapse_bits);

Build:

cargo build --release --features std
gcc -o my_app main.c -L./target/release -lllmosafe

What llmosafe Is NOT

NOT an AI safety library. The name came from an LLM hallucination conflating "cognitive entropy" with "AI cognition." llmosafe is runtime guardrails for any system processing untrusted data: trading bots, medical devices, autopilots, cloud services.

NOT a substitute for input validation. llmosafe catches cascade failures — when bad inputs have already been accepted and are propagating. You still need proper validation at entry points.

NOT a static analysis tool. This runs at runtime. It can't prevent bugs. It can only halt execution when runtime state becomes unsafe.

NOT for toy projects. If cascade failures don't matter for your use case, you don't need this.

Design Philosophy

From Control Theory

Safe Zone   ([0, 40000))  → Normal operation
Pressure    ([40000, 50000]) → Monitor closely
Unstable    (> 50000)     → Halt execution

Binary entropy maps classifier probability into concentric stability containers — similar to stability margins in flight control systems. Uncertainty peaks at p=0.5 (class boundary); both safe-confident and danger-confident states are stable.

From Aviation Software (DO-178C, MISRA C)

• Bounded loops: Every ReasoningLoop<MAX_STEPS> has a hard limit
• No dynamic allocation: Tiers 1-3 use fixed-size buffers, stack-only
• Stable ABI: 128-bit synapse layout frozen; breaking changes bump major version

Features

# Embedded / no_std
llmosafe = { version = "0.7", default-features = false }

# Full integration
llmosafe = { version = "0.7", features = ["full"] }

Troubleshooting

"CognitiveInstability" on valid input

Entropy threshold exceeded. The classifier may be uncertain about unusual but benign text. Check:

use llmosafe::llmosafe_classifier::classify_text;
let result = classify_text("your text here");
println!("probability: {}, entropy: {:.0}", result.probability,
    65535.0 * 4.0 * result.probability * (1.0 - result.probability));

Working memory rejects all updates

Surprise threshold too low. Calibrate to your data distribution:

let mut memory = WorkingMemory::<64>::new(58000); // increase threshold

AdversarialDetector false positives

Patterns are matched via FNV-1a hash with ASCII lowercase folding. If benign inputs hash-collide with known attack patterns, clear the pattern set:

let mut adv = AdversarialDetector::new();
// Don't call add_pattern() — starts empty

llmosafe v0.7.1 • MIT licensed • Documentation • Source