Miyagi

Miyagi is a Rust toolkit for sparse XOR adaptation of true-binary GGUF language models. It discovers supported Qwen/Bonsai Q1_0 tensors, evaluates selected token preferences, searches for reversible row-level patches, and measures the result against probes or generation datasets.

Miyagi is inspired by nikshepsvn/bankai.

Miyagi is built on the sibling wwama crate. wwama owns GGUF loading, tokenization, inference, generation, tensor descriptors, backend transfers, and validated Q1_0 row mutation. Miyagi owns the patch format, architecture mapping, probes, fitness policy, search orchestration, reports, and CLI workflows.

The Mental Model

Miyagi works with three artifacts:

• Model: a GGUF file loaded through wwama. Miyagi currently maps blk.<layer>.ffn_{gate,up,down}.weight Q1_0 tensors.
• Probe set: prompts with a correct and wrong answer string. Miyagi measures the selected-logit gap: correct_logit - wrong_logit.
• Patch: a JSON list of logical (layer, projection, row) coordinates. A patch XORs the packed Q1_0 row bytes and can be removed by applying the same coordinates again.

The normal workflow is:

inspect model -> define targets and controls -> search or load a patch
-> evaluate target and preservation behavior -> test held-out/generalization

A positive target score is only evidence for the probes that were measured. It does not prove broad knowledge, safety, generalization, or cross-format parity.

Agent Skills

The skills/ directory provides reusable Codex workflows for operating Miyagi. Each skill includes focused instructions, agent metadata, and supporting references where the workflow needs a schema, checklist, or report format.

Core workflows

• Inspect a model: verify architecture, Q1_0 tensor support, dimensions, placement, and model signature.
• Author probes: create auditable target, control, and held-out probe files with explicit token semantics.
• Evaluate a patch: validate an artifact and compare baseline versus patched probes and generation.
• Search for a patch: run deterministic, checkpointed search with explicit targets and preservation controls.
• Compose patches: combine row-XOR patches using symmetric-difference semantics and validate the result.
• Benchmark a patch: measure baseline and patched generation on JSON or JSONL datasets.

Adaptation and validation goals

• Inject verified knowledge: adapt from a sourced fact ledger without inventing answers or provenance.
• Correct known errors: repair observed factual or completion errors while testing nearby behavior.
• Preserve capabilities: define and enforce regression gates around a target adaptation.
• Test generalization: measure transfer to paraphrases, held-out facts, contexts, and datasets.
• Diagnose regressions: isolate target gains, control losses, patch interactions, and failing gates.
• Specialize a domain: design a bounded domain adaptation with adjacent-domain and general controls.
• Suppress unwanted behavior: reduce a measured response tendency while defining a replacement behavior.
• Select a patch: compare candidates under common evidence gates and choose the smallest or safest passing artifact.

Requirements

• A Rust toolchain compatible with edition 2024.
• The sibling wwama crate available at ../wwama.
• A supported Q1_0 GGUF model for mutation, search, or patch evaluation.
• Enough memory for a writable model session. Mutable tensors disable mmap in wwama and can use substantially more memory than read-only inspection.

Use the default CPU build for a first run. Forward native accelerator features with --features cuda or --features vulkan when the corresponding wwama backend is available.

Quick Start

Set a model path in the examples below:

MODEL=~/Models/llm/Bonsai-8B-gguf/Bonsai-8B-Q1_0.gguf

1. Inspect the model

Inspection is read-only and does not enable mutable tensors. Use CPU placement for a predictable capability check:

cargo run --release --no-default-features -- --json inspect \
  --model "$MODEL" \
  --n-gpu-layers 0

Check miyagi_supported, the architecture signature, layer count, mapped projection dimensions, row counts, and backend placement. Stop if the report contains an architecture error.

Use --all-tensors when diagnosing a missing mapping or backend placement:

cargo run --release --no-default-features -- inspect \
  --model "$MODEL" \
  --n-gpu-layers 0 \
  --all-tensors

2. Inspect a patch

Read a patch without loading a model:

cargo run --release --no-default-features -- --json info \
  --patch patches/calculus_v1.json

Validate coordinates and architecture identity against a live model:

cargo run --release --no-default-features -- --json info \
  --patch patches/calculus_v1.json \
  --model "$MODEL" \
  --n-gpu-layers 0

Use --allow-model-mismatch only for an explicitly bounded experiment. It allows structural validation to continue; it does not establish that a patch trained for another model or format has equivalent behavior.

3. Evaluate probe behavior

Evaluate a patch against built-in math, code, and knowledge probes:

cargo run --release --no-default-features -- --json eval \
  --model "$MODEL" \
  --n-gpu-layers 0 \
  --patch patches/calculus_v1.json \
  --probes math,code,knowledge \
  --token-mode compatibility \
  --report calculus-evaluation.json

eval measures baseline and patched gaps in one writable session, removes the patch before successful exit, and reports per-probe deltas and sign transitions. Use --token-mode strict for new probe sets when every answer must be exactly one token. Compatibility mode preserves final-token behavior for multi-token answer strings.

4. Compare generation

Compare deterministic generation without and with a patch:

cargo run --release --no-default-features -- --json apply \
  --model "$MODEL" \
  --n-gpu-layers 0 \
  --patch patches/calculus_v1.json \
  --prompt 'Explain why 7 * 8 equals 56.' \
  --max-tokens 120 \
  --seed 42

The command uses the same generation settings for both outputs and restores the model state before exit. It does not rewrite the GGUF file.

5. Search for a patch

Search uses scale-weighted candidate sampling, two-probe screening, deterministic SplitMix64 sampling, control-penalized fitness, strict-improvement acceptance, and XOR rollback for rejected candidates.

Use explicit layers rather than assuming default layers fit every model:

cargo run --release --no-default-features -- search \
  --model "$MODEL" \
  --n-gpu-layers 0 \
  --target math \
  --control code \
  --control knowledge \
  --layers 1,2,3,4 \
  --projections gate_proj,up_proj \
  --iters 200 \
  --fitness mean \
  --penalty 2.0 \
  --seed 42 \
  --screen-probes 2 \
  --checkpoint math.checkpoint.json \
  --report math.search.json \
  --output math.patch.json \
  --name math-adaptation-v1

Search writes the patch only after successful completion. Accepted flips remain applied in the current process until it exits; the model file is never changed. With --checkpoint, cancellation saves resumable state. Resume with the same model, probes, layers, projections, seed, fitness, penalty, screen count, and patch metadata; increase only the iteration ceiling:

cargo run --release --no-default-features -- search \
  --model "$MODEL" \
  --n-gpu-layers 0 \
  --target math \
  --control code \
  --control knowledge \
  --layers 1,2,3,4 \
  --projections gate_proj,up_proj \
  --iters 400 \
  --fitness mean \
  --penalty 2.0 \
  --seed 42 \
  --screen-probes 2 \
  --token-mode compatibility \
  --name math-adaptation-v1 \
  --resume math.checkpoint.json \
  --output math.patch.json

6. Compose patches

Composition uses XOR symmetric difference: coordinates present in an even number of inputs cancel, and coordinates present in an odd number remain.

cargo run --release --no-default-features -- --json compose \
  --patch first.patch.json second.patch.json \
  --name combined-v1 \
  --output combined.patch.json

The input patches must have the same base_model. Validate the result against the intended model and evaluate the combined behavior; patch interactions are not predictable from flip counts alone.

7. Benchmark generation

Miyagi accepts a JSON array or JSONL dataset. Each selected record must contain string question and answer fields. The regexes must put the value to compare in capture group 1.

cargo run --release --no-default-features -- --json benchmark \
  --model "$MODEL" \
  --n-gpu-layers 0 \
  --dataset tests/fixtures/smoke_dataset.json \
  --patch math.patch.json \
  --prompt-template "Solve the problem and end with 'The answer is [number]'. {question}" \
  --answer-regex '(?i)the answer is[:\s]*\$?([-\d,]+)' \
  --gold-regex '^([-\d,]+)$' \
  --limit 20 \
  --seed 42 \
  --report math.benchmark.json

The report includes baseline and patched per-case responses, extracted answers, accuracy counts, and model_restored. A small smoke dataset is not evidence of broad generalization or safety.

Custom Probes

Custom probe files are JSON arrays:

[
  {
    "prompt": "The capital of Australia is",
    "correct": " Canberra",
    "wrong": " Sydney",
    "name": "australia_capital",
    "category": "geography"
  }
]

Probe names must be unique. Keep leading whitespace in answer strings when it is part of the tokenizer behavior. Use separate files for target, control, and held-out probes. See the probe authoring skill for validation guidance.

Built-in selectors are math, code, and knowledge. Any selector that is not one of those names is treated as a JSON probe-file path.

Patch Format

Miyagi writes the canonical miyagi_row_xor_v1 schema:

{
  "version": 1,
  "format": "miyagi_row_xor_v1",
  "name": "example",
  "description": "Sparse row adaptation",
  "base_model": "model.gguf",
  "flips": [
    {"layer": 4, "proj": "gate_proj", "row": 123}
  ],
  "stats": {
    "n_flips": 1,
    "logical_bits_flipped": 4096,
    "compact_binary_estimate_bytes": 12
  },
  "metadata": {}
}

The reader also accepts the alternate type: "row_flip" field. Patch validation checks version, format, projection, duplicate coordinates, row bounds, Q1_0 support, and architecture signature. Logical bit counts come from the live tensor width, not a hard-coded projection size.

CLI Reference

Use --json for machine-readable reports. Search progress with --json is a stream of JSON events followed by the final result; use --report when a clean JSON artifact is needed.

Library Surface

The Rust library exposes the same building blocks for application integration:

• ArchitectureMap, Projection, and TensorInfo for live model mapping;
• WwamaBackend and MiyagiBackend for model operations;
• Patch, ValidatedPatch, and PatchFlip for patch lifecycle management;
• Probe, CompiledProbe, and ProbeMeasurement for probe evaluation;
• FitnessMode for mean or minimum target improvement; and
• SearchConfig, SearchCheckpoint, and SearchResult for deterministic search orchestration.

Build and Test

CPU build and tests:

cargo check --no-default-features
cargo test --no-default-features --all-targets

CUDA or Vulkan feature builds:

cargo check --features cuda
cargo check --features vulkan

The large-model integration tests are opt-in and require a local fixture:

MIYAGI_TEST_MODEL="$MODEL" \
MIYAGI_TEST_GPU_LAYERS=0 \
cargo test --no-default-features --test model_backend

Compatibility and Boundaries

• Miyagi supports descriptor-driven Qwen/Bonsai MLP mapping for two-dimensional Q1_0 tensors. Unsupported architectures fail with structured capability errors before mutation.
• Mutation is row-level Q1_0 XOR. Arbitrary bit, group, floating-point, and ternary mutation are outside the current contract.
• Model mutation is in memory. Miyagi never rewrites the source GGUF file.
• eval, apply, and patched benchmark restore the model before successful exit. Search intentionally returns with accepted flips applied until the process exits.
• Concurrent mutation and inference on one session is unsupported.
• Native CPU, CUDA, and Vulkan paths have been validated against local fixtures. WebAssembly compilation is possible through wwama, but mutable WebAssembly runtime behavior remains unsupported without a runtime fixture.
• Structural patch compatibility does not prove behavioral compatibility between MLX-trained artifacts and converted GGUF models. Use live evaluation and held-out tests.

Miyagi does not modify llama.cpp source files.

License

Licensed under the Apache License, Version 2.0.