mcv-rs

mcv-rs provides lightweight computer-vision primitives for automation loops. It does not capture screens and it does not perform input; it consumes RGBA frames from a capture backend such as msc-rs and returns coordinates or OCR results that can be passed to touch/keyboard backends.

Features

The default feature set is intentionally empty so consumers can depend on the core data types without requiring a local OpenCV or MNN toolchain. Enable only the backends you need, for example:

[dependencies]
mcv-rs = { version = "0.1", features = ["template-matching"] }

OCR is also opt-in because ocr-rs with the mnn-dynamic backend requires MNN headers/libraries at build time and model files at runtime.

Zero-configuration boundary

In other words, cargo add mcv-rs is intentionally zero-config but only exposes the core abstractions. Actual recognition backends are opt-in because they rely on native libraries.

API style

Recognition backends follow the same style as the other crates in this workspace:

• new(...) for the simplest/default constructor.
• builder(...).xxx().open() for ergonomic customization.
• with_options(...) or from_path(..., options) for explicit option structs.

You can also change process-wide per-template-type defaults once during application startup:

mcv_rs::set_default_image_threshold(0.95)?;
mcv_rs::set_default_color_threshold(0.95)?;
mcv_rs::set_default_ocr_threshold(0.80)?;
mcv_rs::set_default_ocr_case_sensitive(false);
mcv_rs::set_default_ocr_thread_count(6)?;
mcv_rs::set_default_ocr_models(mcv_rs::OcrModelFiles::new(
    "models/my-ocr-model",
    "detector.mnn",
    "recognizer.mnn",
    "characters.txt",
))?;

// Uses the new default threshold.
let image_template = mcv_rs::ImageTemplate::new("button.png")?;
let color_template = mcv_rs::MultiColorTemplate::new(
    "#FF5500",
    vec![mcv_rs::ColorOffset::new(10, 5, "#FFAA00")],
)?;
# Ok::<(), mcv_rs::McvError>(())

Explicit options and builder overrides still win over process-wide defaults. OCR model files remain explicit; threshold, case-sensitivity, and thread count can be configured globally.

Example

use mcv_rs::{ImageTemplate, RgbaFrame, VisionTemplate};

fn find_button(width: u32, height: u32, rgba: &[u8]) -> mcv_rs::Result<Option<(i32, i32)>> {
    let mut frame = RgbaFrame::new(width, height, rgba)?;
    let mut template = ImageTemplate::new("button.png")?;
    Ok(template.find(&mut frame)?.map(|result| result.center()))
}

Use find_with_roi when the same template should be reused in different regions:

use mcv_rs::{ImageTemplate, RgbaFrame, Roi, VisionTemplate};

fn find_button_in_region(
    width: u32,
    height: u32,
    rgba: &[u8],
    roi: Roi,
) -> mcv_rs::Result<Option<(i32, i32)>> {
    let mut frame = RgbaFrame::new(width, height, rgba)?;
    let mut template = ImageTemplate::new("button.png")?;
    Ok(template
        .find_with_roi(&mut frame, Some(roi))?
        .map(|result| result.center()))
}

Builder example:

use mcv_rs::{ImageTemplate, Roi};

let template = ImageTemplate::builder("button.png")
    .threshold(0.86)?
    .roi(Roi { x: 0, y: 0, width: 1280, height: 720 })
    .open()?;
# Ok::<(), mcv_rs::McvError>(())

Multi-point color example:

use mcv_rs::{ColorOffset, MultiColorTemplate, RgbaFrame, VisionTemplate};

fn find_color_pattern(
    width: u32,
    height: u32,
    rgba: &[u8],
) -> mcv_rs::Result<Option<(i32, i32)>> {
    let mut frame = RgbaFrame::new(width, height, rgba)?;

    // The first color is the anchor point returned by `find`.
    // Every `ColorOffset` is matched relative to that anchor.
    let offsets = vec![
        ColorOffset::new(10, 5, "#2E942E"),
        ColorOffset::new(20, 5, "#3EA43E"),
    ];
    let mut template = MultiColorTemplate::new("#1E841E", offsets)?
        .with_threshold(0.98)?;

    let left_panel = mcv_rs::Roi {
        x: 0,
        y: 0,
        width: 640,
        height: 720,
    };
    Ok(template
        .find_with_roi(&mut frame, Some(left_panel))?
        .map(|result| result.center()))
}

After the one-time OCR configuration above, creating a text template only requires the text:

use mcv_rs::OcrTemplate;

fn find_start_text() -> mcv_rs::Result<Option<(i32, i32)>> {
    let mut template = OcrTemplate::new("Start")?;

    Ok(template.find_path("screen.png")?.map(|result| result.center))
}

For an in-memory capture frame, find is available directly on OcrTemplate; importing VisionTemplate is not required:

let mut frame = mcv_rs::RgbaFrame::new(width, height, rgba)?;
let result = template.find(&mut frame)?;
# Ok::<(), mcv_rs::McvError>(())

Use OcrTemplate::builder(models) for per-template overrides. Serialized or fully explicit configurations remain available through OcrTemplate::with_options(OcrFindOptions { ... }).

OCR build requirements

When enabling ocr or full, provide MNN locations for the ocr-rs mnn-dynamic backend:

$env:MNN_LIB_DIR = "C:\path\to\mnn\lib"
$env:MNN_INCLUDE_DIR = "C:\path\to\mnn\include"
cargo check -p mcv-rs --features ocr

At runtime, provide OCR model files through OcrModelFiles. On Windows, set MNN_DLL_PATH to an explicitly trusted MNN.dll when the DLL is not already available through the normal Windows loader search path. The runtime library is never inferred from a caller-provided OCR model directory.