Crate apple_vision 

vision

Safe Rust bindings for Apple’s Vision framework — on-device OCR, object detection, face landmarks, and other computer vision tasks on macOS.

Status: v0.16.0 keeps the full Vision request surface, adds a Tier-1 async_api module for one-shot OCR / face / barcode / segmentation workflows, and ships a fully-implemented COVERAGE.md + COVERAGE_AUDIT.md matrix plus a gold-standard multi-file Swift bridge.

Quick start — OCR

use apple_vision::prelude::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let recognizer = TextRecognizer::new()
        .with_recognition_level(RecognitionLevel::Accurate)
        .with_language_correction(true);

    let observations = recognizer.recognize_in_path("screenshot.png")?;
    for obs in &observations {
        println!("[{:.2}] '{}'", obs.confidence, obs.text);
    }
    Ok(())
}

Composes with the rest of the doom-fish stack

screencapturekit-rs / capture ──► IOSurface / PNG ──► vision ──► text
                                                          │
                                                          ▼
                                                  foundation-models
                                                  ("summarise this")

Feature flags

All request-type modules can be enabled independently, and the default feature set still enables the full Vision surface. v0.16.0 also adds an optional async feature for executor-agnostic Future wrappers around the Tier-1 one-shot request surface.

Async API

Enable async plus the request features you need:

apple-vision = { version = "0.16.0", features = ["async", "recognize_text"] }

use apple_vision::async_api::AsyncRecognizeText;
use apple_vision::RecognitionLevel;

let texts = AsyncRecognizeText::new(RecognitionLevel::Accurate, true)
    .recognize_in_path("screenshot.png")
    .await?;
println!("found {} text observations", texts.len());

Tier-1 currently covers background-queue wrappers for OCR, face detection, barcode detection, and person segmentation. Multi-fire delegate / stream-style Vision APIs remain future Tier-2 work.

Roadmap

• Single-image Vision requests (OCR, faces, landmarks, pose, contours, saliency, segmentation, Core ML, and the rest of the stateless request surface)
• Pairwise image-registration requests (VNTranslationalImageRegistrationRequest, VNHomographicImageRegistrationRequest)
• Stateful tracking requests (VNTrackObjectRequest, VNTrackRectangleRequest, VNTrackOpticalFlowRequest, VNTrackTranslationalImageRegistrationRequest, VNTrackHomographicImageRegistrationRequest)
• Header-audited request + observation coverage matrix (COVERAGE.md) with dedicated wrappers for every current request/observation type and a split Swift bridge (all bridge files stay under 500 lines)
• Explicit VNRequest / VNObservation / request-handler / VNVideoProcessor wrappers for OCR pipelines, plus base request/observation helpers reused across the rest of the crate
• Async API (Tier-1 Future wrappers for OCR, face detection, barcode detection, and person segmentation; Tier-2 stream/delegate surfaces still TBD)

License

Licensed under either of Apache-2.0 or MIT at your option.

---

API Documentation

Safe Rust bindings for Apple’s Vision framework — OCR, object detection, face landmarks, and other on-device computer vision tasks.

v0.16.0 adds a Tier-1 async_api module for one-shot OCR / face / barcode / segmentation requests while keeping the full audited Vision request surface and the split Swift bridge / coverage matrix introduced in v0.15.x.

Re-exports

pub use error::VisionError;

pub use geometry::element_type_size;

pub use geometry::image_point_for_face_landmark_point;

pub use geometry::image_point_for_normalized_point;

pub use geometry::image_point_for_normalized_point_using_region_of_interest;

pub use geometry::image_rect_for_normalized_rect;

pub use geometry::image_rect_for_normalized_rect_using_region_of_interest;

pub use geometry::normalized_face_bounding_box_point_for_landmark_point;

pub use geometry::normalized_identity_rect;

pub use geometry::normalized_point_for_image_point;

pub use geometry::normalized_point_for_image_point_using_region_of_interest;

pub use geometry::normalized_rect_for_image_rect;

pub use geometry::normalized_rect_for_image_rect_using_region_of_interest;

pub use geometry::normalized_rect_is_identity_rect;

pub use geometry::Transform3D;

pub use geometry::VisionCircle;

pub use geometry::VisionGeometryUtils;

pub use geometry::VisionPoint;

pub use geometry::VisionPoint3D;

pub use geometry::VisionVector;

pub use recognized_points::HumanBodyRecognizedPoint3D;

pub use recognized_points::RecognizedPoint;

pub use recognized_points::RecognizedPoint3D;

pub use recognized_points::RecognizedPoints3DObservation;

pub use recognized_points::RecognizedPointsObservation;

pub use recognized_points::VisionDetectedPoint;

pub use recognized_points::VisionRecognizedPoint;

pub use recognized_points::VisionRecognizedPoint3D;

pub use request_base::ImageAlignmentObservation;

pub use request_base::ImageBasedRequest;

pub use request_base::ImageRegistrationRequest;

pub use request_base::NormalizedRect;

pub use request_base::PixelBufferObservation;

pub use request_base::RequestProgress;

pub use request_base::RequestProgressHandler;

pub use request_base::RequestProgressProviding;

pub use request_base::RequestRevisionProviding;

pub use request_base::StatefulRequest;

pub use request_base::TargetedImageRequest;

pub use request_base::TrackingLevel;

pub use request_base::TrackingRequest;

pub use sdk::vision_version_number;

pub use sdk::AnimalIdentifier;

pub use sdk::BarcodeCompositeType;

pub use sdk::BarcodeSymbology;

pub use sdk::ComputeStage;

pub use sdk::ElementType;

pub use sdk::ImageCropAndScaleOption;

pub use sdk::ImageOption;

pub use sdk::PointsClassification;

pub use sdk::RecognizedPoint3DGroupKey;

pub use sdk::RecognizedPointGroupKey;

pub use sdk::VisionErrorCode;

pub use sdk::VISION_ERROR_DOMAIN;

pub use processing::ImageRequestHandler;recognize_text

pub use processing::Observation;recognize_text

pub use processing::RecognizedTextObservation;recognize_text

pub use processing::Request;recognize_text

pub use processing::RequestKind;recognize_text

pub use processing::SequenceRequestHandler;recognize_text

pub use processing::TimeRange;recognize_text

pub use processing::VideoCadence;recognize_text

pub use processing::VideoProcessingOptions;recognize_text

pub use processing::VideoProcessor;recognize_text

pub use processing::VideoProcessorCadence;recognize_text

pub use processing::VideoProcessorFrameRateCadence;recognize_text

pub use processing::VideoProcessorRequestProcessingOptions;recognize_text

pub use processing::VideoProcessorTimeIntervalCadence;recognize_text

pub use recognize_text::BoundingBox;recognize_text

pub use recognize_text::RecognitionLevel;recognize_text

pub use recognize_text::RecognizedText;recognize_text

pub use recognize_text::RecognizedTextCandidate;recognize_text

pub use recognize_text::TextRecognizer;recognize_text

pub use detect_faces::DetectedFace;detect_faces

pub use detect_faces::FaceDetector;detect_faces

pub use detect_barcodes::detect_barcodes_in_path;detect_barcodes

pub use detect_barcodes::BarcodeCompositeType as DetectedBarcodeCompositeType;detect_barcodes

pub use detect_barcodes::BarcodeSymbology as DetectedBarcodeSymbology;detect_barcodes

pub use detect_barcodes::DetectedBarcode;detect_barcodes

pub use saliency::attention_saliency_in_path;saliency

pub use saliency::SalientRegion;saliency

pub use face_landmarks::detect_face_landmarks_in_path;face_landmarks

pub use face_landmarks::FaceLandmarkRegion;face_landmarks

pub use face_landmarks::FaceLandmarkRegion2D;face_landmarks

pub use face_landmarks::FaceLandmarks;face_landmarks

pub use face_landmarks::FaceLandmarks2D;face_landmarks

pub use face_landmarks::FaceLandmarksRequest;face_landmarks

pub use face_landmarks::FaceObservationAccepting;face_landmarks

pub use face_landmarks::FaceWithLandmarks;face_landmarks

pub use face_landmarks::LandmarkPoint;face_landmarks

pub use face_landmarks::RequestFaceLandmarksConstellation;face_landmarks

pub use body_pose::detect_human_body_pose_in_path;body_pose

pub use body_pose::detect_human_body_pose_observations_in_path;body_pose

pub use body_pose::DetectedBodyPose;body_pose

pub use body_pose::HumanBodyPoseJointGroupName;body_pose

pub use body_pose::HumanBodyPoseJointName;body_pose

pub use body_pose::HumanBodyPoseObservation;body_pose

pub use body_pose::JointPoint;body_pose

pub use hand_pose::detect_human_hand_pose_in_path;hand_pose

pub use hand_pose::detect_human_hand_pose_observations_in_path;hand_pose

pub use hand_pose::DetectedHandPose;hand_pose

pub use hand_pose::HandChirality;hand_pose

pub use hand_pose::HumanHandPoseJointGroupName;hand_pose

pub use hand_pose::HumanHandPoseJointName;hand_pose

pub use hand_pose::HumanHandPoseObservation;hand_pose

pub use contours::detect_contours_in_path;contours

pub use contours::detect_contours_observation_in_path;contours

pub use contours::Contour;contours

pub use contours::ContourOptions;contours

pub use contours::ContoursObservation;contours

pub use contours::VisionContour;contours

pub use animals::recognize_animals_in_path;animals

pub use animals::AnimalIdentifier as RecognizedAnimalIdentifier;animals

pub use animals::RecognizedAnimal;animals

pub use classify::classify_image_in_path;classify

pub use classify::Classification;classify

pub use rectangles::detect_document_segmentation_in_path;rectangles

pub use rectangles::detect_rectangles_in_path;rectangles

pub use rectangles::RectangleObservation;rectangles

pub use rectangles::RectangleOptions;rectangles

pub use horizon::detect_horizon_in_path;horizon

pub use horizon::detect_horizon_observation_in_path;horizon

pub use horizon::AffineTransform;horizon

pub use horizon::HorizonObservation;horizon

pub use feature_print::generate_image_feature_print_in_path;feature_print

pub use feature_print::FeaturePrint;feature_print

pub use humans::detect_human_rectangles_in_path;humans

pub use humans::DetectedHuman;humans

pub use aesthetics::calculate_aesthetics_scores_in_path;aesthetics

pub use aesthetics::detect_face_capture_quality_in_path;aesthetics

pub use aesthetics::AestheticsScores;aesthetics

pub use aesthetics::FaceCaptureQuality;aesthetics

pub use segmentation::generate_foreground_instance_mask_in_path;segmentation

pub use segmentation::generate_foreground_instance_mask_observation_in_path;segmentation

pub use segmentation::generate_person_segmentation_in_path;segmentation

pub use segmentation::InstanceMask;segmentation

pub use segmentation::InstanceMaskObservation;segmentation

pub use segmentation::SegmentationMask;segmentation

pub use segmentation::SegmentationQuality;segmentation

pub use optical_flow::generate_optical_flow_in_paths;optical_flow

pub use optical_flow::generate_optical_flow_observation_in_paths;optical_flow

pub use optical_flow::OpticalFlowAccuracy;optical_flow

pub use coreml::coreml_classify_in_path;coreml

pub use coreml::coreml_feature_value_in_path;coreml

pub use coreml::CoreMLFeatureValue;coreml

pub use coreml::CoreMLFeatureValueObservation;coreml

pub use coreml::CoreMLImageCropAndScaleOption;coreml

pub use coreml::CoreMLModel;coreml

pub use coreml::CoreMLRequest;coreml

pub use animal_body_pose::detect_animal_body_pose;

pub use animal_body_pose::AnimalBodyPoseJointGroupName;

pub use animal_body_pose::AnimalBodyPoseJointName;

pub use animal_body_pose::AnimalJoint;

pub use human_body_pose_3d::detect_human_body_pose_3d;

pub use human_body_pose_3d::detect_human_body_pose_3d_observations;

pub use human_body_pose_3d::detect_human_body_recognized_points_3d;

pub use human_body_pose_3d::BodyHeightEstimation;

pub use human_body_pose_3d::HumanBodyPose3DJointGroupName;

pub use human_body_pose_3d::HumanBodyPose3DJointName;

pub use human_body_pose_3d::HumanBodyPose3DObservation;

pub use human_body_pose_3d::HumanBodyPose3DObservationHeightEstimation;

pub use human_body_pose_3d::HumanJoint3D;

pub use objectness_saliency::objectness_saliency;

pub use objectness_saliency::ObjectnessRegion;

pub use person_instance_mask::person_instance_mask;

pub use person_instance_mask::PersonInstanceMask;

pub use registration::register_homographic;

pub use registration::register_homographic_observation;

pub use registration::register_translational;

pub use registration::register_translational_observation;

pub use registration::HomographicAlignment;

pub use registration::TranslationalAlignment;

pub use text_rectangles::detect_text_observations;

pub use text_rectangles::detect_text_rectangles;

pub use text_rectangles::TextObservation;

pub use text_rectangles::TextRect;

pub use text_rectangles::TextRectanglesRequest;

pub use trajectories::detect_trajectories;

pub use trajectories::Trajectory;

pub use tracking::HomographicImageTracker;tracking

pub use tracking::ObjectTracker;tracking

pub use tracking::OpticalFlowFrame;tracking

pub use tracking::OpticalFlowTracker;tracking

pub use tracking::RectangleTracker;tracking

pub use tracking::TrackOpticalFlowRequestComputationAccuracy;tracking

pub use tracking::TranslationalImageTracker;tracking

Modules

aestheticsaesthetics

Aesthetics scoring (VNCalculateImageAestheticsScoresRequest) and face capture quality (VNDetectFaceCaptureQualityRequest).

animal_body_pose

VNDetectAnimalBodyPoseRequest — body-pose keypoints for cats, dogs and similar quadrupeds. Available on macOS 14+.

animalsanimals

Animal recognition (VNRecognizeAnimalsRequest).

async_apiasync

Async Vision API — Future-based wrappers for VNImageRequestHandler and friends.

body_posebody_pose

Human body pose detection (VNDetectHumanBodyPoseRequest).

classifyclassify

General-purpose image classification (VNClassifyImageRequest).

contourscontours

Edge contour detection (VNDetectContoursRequest).

coremlcoreml

CoreML inference via Vision (VNCoreMLModel, VNCoreMLRequest, and VNCoreMLFeatureValueObservation).

detect_barcodesdetect_barcodes

Barcode detection via VNDetectBarcodesRequest (Vision v0.4).

detect_facesdetect_faces

FaceDetector — wraps VNDetectFaceRectanglesRequest.

error

Errors from the Vision bridge.

face_landmarksface_landmarks

detect_face_landmarks_in_path — wraps VNDetectFaceLandmarksRequest.

feature_printfeature_print

Image feature print (VNGenerateImageFeaturePrintRequest) — semantic image embedding for content-based similarity.

ffi

Raw FFI declarations matching the Swift bridge in swift-bridge/Sources/VisionBridge/*.swift.

geometry

Geometry wrappers and utility helpers mirroring Vision’s VNGeometry* and VNUtils surfaces.

hand_posehand_pose

Human hand pose detection (VNDetectHumanHandPoseRequest).

horizonhorizon

Horizon detection (VNDetectHorizonRequest).

human_body_pose_3d

VNDetectHumanBodyPose3DRequest — 3D human-body keypoints (macOS 14+).

humanshumans

Human-rectangle detection (VNDetectHumanRectanglesRequest) — lightweight person bounding boxes without joint skeletons.

objectness_saliency

VNGenerateObjectnessBasedSaliencyImageRequest — discrete object regions an attention model thinks are salient.

optical_flowoptical_flow

Optical flow generation (VNGenerateOpticalFlowRequest).

person_instance_mask

VNGeneratePersonInstanceMaskRequest — per-person instance mask (macOS 14+).

prelude

Common imports.

processingrecognize_text

Explicit request / handler / video-processing wrappers backed by Vision.

recognize_textrecognize_text

TextRecognizer — wraps VNRecognizeTextRequest for image-file OCR.

recognized_points

Generic recognized-point wrappers shared by pose observations.

rectanglesrectangles

Rectangle + document-segmentation detection.

registration

VNTranslationalImageRegistrationRequest + VNHomographicImageRegistrationRequest — pixel-space alignment between two images.

request_base

Shared request / observation building blocks for Vision base classes.

saliencysaliency

Attention-based saliency detection via VNGenerateAttentionBasedSaliencyImageRequest.

sdk

Vision SDK-wide enums, string constants, and version helpers.

segmentationsegmentation

Segmentation mask generation — VNGeneratePersonSegmentationRequest and VNGenerateForegroundInstanceMaskRequest.

text_rectangles

VNDetectTextRectanglesRequest — text-region detection (no OCR).

trackingtracking

Stateful Vision tracking requests backed by retained Swift sessions.

trajectories

VNDetectTrajectoriesRequest — parabolic-trajectory detection.