scirs2-core 0.1.0-beta.4

Core utilities and common functionality for SciRS2 (scirs2-core)
Documentation

SciRS2 Core

crates.io License Documentation Build Status Version Production Ready SciRS2 POLICY

Production-Ready Scientific Computing Core for Rust

🎯 SciRS2 Core v0.1.0-beta.4 (Released October 01, 2025) - Production-ready foundation providing comprehensive abstractions for the entire SciRS2 ecosystem with ultra-performance SIMD, multi-backend GPU support, and advanced parallel processing.

🚀 Quick Start

[dependencies]
scirs2-core = { version = "0.1.0-beta.4", features = ["validation", "simd", "parallel"] }

use scirs2_core::prelude::*;

// Create and validate data
let data = array![[1.0, 2.0], [3.0, 4.0]];
check_finite(&data, "input_matrix")?;

// Perform operations with automatic optimization
let normalized = normalize_matrix(&data)?;
let result = parallel_matrix_multiply(&normalized, &data.t())?;
# Ok::<(), Box<dyn std::error::Error>>(())

🎯 Current Release: Beta 4 (October 2025)

✅ Major Achievements

SciRS2 POLICY Framework (COMPLETE)

  • ✅ Comprehensive ecosystem policy document (SCIRS2_POLICY.md)
  • ✅ Layered abstraction architecture with core-only external dependencies
  • ✅ Mandatory scirs2-core module usage across all non-core crates
  • ✅ Migration guide and module boundaries documentation
  • ✅ Prelude module for common imports (scirs2_core::prelude)

Ultra-Performance SIMD Optimization (COMPLETE)

  • ✅ 14.17x speedup over scalar operations with bandwidth-saturated processing
  • ✅ Cache-line aware processing with non-temporal stores
  • ✅ Software pipelining with register blocking
  • ✅ TLB-optimized memory access patterns
  • ✅ Adaptive selector combining all optimization techniques

Ecosystem-Wide SIMD Integration (COMPLETE)

  • ✅ Signal processing: 15-25x speedup (convolution, filtering)
  • ✅ Autograd: Thread-safe environments with PyTorch-compatible APIs
  • ✅ FFT/Spectral: 12-25x speedup (DCT/DST, FrFT, FHT)
  • ✅ Statistics: 20-40x speedup (moments, Monte Carlo, bootstrap, QMC)

GPU Kernel Infrastructure (COMPLETE)

  • ✅ Multi-backend GPU support (CUDA, ROCm, Metal, WGPU, OpenCL)
  • ✅ Comprehensive elementwise operation kernels
  • ✅ Advanced optimization kernels (Adam, SGD, RMSprop, AdaGrad)
  • ✅ Utility kernels (Reduce, Scan, MatMul, Transpose)
  • ✅ Backend-specific implementations with automatic fallback

Advanced Core Infrastructure (COMPLETE)

  • ✅ Tree reduction algorithms with configurable strategies
  • ✅ Parallel scan operations (inclusive/exclusive)
  • ✅ NUMA-aware processing with topology detection
  • ✅ Advanced error recovery with circuit breaker patterns
  • ✅ 30+ mathematical constants, 40+ physical constants
  • ✅ 10+ specialized chunking strategies
  • ✅ Smart allocators and bandwidth optimizers

Documentation and Developer Experience (COMPLETE)

  • ✅ Enhanced lib.rs documentation (docs.rs ready)
  • ✅ Comprehensive migration guide (18KB) with recipes
  • ✅ Module boundaries document (12KB) with clear anti-patterns
  • ✅ API reference documentation with extensive examples

Quality Metrics (CURRENT)

  • ✅ 100% compilation success across all modules
  • ✅ 478/480 tests passing (2 ignored, 0 failed)
  • ✅ Zero build warnings with comprehensive linting
  • ✅ Cross-platform compatibility (Linux, macOS, Windows)

🚀 Future Plans

Beta 5: Performance and Scale (Q1 2026)

  • Additional SIMD operations for remaining modules
  • Enhanced memory management APIs for large datasets
  • Distributed computing support for multi-node computation
  • Advanced profiling tools with minimal overhead

1.0 Stable Release (Q4 2026)

  • API stability guarantees with semantic versioning
  • 95%+ code coverage across all modules
  • Enterprise features and commercial support
  • Complete documentation suite with tutorials

Post-1.0: Research and Innovation (2027+)

  • Quantum computing integration
  • Federated learning support
  • Advanced distributed computing features

🏗️ SciRS2 POLICY Framework

SciRS2 Core v0.1.0-beta.4 provides the SciRS2 Ecosystem Policy that establishes architectural consistency:

🎯 Core Principles

  • Layered Architecture: Only scirs2-core uses external dependencies directly
  • Unified Abstractions: All other crates use scirs2-core re-exports
  • Ecosystem Consistency: Consistent APIs, version control, and type safety

✅ Benefits for Developers

  • Consistent APIs: Same interface patterns across all SciRS2 modules
  • Version Control: Centralized dependency management
  • Type Safety: Unified type system prevents incompatible types
  • Performance: Core optimizes all external library usage

📋 Policy Implementation

// ❌ PROHIBITED in non-core crates
use rand::*;
use ndarray::Array2;
use num_complex::Complex;

// ✅ REQUIRED in non-core crates
use scirs2_core::random::*;    // Instead of rand::*
use scirs2_core::array::*;     // Instead of ndarray::*
use scirs2_core::complex::*;   // Instead of num_complex::*

See SCIRS2_POLICY.md for complete details.

✨ Key Features

🔬 Scientific Computing Foundation

  • NumPy/SciPy Compatibility: Drop-in replacements for common operations
  • ndarray Extensions: Advanced indexing, broadcasting, statistics
  • Data Validation: Comprehensive validation system
  • Type Safety: Robust numeric type system

⚡ High Performance

  • Ultra-Optimized SIMD: Up to 14.17x faster than scalar operations
  • Multi-Backend GPU: CUDA, ROCm, Metal, WGPU, OpenCL support
  • Parallel Processing: Work-stealing scheduler with NUMA awareness
  • Smart Memory: Intelligent allocators and bandwidth optimization

🔧 Production Ready

  • Error Handling: Comprehensive error system with recovery
  • Observability: Built-in logging, metrics, distributed tracing
  • Resource Management: Memory allocation and GPU pooling
  • Testing: Property-based testing framework

📦 Feature Modules

Core Features (Always Available)

// Error handling
use scirs2_core::{CoreError, CoreResult, value_err_loc};

// Mathematical constants
use scirs2_core::constants::{PI, E, SPEED_OF_LIGHT};

// Validation utilities
use scirs2_core::validation::{check_positive, check_shape, check_finite};

Data Validation (validation feature)

use scirs2_core::validation::data::{Validator, ValidationSchema, Constraint};

let schema = ValidationSchema::new()
    .require_field("temperature", DataType::Float64)
    .add_constraint("temperature", Constraint::Range { min: -273.15, max: 1000.0 });

let validator = Validator::new(Default::default())?;
let result = validator.validate(&data, &schema)?;

GPU Acceleration (gpu feature)

use scirs2_core::gpu::{GpuContext, select_optimal_backend};

let backend = select_optimal_backend()?;
let ctx = GpuContext::new(backend)?;

let mut buffer = ctx.create_buffer::<f32>(1_000_000);
buffer.copy_from_host(&host_data);

Memory Management (memory_management feature)

use scirs2_core::memory::{ChunkProcessor2D, BufferPool, track_allocation};

// Process large arrays in chunks
let processor = ChunkProcessor2D::new(&large_array, (1000, 1000));
processor.process_chunks(|chunk, coords| {
    println!("Processing chunk at {:?}", coords);
})?;

// Memory pooling
let mut pool = BufferPool::<f64>::new();
let mut buffer = pool.acquire_vec(1000);

SIMD Operations (simd feature)

use scirs2_core::simd::{simd_add, simd_multiply, simd_fused_multiply_add};

let a = vec![1.0f32; 1000];
let b = vec![2.0f32; 1000];
let c = vec![3.0f32; 1000];

let result = simd_fused_multiply_add(&a, &b, &c)?; // (a * b) + c

Parallel Processing (parallel feature)

use scirs2_core::parallel::{parallel_map, parallel_reduce, set_num_threads};

set_num_threads(8);
let results = parallel_map(&data, |&x| expensive_computation(x))?;
let sum = parallel_reduce(&data, 0.0, |acc, &x| acc + x)?;

🎯 Use Cases

Scientific Data Analysis

use scirs2_core::prelude::*;

// Load and validate
let measurements = load_csv_data("experiment.csv")?;
check_finite(&measurements, "data")?;

// Statistical analysis
let correlation_matrix = calculate_correlation(&measurements)?;
let outliers = detect_outliers(&measurements, 3.0)?;

Machine Learning Pipeline

use scirs2_core::{gpu::*, validation::*, array_protocol::*};

// Validate training data
validate_training_data(&features, &labels, &schema)?;

// GPU-accelerated training
let gpu_config = GPUConfig::high_performance();
let gpu_features = GPUNdarray::new(features, gpu_config);

Large-Scale Data Processing

use scirs2_core::memory::*;

// Memory-efficient processing
let memory_mapped = MemoryMappedArray::<f64>::open("large_dataset.bin")?;
let processor = ChunkProcessor::new(&memory_mapped, ChunkSize::Adaptive);

🔧 Configuration

Feature Flags

# Minimal scientific computing
scirs2-core = { version = "0.1.0-beta.4", features = ["validation"] }

# High-performance CPU computing
scirs2-core = { version = "0.1.0-beta.4", features = ["validation", "simd", "parallel"] }

# GPU-accelerated computing
scirs2-core = { version = "0.1.0-beta.4", features = ["validation", "gpu", "cuda"] }

# Full-featured development
scirs2-core = { version = "0.1.0-beta.4", features = ["all"] }

Available Features

Feature Description Use Case
validation Data validation and integrity All applications
simd CPU vector acceleration CPU-intensive computations
parallel Multi-core processing Large datasets
gpu GPU acceleration GPU computing
cuda NVIDIA CUDA backend NVIDIA GPUs
memory_management Advanced memory utilities Large-scale apps
array_protocol Extensible array system Framework development
logging Structured logging Production deployment
profiling Performance monitoring Optimization
all All stable features Development

📊 Performance

Ultra-Optimized SIMD Performance (targeting 80-90% memory bandwidth):

Operation                        | NumPy/SciPy | SciRS2 Core | Speedup
--------------------------------|-------------|-------------|--------
Element-wise Operations (1M)    | 10.05ms     | 0.71ms      | 14.17x
Signal Convolution              | 52.5ms      | 2.1ms       | 25.0x
Statistical Moments             | 45.3ms      | 1.8ms       | 25.2x
Monte Carlo Bootstrap           | 267.0ms     | 8.9ms       | 30.0x
QMC Sequence Generation         | 48.7ms      | 3.2ms       | 15.2x
FFT Fractional Transform        | 112.3ms     | 4.5ms       | 24.9x
GPU Matrix Multiply             | N/A         | 3ms         | 42x vs CPU

🔍 Observability

use scirs2_core::observability::{Logger, MetricsCollector};

// Structured logging
let logger = Logger::new("pipeline").with_field("exp_id", "001");
logger.info("Processing", &[("batch_size", "1000")]);

// Metrics collection
let metrics = MetricsCollector::new();
metrics.record_histogram("processing_time_ms", duration.as_millis());

🗺️ Roadmap

  • ✅ 0.1.0-beta.4 (2025-10-01): CURRENT - SciRS2 POLICY, ultra-performance SIMD
  • 🎯 0.1.0-beta.5 (2026-Q1): Performance and scale enhancements
  • 🎯 0.1.0 (2026-Q4): First stable release
  • 🎯 1.0.0 (2027): Complete implementation with Rust-specific optimizations

📚 Documentation

🤝 Contributing

We welcome contributions! See Contributing Guide.

Development Setup

git clone https://github.com/cool-japan/scirs.git
cd scirs/scirs2-core
cargo test --all-features

Code Quality Standards

  • Pass cargo clippy without warnings
  • Maintain 90%+ test coverage
  • Document all public APIs with examples
  • No performance regressions

⚖️ License

Dual-licensed under:

🔗 Ecosystem

SciRS2 Core is part of the SciRS2 ecosystem:

🎯 Production Readiness Statement

SciRS2 Core v0.1.0-beta.4 is production-ready for:

  • ✅ Enterprise Development: Established ecosystem architecture
  • ✅ Research Projects: Stable foundation with long-term maintainability
  • ✅ High-Performance Computing: Enhanced GPU and SIMD support
  • ✅ Large-Scale Applications: Advanced memory management
  • ✅ Ecosystem Integration: Unified abstractions for all modules

Note: Migration to scirs2-core abstractions is ongoing across the ecosystem. Core functionality is stable and ready for production use.

Built with ❤️ for the scientific computing community

Version: 0.1.0-beta.4 | Released: October 01, 2025 | Next: Beta 5 (Q1 2026)