num-valid 0.3.3

# Copilot Instructions for num-valid

## Project Overview

`num-valid` is a Rust library for robust numerical computation using validated types to prevent floating-point errors like NaN propagation. It provides a generic, layered architecture supporting both native f64 and arbitrary-precision arithmetic (via the `rug` crate).

**Key Concept:** Use validated wrapper types (like `RealNative64StrictFinite`) instead of raw primitives to enforce correctness at the type level.

## Critical Requirements

### Rust Nightly Toolchain
- **Required:** This project uses unstable Rust features (`trait_alias`, `error_generic_member_access`)
- The toolchain is configured in `rust-toolchain.toml` with `channel = "nightly"`
- Never suggest stable-only solutions - assume nightly features are available

### Feature Flags
- **Default:** Native f64 backend with strict validation
- **`rug`:** Enables arbitrary-precision arithmetic (LGPL-3.0 licensed dependency)
- **`backtrace`:** Enables backtrace capture in errors (disabled by default for performance)

## Architecture: 4-Layer Design

Understanding this layered architecture is essential for making changes.

**📚 For comprehensive details, see [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md)** - a complete 25+ page guide covering:
- Detailed explanation of each layer with examples and rationale
- Step-by-step guides for adding backends and mathematical functions
- Error handling architecture and performance considerations
- Design patterns reference and macro system documentation

**Quick Reference:**

### Layer 1: Raw Trait Contracts
**Files:** `src/core/traits/raw.rs`, `src/backends/native64/raw.rs`, `src/backends/rug/raw.rs`

- **Traits:** `RawScalarTrait`, `RawRealTrait`, `RawComplexTrait`
- **Purpose:** Low-level, unchecked operations (e.g., `unchecked_sqrt`, `unchecked_reciprocal`)
- **Contract:** Caller guarantees input validity - no validation performed
- **Key Pattern:** All methods prefixed with `unchecked_*` assume valid inputs
- **Implementations:** `f64`, `Complex<f64>`, `rug::Float`, `rug::Complex`

### Layer 2: Validation Policies
**Files:** `src/core/policies.rs`, `src/core/traits/validation.rs`, `src/core/traits.rs`

- **Core traits:** `RawKernel` and `NumKernel` (in `src/core/traits.rs`) - bundle raw types with validation policies
- **Kernel structs:** `NumKernelStrictFinite`, `NumKernelStrictFiniteInDebug` (in `src/kernels.rs`)
- **Primary policy:** `StrictFinitePolicy` - ensures values are finite and not subnormal
- **Alternative:** `DebugValidationPolicy<P>` - validates only in debug builds
- **Type-level guarantees:** `GuaranteesFiniteRealValues` marker enables `Eq` + `Hash` for HashMap keys

### Layer 3: Validated Wrappers
**Files:** `src/core/types.rs`

- **Types:** `RealValidated<K>`, `ComplexValidated<K>` (generic over `NumKernel`)
- **Concrete types:** `RealNative64StrictFinite`, `ComplexNative64StrictFinite`, etc.
- **Pattern:** Validate input → call `unchecked_*` method → validate output
- **Copy semantics:** Automatically `Copy` when underlying raw type is `Copy`
- **Defined via:** Modular macros (`define_validated_struct_type!`, `impl_validated_*!`)

### Layer 4: High-Level Traits
**Files:** `src/lib.rs`

- **Core trait:** `FpScalar` - universal interface for all scalar types
- **Specialized:** `RealScalar` (for real numbers), `ComplexScalar` (for complex)
- **Mutual exclusion:** Associated type `Kind` (sealed) prevents a type being both real and complex
- **Usage:** Write generic code using `T: RealScalar` bounds

## Project Structure

```
src/
├── lib.rs                    # Main entry point, FpScalar/RealScalar/ComplexScalar traits
├── kernels.rs                # NumKernelStrictFinite, NumKernelStrictFiniteInDebug structs
├── macros.rs                 # Modular macro system (real!, complex!, validated struct macros)
├── scalars.rs                # Constrained types: AbsoluteTolerance, PositiveRealScalar, etc.
├── prelude.rs                # Common re-exports
├── core/
│   ├── traits.rs             # RawKernel, NumKernel traits
│   ├── traits/
│   │   ├── raw.rs            # RawScalarTrait, RawRealTrait, RawComplexTrait
│   │   └── validation.rs     # ValidationPolicyReal/Complex, GuaranteesFinite* markers
│   ├── policies.rs           # StrictFinitePolicy, DebugValidationPolicy
│   ├── types.rs              # RealValidated<K>, ComplexValidated<K> (2800+ lines)
│   └── errors.rs             # Error types, capture_backtrace() function
├── backends/
│   ├── native64/
│   │   ├── raw.rs            # f64/Complex<f64> raw trait implementations
│   │   └── validated.rs      # Native64StrictFinite, concrete type aliases
│   └── rug/                  # (feature-gated)
│       ├── raw.rs            # rug::Float/Complex raw trait implementations
│       └── validated.rs      # RugStrictFinite, concrete type aliases
├── functions/                # Mathematical function traits
│   ├── sqrt.rs, trigonometric.rs, exponential.rs, logarithm.rs, ...
│   └── function_docs.rs      # Shared documentation
├── algorithms/
│   └── neumaier_sum.rs       # Compensated summation algorithm
└── functions.rs              # Re-exports all function traits
```

## Common Development Patterns

### Adding Mathematical Functions
1. Define trait in `src/functions/<category>.rs` (e.g., `trigonometric.rs`)
2. Implement for raw types (`f64`, `Complex<f64>`, rug types if needed)
3. Implement for validated wrappers in `src/core/types.rs` using `#[duplicate_item]` macro
4. Provide both panicking (`fn sqrt()`) and fallible (`fn try_sqrt()`) variants
5. Define specific error types for the function's domain (e.g., `SqrtRealInputErrors`)

### Macro Usage Pattern
The codebase heavily uses the `duplicate` crate for code generation:
```rust
#[duplicate_item(
    kernel_type;
    [Native64StrictFinite];
    [Native64StrictFiniteInDebug];
)]
impl MyTrait for RealValidated<kernel_type> {
    // implementation
}
```
This reduces duplication across validation policies and backends.

### Error Handling Convention
- **Input errors:** Issues with function arguments (e.g., negative value for sqrt)
- **Output errors:** Results that violate validation policy (e.g., operation produces NaN)
- All error types derive from `thiserror::Error`
- Backtraces captured only when `backtrace` feature enabled
- Use `capture_backtrace()` from `src/core/errors.rs`

### Error Handling Patterns
```rust
use num_valid::{RealNative64StrictFinite, RealScalar, functions::Sqrt};

// Pattern 1: Propagate errors with ?
fn compute_radius(area: f64) -> Result<RealNative64StrictFinite, Box<dyn std::error::Error>> {
    let validated_area = RealNative64StrictFinite::try_from_f64(area)?;
    let pi = RealNative64StrictFinite::try_from_f64(std::f64::consts::PI)?;
    let radius_squared = validated_area / pi;
    Ok(radius_squared.try_sqrt()?)
}

// Pattern 2: Match on specific error variants
use num_valid::functions::{SqrtRealErrors, SqrtRealInputErrors};

fn sqrt_with_fallback(x: RealNative64StrictFinite) -> RealNative64StrictFinite {
    match x.try_sqrt() {
        Ok(result) => result,
        Err(SqrtRealErrors::Input { source: SqrtRealInputErrors::NegativeValue { .. } }) => {
            // Handle negative input: return sqrt(|x|)
            (-x).sqrt()
        }
        Err(e) => panic!("Unexpected error: {e}"),
    }
}
```

### Testing Strategy
- Tests organized in module-level `mod tests` blocks
- Feature-gated tests for `rug` backend: `#[cfg(feature = "rug")]`
- Generic test functions parameterized by scalar type (e.g., `test_add::<T: FpScalar>()`)
- Run tests with: `cargo test --features=rug`
- For panicking constructors, use `#[should_panic(expected = "...")]` attribute

### API Design Patterns
- **Fallible + Panicking Pairs**: Provide both `try_*()` and panicking variants
  - Example: `try_from_f64()` (returns `Result`) + `from_f64()` (panics on error)
  - Panicking versions useful for constants and tests where validity is guaranteed
  - Document panic conditions clearly in docstrings

### Naming Conventions
- **`kernel_` Prefix Policy**: Reserved for primitive type operations that manipulate underlying type properties and could conflict with inherent methods:

  | Category | WITH `kernel_` prefix | WITHOUT prefix |
  |----------|----------------------|----------------|
  | Rounding | `kernel_ceil`, `kernel_floor`, `kernel_round`, `kernel_trunc`, `kernel_fract` | - |
  | Sign | `kernel_copysign`, `kernel_signum`, `kernel_is_sign_positive`, `kernel_is_sign_negative` | - |
  | Math | - | `sin`, `cos`, `sqrt`, `exp`, `ln`, `hypot`, `clamp`, `classify`, `exp_m1`, `ln_1p` |

  - **Rationale**: The prefix prevents method resolution ambiguity when calling trait methods on types that have inherent methods with the same name.
  - **UFCS in Doctests**: When trait methods have same names as inherent methods (e.g., `clamp`, `classify`), use fully qualified syntax:
    ```rust
    // Instead of: value.clamp(&min, &max)  // May call f64::clamp
    Clamp::clamp(value, &min, &max)          // Always calls trait method
    ```

### Performance Optimization Pattern (rug backend)
- **Use MPFR precomputed constants** when available instead of calculations
  - Example: `rug::float::Constant::Pi` (~10x faster than `acos(-1)`)
  - Example: `rug::float::Constant::Log2` (~10x faster than `ln(2)`)
  - Available: `Pi`, `Log2`, `Euler`, `Catalan`
  - Not available: `E` (must use `exp(1)`)
- **Cannot use LazyLock caching** inside `impl<const PRECISION>` (Rust limitation)
- Raw functions (`raw_pi`, `raw_ln_2`) should use MPFR constants directly

## Build & Test Commands

```bash
# Format code (always run before committing)
cargo fmt

# Run tests for native backend
cargo test

# Run tests with rug (arbitrary-precision)
cargo test --features=rug

# Run benchmarks
cargo bench

# Generate documentation with rug feature
cargo doc --features=rug --open

# Check coverage (requires tarpaulin)
cargo tarpaulin --config tarpaulin.toml
```

## Type System Conventions

### Naming Patterns
- `Real*` - Real number types
- `Complex*` - Complex number types  
- `*Native64*` - Uses f64 backend
- `*Rug*` - Uses arbitrary-precision backend
- `*StrictFinite` - Always validates (debug + release)
- `*StrictFiniteInDebug` - Validates only in debug mode
- `Raw*Trait` - Layer 1 unchecked operations
- `*Validated` - Layer 3 validated wrappers

### Associated Types Pattern
- `RealType` - The real component type (for complex scalars)
- `RawReal` / `RawComplex` - Underlying raw type
- `Kind` - Sealed type for real/complex distinction

## Special Considerations

### Precision in rug Backend
- Rug types use const generic precision: `RealRugStrictFinite<100>` (100-bit precision)
- Always validate f64→rug conversion for exact representability
- Non-exact f64 representations trigger `ErrorsTryFromf64::NonRepresentableExactly`

### Bytemuck Integration
- `RealNative64StrictFinite` implements `CheckedBitPattern` + `NoUninit`
- Enables zero-copy conversions from byte arrays
- Invalid bit patterns (NaN, infinity, subnormals) rejected during conversion

### Reference vs. Value Semantics
- Prefer reference-based operations to avoid cloning arbitrary-precision types
- All arithmetic ops support: `T op T`, `T op &T`, `&T op T`, `&T op &T`
- Max/Min use reference-based API (not `Ord` trait) for efficiency

### Compensated Summation
- Default `Sum` implementation uses Neumaier algorithm for accuracy
- Example: `data.iter().sum::<RealNative64StrictFinite>()` uses compensated sum
- See `src/algorithms/neumaier_sum.rs`

## Documentation Standards

- All public items require doc comments with examples
- Use intra-doc links: `[`RealScalar`]` instead of plain text
- Include error scenarios in function docs
- Document validation behavior (when checks occur)
- Mark unstable features with clear warnings

## Precision Requirements (rug Backend)

### Choosing Precision Values
- **Precision is bits in significand**, not decimal digits
- Common values: `53` (f64 equivalent), `100`, `200`, `500`, `1000`
- Precision conversion: `decimal_digits ≈ precision * log10(2) ≈ precision * 0.301`
- Example: 100-bit precision ≈ 30 decimal digits

### Type Declaration Pattern
```rust
// Define precision as const generic
type MyReal = RealRugStrictFinite<100>;  // 100-bit precision
type MyComplex = ComplexRugStrictFinite<100>;

// Precision must match across compatible types
let a = MyReal::try_from_f64(1.5)?;  // OK: validates exact representation
let b = MyReal::try_from_f64(0.1)?;  // Error: 0.1 not exactly representable
```

### Precision Validation Rules
- `try_from_f64()` requires **exact representation** at target precision
- Non-exact conversions trigger `ErrorsTryFromf64::NonRepresentableExactly` (e.g., `0.1` cannot be exactly represented)
- Precision mismatches between `rug::Float` values and expected kernel precision trigger `ErrorsValidationRawReal::PrecisionMismatch`
- Precision mismatches between types trigger compile-time errors
- Operations preserve precision: `RealRugStrictFinite<100> + RealRugStrictFinite<100> = RealRugStrictFinite<100>`

### Performance vs. Precision Trade-offs
- **53 bits:** Native f64 precision, minimal overhead
- **100-200 bits:** Good balance for most scientific computing
- **500+ bits:** High precision but noticeable performance impact
- **1000+ bits:** Use only when mathematically required

## CI/CD and GitLab Pipeline

### Pipeline Structure
The project uses GitLab CI with comprehensive test coverage:
- **4 build configurations:** Debug/Release × with/without rug
- **Test matrix:** All feature combinations tested separately
- **Coverage reporting:** `tarpaulin` with 4 configurations (see `tarpaulin.toml`)
- **Security:** `cargo-audit` runs on schedule
- **Formatting/Linting:** `cargo fmt` and `clippy` checks

### Running Coverage Locally
```bash
# Install tarpaulin
cargo install cargo-tarpaulin

# Run specific configuration from tarpaulin.toml
cargo tarpaulin --config debug_no_rug
cargo tarpaulin --config release_rug

# Generate HTML report
cargo tarpaulin --out Html --output-dir coverage/
```

### CI Requirements
- **Nightly toolchain:** Pipeline uses `rustlang/rust:nightly` Docker image
- **System dependencies:** libgmp-dev, libmpfr-dev, libmpc-dev (for rug)
- **Coverage threshold:** Aim for ≥90% (green badge), minimum 75% (yellow)

### Badge Configuration
Update README badges with your GitLab username:
```markdown
[![Pipeline Status](https://gitlab.com/YOUR_USERNAME/num-valid/badges/master/pipeline.svg)]
[![Coverage](https://gitlab.com/YOUR_USERNAME/num-valid/badges/master/coverage.svg)]
```

Enable coverage parsing in GitLab Settings → CI/CD with regex: `Coverage: (\d+\.?\d*)%`

## Benchmarking and Performance

### Running Benchmarks
```bash
# Run all benchmarks
cargo bench

# Run specific benchmark
cargo bench --bench performance_benchmark

# Generate detailed reports (HTML output in target/criterion/)
cargo bench -- --verbose
```

### Benchmark Organization
- Located in `benches/performance_benchmark.rs`
- Uses `criterion` crate for statistical analysis
- Test vector size: 10,000 elements (adjust `VECTOR_SIZE` constant for different scales)

### Performance Expectations

#### Native64 Backend (f64)
- **Validated types overhead:** ~5-15% compared to raw f64 (Release mode)
- **StrictFinite vs StrictFiniteInDebug:** Identical performance in Release builds
- **Compensated summation:** ~2-3x slower than naive sum, but numerically superior

#### Rug Backend (arbitrary-precision)
- **Precision impact:** Linear scaling with bit width for most operations
- **100-bit precision:** ~10-50x slower than f64 depending on operation
- **Memory usage:** Increases with precision (approximately `precision/8` bytes per value)

### Optimization Checklist
- Use `StrictFiniteInDebug` for performance-critical Release builds
- Prefer `&T` (references) over `T` (values) for rug types to avoid clones
- Use `mul_add_ref()` instead of separate multiply and add
- Consider `fold()` instead of `sum()` when accuracy is less critical than speed

## Contributing and Release Workflow

### Version Management
- **Versioning:** Follows [Semantic Versioning](https://semver.org/) (MAJOR.MINOR.PATCH)
- **Release tool:** Uses `cargo-release` configured in `Release.toml`
- **Protected branch:** Releases only from `master` branch
- **Changelog:** Maintained manually in `CHANGELOG.md` following [Keep a Changelog](https://keepachangelog.com/)

### Release Process
```bash
# 1. Update CHANGELOG.md with new version section
# Move items from [Unreleased] to [X.Y.Z] - YYYY-MM-DD

# 2. Run cargo-release (dry-run first)
cargo release patch --dry-run   # For bug fixes (0.2.4 -> 0.2.5)
cargo release minor --dry-run   # For new features (0.2.4 -> 0.3.0)
cargo release major --dry-run   # For breaking changes (0.2.4 -> 1.0.0)

# 3. Execute release (signs commits and tags)
cargo release patch --execute

# 4. Verify GitLab pipeline passes
# 5. Manually publish to crates.io (publish = false in Release.toml)
cargo publish
```

### Commit and Change Conventions

#### Changelog Categories
- **Added:** New features, traits, functions, or types
- **Changed:** Modifications to existing functionality
- **Fixed:** Bug fixes
- **Removed:** Deprecated/removed features

#### Feature Flag Changes
When adding/modifying features:
1. Update `Cargo.toml` feature flags
2. Add feature-gated tests: `#[cfg(feature = "new_feature")]`
3. Update documentation in both README.md and lib.rs
4. Add examples demonstrating the feature
5. Update CHANGELOG.md under "Added" or "Changed"

#### Breaking Changes
- Increment MAJOR version
- Clearly document migration path in CHANGELOG
- Consider deprecation period for public APIs
- Update all examples and tests

### Pre-Commit Checklist
```bash
# Format code
cargo fmt

# Check compilation for all feature combinations
cargo check
cargo check --features=rug
cargo check --features=backtrace
cargo check --all-features

# Run tests
cargo test
cargo test --features=rug
cargo test --all-features

# Check for common mistakes
cargo clippy -- -D warnings

# Verify documentation builds
cargo doc --features=rug --no-deps
```

### Documentation Updates
- **API changes:** Update doc comments with examples
- **Architecture changes:** Update `.github/copilot-instructions.md` and `docs/ARCHITECTURE.md`
- **User-facing changes:** Update README.md
- **Internal changes:** Document in code comments

## When Working With This Codebase

**📖 Essential Reading:** Before making significant changes, read [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md) to understand the design philosophy and implementation patterns.

1. **Before adding raw operations:** Ensure they follow the `unchecked_*` naming convention
2. **When adding traits:** Use `#[duplicate_item]` macro to reduce boilerplate
3. **For new validation policies:** Implement `GuaranteesFiniteRealValues` if guaranteeing finite values
4. **Adding rug support:** Always feature-gate with `#[cfg(feature = "rug")]`
5. **Error types:** Use `thiserror` and follow input/output error categorization
6. **Generic bounds:** Use `FpScalar`, `RealScalar`, or `ComplexScalar` for maximum flexibility
7. **Before committing:** Run `cargo fmt ; cargo test --features=rug`
8. **Performance-critical code:** Benchmark before and after with `cargo bench`
9. **Breaking changes:** Update CHANGELOG and consider MAJOR version bump
10. **Documentation changes:** Update README.md, lib.rs docs, and ARCHITECTURE.md as needed