cortex_rust 0.6.0

High-performance LLM inference with 4-bit quantization and Test-Time Training (TTT)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

# Cortex Rust Engine - WebAssembly Support


## Prerequisites

- Rust (latest stable version)
- wasm-pack
- Node.js and npm

## Installation

1. Install wasm-pack:
```bash
cargo install wasm-pack

```

2. Add WebAssembly target:
```bash
rustup target add wasm32-unknown-unknown

```

## Building the WebAssembly Package

```bash
# Build WASM package

wasm-pack build --target web --features wasm

# Build npm package

wasm-pack build --target npm --features wasm
```

## Usage in Browser

```javascript
import init, { wasm_infer, wasm_log, init_panic_hook } from './pkg/cortex_rust.js';

async function run() {
  // Initialize WASM module
  await init();
  
  // Set up panic hook for better error reporting
  init_panic_hook();

  // Basic inference
  try {
    const result = wasm_infer("Your input text here");
    console.log(result);
  } catch (error) {
    console.error("Inference error:", error);
  }

  // Optional logging
  wasm_log("WebAssembly module initialized");
}

run();
```

## Limitations

- No CUDA support (CPU-only)
- Limited file I/O capabilities
- Memory constraints inherent to WebAssembly

## Notes

- Requires modern browser with WebAssembly support
- Performance may differ from native Rust execution