polyfill-rs 0.2.3

polyfill-rs

A high-performance, drop-in replacement for polymarket-rs-client with latency-optimized data structures and zero-allocation hot paths.

Quick Start

Add to your Cargo.toml:

[dependencies]
polyfill-rs = "0.2.3"

Replace your imports:

// Before: use polymarket_rs_client::{ClobClient, Side, OrderType};
use polyfill_rs::{ClobClient, Side, OrderType};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClobClient::new("https://clob.polymarket.com");
    let markets = client.get_sampling_markets(None).await?;
    println!("Found {} markets", markets.data.len());
    Ok(())
}

That's it! Your existing code works unchanged, but now runs significantly faster.

Why polyfill-rs?

100% API Compatible: Drop-in replacement for polymarket-rs-client with identical method signatures

Latency Optimized: Fixed-point arithmetic with cache-friendly data layouts for sub-microsecond order book operations

Market Microstructure Aware: Handles tick alignment, sequence validation, and market impact calculations with nanosecond precision

Production Hardened: Designed for co-located environments processing 100k+ market data updates per second

Performance Comparison

Real-World API Performance (with network I/O)

End-to-end performance with Polymarket's API, including network latency, JSON parsing, and decompression:

Operation	polyfill-rs	polymarket-rs-client	Official Python Client
Fetch Markets	321.6 ms ± 92.9 ms	409.3 ms ± 137.6 ms	1.366 s ± 0.048 s

Performance vs Competition:

21.4% faster than polymarket-rs-client - 87.6ms improvement
32.5% more consistent than polymarket-rs-client
4.2x faster than Official Python Client

Benchmark Methodology: All benchmarks run side-by-side on the same machine, same network, same time using identical testing methodology (20 iterations, 100ms delay between requests, /simplified-markets endpoint). Best performance achieved with connection keep-alive enabled. See examples/side_by_side_benchmark.rs for the complete benchmark implementation.

Computational Performance (pure CPU, no I/O)

Operation	Performance	Notes
Order Book Updates (1000 ops)	159.6 µs ± 32 µs	6,260 updates/sec, zero-allocation
Spread/Mid Calculations	70 ns ± 77 ns	14.3M ops/sec, optimized BTreeMap
JSON Parsing (480KB)	~2.3 ms	SIMD-accelerated parsing (1.77x faster than serde_json)

Key Performance Optimizations:

polyfill-rs achieves 21.4% better performance than polymarket-rs-client through several targeted optimizations and infrastructure integration, as verified through side-by-side benchmarking on identical infrastructure. We use simd-json for SIMD-accelerated JSON parsing, which provides a 1.77x speedup over standard serde_json deserialization and saves approximately 1-2ms per request. Our HTTP/2 configuration has been tuned through systematic benchmarking, with a 512KB initial stream window size proving optimal for the typical 469KB payload sizes from Polymarket's API. The client includes integrated DNS caching to eliminate redundant lookups, a connection manager with background keep-alive to maintain warm connections (preventing costly reconnections), and a buffer pool to reduce memory allocation overhead during request processing. These optimizations collectively achieve 321.6ms mean latency compared to polymarket-rs-client's 409.3ms while maintaining production-safe, conservative approaches.

Performance Breakdown:

Network (DNS/TCP/TLS): ~150ms (optimized with DNS caching and HTTP/2 tuning)
Download: ~230ms (improved with 512KB stream window)
JSON Parse: ~2.3ms (SIMD-accelerated, 1.77x faster than standard parsing)
Payload: 469KB compressed for simplified markets

Connection Reuse is Critical:

First request: ~500ms (connection establishment)
Subsequent requests: ~220-280ms (35.5% faster with connection pooling)
Keep client alive between requests for best performance

Real Performance Factors:

Network latency dominates (200-400ms)
Payload size matters (simplified: 480KB, full: 2.4MB)
Connection reuse critical for performance
Different endpoints serve different use cases

Benchmarking Methodology

Side-by-Side Testing: To ensure fair comparison, we benchmark polyfill-rs and polymarket-rs-client side-by-side on the same machine under identical conditions. Both clients are tested sequentially with the same network state, same API endpoint (/simplified-markets), and identical testing parameters (20 iterations, 100ms delay between requests). This eliminates variables like network conditions, time of day, or geographic differences that could skew results. The side-by-side benchmark reveals that polymarket-rs-client's claimed variance of ±22.9ms significantly understates their actual variance of ±137.6ms (500% higher), while our measurements remain consistent and reproducible.

What We Measure:

Real-world API performance with actual network I/O
Statistical analysis with multiple runs (mean ± standard deviation)
Connection establishment overhead and warm connection performance
Variance analysis to measure consistency

Reproducible Benchmarks:

# Run real-world performance benchmarks (requires .env with API credentials)
cargo run --example performance_benchmark --release

# Run side-by-side comparison with polymarket-rs-client
# (Requires uncommenting polymarket-rs-client in Cargo.toml dev-dependencies)
cargo run --example side_by_side_benchmark --release

All benchmarks use identical testing methodology and are reproducible on any machine with the same network conditions. The side-by-side benchmark validates our performance claims by running both clients sequentially under identical conditions.

Migration from polymarket-rs-client

Drop-in replacement in 2 steps:

Update Cargo.toml:

# Before: polymarket-rs-client = "0.x.x"
polyfill-rs = "0.2.3"

Update imports:

// Before: use polymarket_rs_client::{ClobClient, Side, OrderType};
use polyfill_rs::{ClobClient, Side, OrderType};

Usage Examples

Basic Trading Bot:

use polyfill_rs::{ClobClient, OrderArgs, Side, OrderType};
use rust_decimal_macros::dec;

let client = ClobClient::with_l2_headers(host, private_key, chain_id, api_creds);

// Create and submit order
let order_args = OrderArgs::new("token_id", dec!(0.75), dec!(100.0), Side::BUY);
let result = client.create_and_post_order(&order_args).await?;

High-Frequency Market Making:

use polyfill_rs::{OrderBookImpl, WebSocketStream};

// Real-time order book with fixed-point optimizations
let mut book = OrderBookImpl::new("token_id".to_string(), 100);
let mut stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market").await?;

// Process thousands of updates per second
while let Some(update) = stream.next().await {
    book.apply_delta_fast(&update.into())?;
    let spread = book.spread_fast(); // Returns in ticks for maximum speed
}

How It Works

The library has four main pieces that work together:

Order Book Engine

Critical path optimization through fixed-point arithmetic and memory layout design:

Before: BTreeMap<Decimal, Decimal> (heap allocations, decimal arithmetic overhead)
After: BTreeMap<u32, i64> (stack-allocated keys, branchless integer operations)

Order book updates achieve ~10x throughput improvement by eliminating decimal parsing in the critical path. Price quantization happens at ingress boundaries, maintaining IEEE 754 compatibility at API surfaces while using fixed-point internally for cache efficiency.

Want to see how this works? Check out src/book.rs - every optimization has the commented-out "before" code so you can see exactly what changed and why.

Market Impact Engine

Liquidity-aware execution simulation with configurable market impact models:

let impact = book.calculate_market_impact(Side::BUY, Decimal::from(1000));
// Returns: VWAP, total cost, basis point impact, liquidity consumption

Implements linear and square-root market impact models with parameterizable liquidity curves. Includes circuit breakers for adverse selection protection and maximum drawdown controls.

Market Data Infrastructure

Fault-tolerant WebSocket implementation with sequence gap detection and automatic recovery. Exponential backoff with jitter prevents thundering herd reconnection patterns. Message ordering guarantees maintained across reconnection boundaries.

Protocol Layer

EIP-712 signature validation, HMAC-SHA256 authentication, and adaptive rate limiting with token bucket algorithms. Request pipelining and connection pooling optimized for co-located deployment patterns.

Performance Characteristics

Designed for deterministic latency profiles in high-frequency environments:

Critical Path Optimizations

The library achieves deterministic latency through several fundamental design choices. Fixed-point arithmetic eliminates floating-point pipeline stalls and decimal parsing overhead that would otherwise introduce variable execution times. Lock-free updates using compare-and-swap operations enable concurrent book modifications without mutex contention or priority inversion. Cache-aligned structures maintain 64-byte alignment for optimal L1/L2 cache utilization, ensuring that hot data structures fit within single cache lines. Vectorized operations leverage SIMD-friendly data layouts to enable batch price level processing, allowing modern CPUs to process multiple price levels in parallel.

Memory Architecture

The memory subsystem is designed around predictable allocation patterns to prevent latency spikes. Pre-allocated pools eliminate garbage collection pressure and allocation latency spikes by maintaining warm buffers ready for immediate use. Configurable book depth limiting prevents memory bloat in illiquid markets where maintaining deep order books provides diminishing returns. Hot data structures are designed with temporal locality in mind, grouping frequently-accessed fields together to maximize cache line efficiency and minimize memory bandwidth consumption.

Architectural Principles

The library optimizes the precision-performance tradeoff through strategic boundary quantization. At system ingress, all price data converts to fixed-point representation at system boundaries while maintaining tick-aligned precision required by exchange protocols. The critical path operates exclusively on integer arithmetic with branchless comparisons and arithmetic operations in order matching logic, eliminating conditional jumps that would pollute the branch predictor. At system egress, all data converts back to IEEE 754 floating-point representation to ensure API surface compatibility with downstream consumers. This architecture enables deterministic execution with predictable instruction counts for latency-sensitive code paths. Performance-critical sections include cycle count analysis and memory access pattern documentation, with cache miss profiling and branch prediction optimization detailed in inline comments.

Performance Advantages

The library achieves superior performance through multiple orthogonal optimization strategies working in concert. Fixed-point arithmetic enables sub-nanosecond price calculations compared to the overhead of decimal operations, while zero-allocation updates allow order book modifications without triggering memory allocation or garbage collection pauses. Data structures use cache-optimized layouts with careful alignment to maximize CPU cache efficiency and minimize memory bandwidth requirements. Lock-free operations enable concurrent access patterns without mutex contention or context switching overhead. Network optimizations including HTTP/2 multiplexing, connection pooling, TCP_NODELAY for immediate packet transmission, and adaptive timeouts reduce end-to-end latency. Connection pre-warming provides 1.7x faster subsequent requests by maintaining warm TCP connections and pre-resolved DNS entries. Request parallelization achieves 3x speedup when batching operations by maximizing connection utilization and reducing round-trip overhead. Run benchmarks using cargo bench --bench comparison_benchmarks to measure these improvements on your hardware.

Network Optimization Deep Dive

How We Achieve Superior Network Performance

polyfill-rs implements advanced HTTP client optimizations specifically designed for latency-sensitive trading:

HTTP/2 Connection Management

// Optimized client with connection pooling
let client = ClobClient::new_internet("https://clob.polymarket.com");

// Pre-warm connections for 70% faster subsequent requests
client.prewarm_connections().await?;

Connection pooling: 5-20 persistent connections per host
TCP_NODELAY: Disables Nagle's algorithm for immediate packet transmission
HTTP/2 multiplexing: Multiple requests over single connection
Keep-alive optimization: Reduces connection establishment overhead

Request Batching & Parallelization

// Sequential requests (slow)
for token_id in token_ids {
    let price = client.get_price(&token_id).await?;
}

// Parallel requests (200% faster)
let futures = token_ids.iter().map(|id| client.get_price(id));
let prices = futures_util::future::join_all(futures).await;

Adaptive Network Resilience

Circuit breaker pattern: Prevents cascade failures during network instability
Adaptive timeouts: Dynamic timeout adjustment based on network conditions
Connection affinity: Sticky connections for consistent performance
Automatic retry logic: Exponential backoff with jitter

Measured Network Improvements

Optimization Technique	Performance Gain	Use Case
Optimized HTTP client	11% baseline improvement	Every API call
Connection pre-warming	70% faster subsequent requests	Application startup
Request parallelization	200% faster batch operations	Multi-market data fetching
Circuit breaker resilience	Better uptime during instability	Production trading systems

Environment-Specific Configurations

// For co-located servers (aggressive settings)
let client = ClobClient::new_colocated("https://clob.polymarket.com");

// For internet connections (conservative, reliable)
let client = ClobClient::new_internet("https://clob.polymarket.com");

// Standard balanced configuration
let client = ClobClient::new("https://clob.polymarket.com");

Configuration details:

Colocated: 20 connections, 1s timeouts, no compression (CPU optimization)
Internet: 5 connections, 60s timeouts, full compression (bandwidth optimization)
Standard: 10 connections, 30s timeouts, balanced settings

Getting Started

[dependencies]
polyfill-rs = "0.2.3"

Basic Usage

If You're Coming From polymarket-rs-client

Good news: your existing code should work without changes. I kept the same API.

use polyfill_rs::{ClobClient, OrderArgs, Side};
use rust_decimal::Decimal;

// Same initialization as before
let mut client = ClobClient::with_l1_headers(
    "https://clob.polymarket.com",
    "your_private_key",
    137,
);

// Same API calls
let api_creds = client.create_or_derive_api_key(None).await?;
client.set_api_creds(api_creds);

// Same order creation
let order_args = OrderArgs::new(
    "token_id",
    Decimal::from_str("0.75")?,
    Decimal::from_str("100.0")?,
    Side::BUY,
);

let result = client.create_and_post_order(&order_args).await?;

The difference is sub-microsecond order book operations and deterministic latency profiles.

Real-Time Order Book Tracking

Here's where it gets interesting. You can track live order books for multiple tokens:

use polyfill_rs::{OrderBookManager, OrderDelta, Side};

let mut book_manager = OrderBookManager::new(50); // Keep top 50 price levels

// This is what happens when you get a WebSocket update
let delta = OrderDelta {
    token_id: "market_token".to_string(),
    timestamp: chrono::Utc::now(),
    side: Side::BUY,
    price: Decimal::from_str("0.75")?,
    size: Decimal::from_str("100.0")?,  // 0 means remove this price level
    sequence: 1,
};

book_manager.apply_delta(delta)?;  // This is now super fast

// Get current market state
let book = book_manager.get_book("market_token")?;
let spread = book.spread();           // How tight is the market?
let mid_price = book.mid_price();     // Fair value estimate
let best_bid = book.best_bid();       // Highest buy price
let best_ask = book.best_ask();       // Lowest sell price

The apply_delta operation now executes in constant time with predictable cache behavior.

Market Impact Analysis

Before you place a big order, you probably want to know what it'll cost you:

use polyfill_rs::FillEngine;

let mut fill_engine = FillEngine::new(
    Decimal::from_str("0.001")?, // max slippage: 0.1%
    Decimal::from_str("0.02")?,  // fee rate: 2%
    10,                          // fee in basis points
);

// Simulate buying $1000 worth
let order = MarketOrderRequest {
    token_id: "market_token".to_string(),
    side: Side::BUY,
    amount: Decimal::from_str("1000.0")?,
    slippage_tolerance: Some(Decimal::from_str("0.005")?), // 0.5%
    client_id: None,
};

let result = fill_engine.execute_market_order(&order, &book)?;

println!("If you bought $1000 worth right now:");
println!("- Average price: ${}", result.average_price);
println!("- Total tokens: {}", result.total_size);
println!("- Fees: ${}", result.fees);
println!("- Market impact: {}%", result.impact_pct * 100);

This tells you exactly what would happen without actually placing the order. Super useful for position sizing.

WebSocket Streaming (The Fun Part)

Here's how you connect to live market data. The library handles all the annoying reconnection stuff:

use polyfill_rs::{WebSocketStream, StreamManager};

let mut stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market");

// Set up authentication (you'll need API credentials)
let auth = WssAuth {
    address: "your_eth_address".to_string(),
    signature: "your_signature".to_string(),
    timestamp: chrono::Utc::now().timestamp() as u64,
    nonce: "random_nonce".to_string(),
};
stream = stream.with_auth(auth);

// Subscribe to specific markets
stream.subscribe_market_channel(vec!["token_id_1".to_string(), "token_id_2".to_string()]).await?;

// Process live updates
while let Some(message) = stream.next().await {
    match message? {
        StreamMessage::MarketBookUpdate { data } => {
            // This is where the fast order book updates happen
            book_manager.apply_delta_fast(data)?;
        }
        StreamMessage::MarketTrade { data } => {
            println!("Trade: {} tokens at ${}", data.size, data.price);
        }
        StreamMessage::Heartbeat { .. } => {
            // Connection is alive
        }
        _ => {}
    }
}

The stream automatically reconnects when it drops. You just keep processing messages.

Example: Simple Spread Trading Bot

Here's a basic bot that looks for wide spreads and tries to capture them:

use polyfill_rs::{ClobClient, OrderBookManager, FillEngine};

struct SpreadBot {
    client: ClobClient,
    book_manager: OrderBookManager,
    min_spread_pct: Decimal,  // Only trade if spread > this %
    position_size: Decimal,   // How much to trade each time
}

impl SpreadBot {
    async fn check_opportunity(&mut self, token_id: &str) -> Result<bool> {
        let book = self.book_manager.get_book(token_id)?;
        
        // Get current market state
        let spread_pct = book.spread_pct().unwrap_or_default();
        let best_bid = book.best_bid();
        let best_ask = book.best_ask();
        
        // Only trade if spread is wide enough and we have liquidity
        if spread_pct > self.min_spread_pct && best_bid.is_some() && best_ask.is_some() {
            println!("Found opportunity: {}% spread on {}", spread_pct, token_id);
            
            // Check if our order size would move the market too much
            let impact = book.calculate_market_impact(Side::BUY, self.position_size);
            if let Some(impact) = impact {
                if impact.impact_pct < Decimal::from_str("0.01")? { // < 1% impact
                    return Ok(true);
                }
            }
        }
        
        Ok(false)
    }
    
    async fn execute_trade(&mut self, token_id: &str) -> Result<()> {
        // This is where you'd actually place orders
        // Left as an exercise for the reader :)
        println!("Would place orders for {}", token_id);
        Ok(())
    }
}

The key insight: with fast order book updates, you can check hundreds of tokens for opportunities without the library being the bottleneck.

Pro tip: The trading strategy examples in the code include detailed comments about market microstructure, order flow, and risk management techniques.

Configuration Tips

Order Book Depth Settings

The most important performance knob is how many price levels to track:

// For most trading bots: 10-50 levels is plenty
let book_manager = OrderBookManager::new(20);

// For market making: maybe 100+ levels
let book_manager = OrderBookManager::new(100);

// For analysis/research: could go higher, but memory usage grows
let book_manager = OrderBookManager::new(500);

Why this matters: Each price level takes memory, but 90% of trading happens in the top 10 levels anyway. More levels = more memory usage for diminishing returns.

The code comments in src/book.rs explain the memory layout and why we chose these specific data structures for different use cases.

WebSocket Reconnection

The defaults are pretty good, but you can tune them:

let reconnect_config = ReconnectConfig {
    max_retries: 5,                                    // Give up after 5 attempts
    base_delay: Duration::from_secs(1),               // Start with 1 second delay
    max_delay: Duration::from_secs(60),               // Cap at 1 minute
    backoff_multiplier: 2.0,                          // Double delay each time
};

let stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market")
    .with_reconnect_config(reconnect_config);

Memory Usage

If you're tracking lots of tokens, you might want to clean up stale books:

// Remove books that haven't updated in 5 minutes
let removed = book_manager.cleanup_stale_books(Duration::from_secs(300))?;
println!("Cleaned up {} stale order books", removed);

Market Microstructure Compliance

Automatic tick size validation and price quantization prevent market fragmentation and ensure exchange compatibility. Sub-tick pricing rejection happens at ingress with zero-cost integer modulo operations.

Tick alignment implementation includes detailed analysis of market maker adverse selection and the role of minimum price increments in maintaining orderly markets.

Memory Management

Bounded memory growth through configurable depth limits and automatic stale data eviction. Memory usage scales linearly with active price levels rather than total market depth, preventing memory exhaustion in volatile market conditions.