Crate oximedia_distributed

Expand description

Distributed encoding coordinator for OxiMedia.

This crate provides a distributed video encoding system with:

Central coordinator for job management
Worker nodes for distributed encoding
Multiple splitting strategies (segment, tile, GOP-based)
Load balancing and fault tolerance
TCP-based coordinator control server (JSON protocol)
WebSocket-style real-time job event notifications (broadcast channel)
Cross-region geo-aware task placement
S3/object-storage segment I/O (behind s3 feature)
Kubernetes HPA auto-scaling hooks (behind k8s feature)
Raft consensus primitives with latency profiling

§gRPC / TCP API

The coordinator exposes a plain-text TCP control server on the configured coordinator_addr. Connect with any TCP client and send newline-terminated commands:

Command	Response
`status`	JSON object with aggregate counters
`nodes`	JSON array of registered workers
`jobs`	JSON array of in-flight jobs with progress

The protobuf/gRPC types live in pb and the full CoordinatorService gRPC implementation is in coordinator::CoordinatorServiceImpl.

§Deployment Architecture

┌─────────────────────────────────────────────────────────┐
│                        Client                           │
│   submit_job / job_status / cancel_job (TCP JSON RPC)   │
└────────────────────────┬────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│              Coordinator (DistributedEncoder)            │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐  │
│  │ Job Scheduler│  │ Health Check │  │Circuit Break│  │
│  │ (backpressure│  │    Loop      │  │    er       │  │
│  │  / priority) │  │(90s timeout) │  │             │  │
│  └──────────────┘  └──────────────┘  └─────────────┘  │
│  ┌──────────────────────────────────────────────────┐  │
│  │          NotificationBus (broadcast::channel)     │  │
│  └──────────────────────────────────────────────────┘  │
└────────────────────────┬────────────────────────────────┘
                         │   assign / heartbeat / result
         ┌───────────────┼────────────────────┐
         ▼               ▼                    ▼
┌─────────────┐  ┌─────────────┐    ┌─────────────┐
│  Worker 1   │  │  Worker 2   │ …  │  Worker N   │
│ (us-east-1) │  │ (eu-west-1) │    │ (ap-south)  │
└──────┬──────┘  └──────┬──────┘    └──────┬──────┘
       │                │                   │
       └────────────────┴───────────────────┘
                        │ upload segments
                        ▼
            ┌─────────────────────┐
            │   S3 / Object Store  │
            │  (s3_integration)    │
            └─────────────────────┘

Backpressure: Workers report load via heartbeat; the coordinator withholds new assignments when a worker is saturated (see backpressure).

Circuit breaker: Repeated worker failures trip the circuit breaker (see circuit_breaker); the coordinator stops routing jobs to that worker for a configurable cooldown period.

Geo-aware placement: When multiple workers are available, the scheduler uses geo_placement::select_worker_by_region to prefer low-latency workers in the target region.

Modules§

audit_log: Audit logging for coordinator state changes.
backpressure: Backpressure and rate-limiting for distributed encoding pipelines.
checkpointing: Distributed checkpoint management.
circuit_breaker: Circuit breaker pattern for distributed systems fault isolation.
cluster: Distributed cluster management.
compaction: Raft log compaction via snapshotting.
connection_pool: Connection pooling and retry for coordinator client connections.
consensus: Distributed consensus module (Raft-inspired).
coordinator: Central coordinator service for distributed encoding.
discovery: Worker discovery and health monitoring.
distributed_enhancements: Enhanced distributed primitives: Raft vote, work-stealing queue, backpressure controller, distributed checkpointing, consistent hash ring, distributed circuit breaker, shard allocator, service registry, and replication manager.
fault_tolerance: Distributed fault tolerance.
geo_placement: Cross-region geo-aware worker placement.
heartbeat: Node heartbeat management for distributed cluster health monitoring.
job_dag: Directed Acyclic Graph (DAG) for distributed job dependencies.
job_preemption: Job preemption for higher-priority jobs.
job_tracker: Distributed job tracking.
leader_election: Leader election primitives for OxiMedia distributed cluster.
lease: Raft leader lease implementation.
load_balancer: Distributed load balancing.
membership: Cluster membership management.
message_bus: Publish/subscribe message bus for distributed coordination.
message_queue: Distributed message queue primitives for OxiMedia.
metrics_aggregator: Distributed metrics collection and aggregation.
node_health: Node health monitoring for distributed systems.
node_registry: Node registry for cluster membership management.
node_topology: Network topology awareness for data-local task scheduling.
notifications: WebSocket-style real-time job status notification bus.
partition: Data partitioning for distributed encoding using consistent hashing.
pb: Protocol buffer generated code stub This is a minimal stub to allow the crate to compile Full protobuf generation requires tonic build configuration
raft_primitives: Raft-like consensus primitives.
replication: Log replication and term tracking for distributed consensus support.
resource_quota: Resource quota management for distributed workers.
scheduler: Job scheduling algorithms and priority management.
segment: Video segmentation and reassembly for distributed encoding.
segment_merge: Distributed merge/concatenation of encoded segments after parallel encoding.
shard: Data sharding and consistent hashing for distributed workload placement.
shard_map: Shard mapping for distributing data across nodes using consistent hashing.
snapshot_store: Snapshot storage primitives for OxiMedia distributed cluster.
task_distribution: Intelligent task distribution across worker nodes.
task_priority_queue: Priority-based task scheduling queue.
task_queue: Distributed task queue with priority ordering.
task_retry: Task retry logic with configurable backoff strategies for distributed jobs.
twopc: Two-phase commit (2PC) coordinator.
weighted_round_robin: Weighted Round-Robin (WRR) load balancing for distributed worker selection.
work_stealing: Work-stealing queue for task distribution.
worker: Worker node implementation for distributed encoding.
worker_draining: Worker draining — graceful shutdown for distributed encoding workers.

Structs§

DistributedConfig: Configuration for the distributed encoder
DistributedEncoder: Main distributed encoder interface.
DistributedJob: Represents a distributed encoding job
EncodingParams: Encoding parameters

Enums§

DiscoveryMethod: Worker discovery methods
DistributedError: Errors that can occur in distributed encoding
JobPriority: Job priority levels
JobStatus: Job execution status
SplitStrategy: Job splitting strategy

Type Aliases§

Result: Result type for distributed operations