List of all items

Structs

• BackendConfig
• BatchId
• ClientId
• ComponentHealth
• ComponentStatus
• EngineConfig
• EngineMetrics
• EngineStatus
• HealthStatus
• InferenceRequest
• InferenceResponse
• MemoryUsage
• ModelId
• ModelInfo
• RequestId
• SamplingParams
• SchedulerConfig
• SchedulerStats
• SessionId
• SpecialTokens
• StreamChunk
• TaskId
• TokenId
• TokenizerConfig
• backend::BackendCapabilities
• backend::BackendConfig
• backend::BackendMemoryConfig
• backend::BackendRequirements
• backend::BackendStatus
• backend::KernelHandle
• backend::KernelInfo
• backend::TensorSpec
• backend::WeightLoaderCapabilities
• backend::WeightLoaderConfig
• backend::WeightMetadata
• kernel_ops::AttentionParams
• kernel_ops::KernelOpsDispatch
• kernel_ops::RoPEConfig
• kernel_ops::SamplingParams
• kv_cache::AllocationRequest
• kv_cache::BlockTable
• kv_cache::CacheConfig
• kv_cache::CacheGcStats
• kv_cache::CacheHandleStats
• kv_cache::CacheManagerStats
• kv_cache::CompressionStats
• kv_cache::LruEvictionPolicy
• kv_cache::MemoryPressureThresholds
• kv_cache::PrefixCacheConfig
• memory::AllocationRequirements
• memory::BestFitStrategy
• memory::DefragmentationStats
• memory::EventHandle
• memory::MemoryAccessStats
• memory::MemoryHandle
• memory::MemoryHandleInfo
• memory::MemoryInfo
• memory::MemoryManagerConfig
• memory::MemoryPoolConfig
• memory::MemoryTransfer
• memory::StreamHandle
• model_builder::ArchitectureDefinition
• model_builder::BuildOptions
• model_builder::BuildProgress
• model_builder::BuildTimeBreakdown
• model_builder::BuildTimeEstimate
• model_builder::BuildTimeFactor
• model_builder::BuilderCapabilities
• model_builder::BuilderInfo
• model_builder::GraphDefinition
• model_builder::GraphEdge
• model_builder::GraphNode
• model_builder::LayerDefinition
• model_builder::ModelArchitecture
• model_builder::ModelDimensions
• model_builder::ModelIR
• model_builder::ModelMetadata
• model_builder::OptimizationResult
• model_builder::OptimizationStats
• model_builder::ParameterSpec
• model_builder::TensorSpec
• model_builder::ValidationIssue
• model_executor::DecodeInput
• model_executor::DecodeOutput
• model_executor::ExecutorAttentionConfig
• model_executor::ExecutorCapabilities
• model_executor::ExecutorConfig
• model_executor::ExecutorMemoryConfig
• model_executor::ExecutorMemoryUsage
• model_executor::ExecutorMetrics
• model_executor::ExecutorStatus
• model_executor::MemoryRequirements
• model_executor::OptimizationConfig
• model_executor::PrefillInput
• model_executor::PrefillOutput
• model_executor::SpeculativeDecodeOutput
• sampler::GreedySampler
• sampler::LogitsProcessorChain
• sampler::MultinomialSampler
• sampler::RepetitionPenaltyProcessor
• sampler::SamplingConfig
• sampler::SamplingConfigBuilder
• sampler::SamplingContext
• sampler::SamplingStats
• sampler::TemperatureProcessor
• sampler::TopKProcessor
• sampler::TopPProcessor
• scheduler::AllocatedResources
• scheduler::BatchHint
• scheduler::BatchPlan
• scheduler::BatchResourceRequirements
• scheduler::BottleneckAnalysis
• scheduler::ClientResourceLimits
• scheduler::DynamicBatchingConfig
• scheduler::FairShareConfig
• scheduler::PreemptionResult
• scheduler::PreemptionState
• scheduler::QueueAnalysis
• scheduler::RequestPatternAnalysis
• scheduler::ResourceAwarenessConfig
• scheduler::ResourceConstraints
• scheduler::ResourceLimits
• scheduler::ResourceStats
• scheduler::ScheduledRequest
• scheduler::SchedulingSimulationResult
• scheduler::SeasonalityPattern
• scheduler::SimulatedWorkload
• scheduler::SlaConfig
• scheduler::SlaRequirements
• scheduler::WaitTimeDistribution
• tensor::TensorMemoryStats
• tokenizer::ChatMessage
• tokenizer::PaddingConfig
• tokenizer::TokenizerConfig
• tokenizer::TokenizerInfo
• tokenizer::TokenizerStats
• tokenizer::TruncationConfig
• transformer::TransformerConfig

Enums

• DataType
• Device
• FerrumError
• FinishReason
• ModelSource
• ModelType
• Priority
• backend::BackendType
• backend::KernelArg
• backend::MemoryGrowthStrategy
• backend::QuantizationConfig
• backend::ScalarValue
• backend::TensorTransformation
• backend::TransformationType
• backend::WeightFormat
• backend::WeightSource
• backend::WeightSourceType
• kernel_ops::QuantScheme
• kv_cache::MemoryPressure
• memory::AccessPatternType
• memory::MemoryAccess
• memory::MemoryPressure
• memory::MemoryType
• memory::MemoryUsageHint
• model_builder::BuildPhase
• model_builder::InitializationStrategy
• model_builder::ModelArchitectureFamily
• model_builder::OptimizationTechnique
• model_builder::ValidationSeverity
• model_executor::AttentionType
• model_executor::ExecutorState
• model_executor::ExecutorType
• sampler::ProcessorPriority
• scheduler::AdmissionDecision
• scheduler::ArrivalPattern
• scheduler::BottleneckType
• scheduler::RateTrend
• scheduler::SchedulingPolicy
• scheduler::SizeDistribution
• tokenizer::PaddingDirection
• tokenizer::PaddingStrategy
• tokenizer::TokenType
• tokenizer::TokenizerType
• tokenizer::TruncationStrategy

Traits

• backend::BackendFactory
• backend::BackendRegistry
• backend::ComputeBackend
• backend::KernelExecutor
• backend::WeightLoader
• decode_backend::DecodeBackend
• engine::AdvancedInferenceEngine
• engine::InferenceEngine
• kernel_ops::ActivationOps
• kernel_ops::AttentionOps
• kernel_ops::KernelOps
• kernel_ops::LinearOps
• kernel_ops::NormOps
• kernel_ops::PositionOps
• kernel_ops::SamplingOps
• kv_cache::AdvancedKvCacheManager
• kv_cache::BlockAllocator
• kv_cache::CacheEvictionPolicy
• kv_cache::KvCacheHandle
• kv_cache::KvCacheManager
• kv_cache::MultiDeviceCacheManager
• memory::AdvancedMemoryManager
• memory::AllocationStrategy
• memory::DeviceMemoryManager
• memory::GlobalMemoryMonitor
• memory::MemoryManagerFactory
• memory::StreamManager
• model_builder::AdvancedModelBuilder
• model_builder::CustomLayer
• model_builder::ModelBuilder
• model_builder::ModelBuilderFactory
• model_builder::ModelRegistry
• model_builder::OptimizationPass
• model_executor::BatchModelExecutor
• model_executor::ExecutorRegistry
• model_executor::ModelExecutor
• model_executor::ModelExecutorFactory
• model_executor::SpeculativeExecutor
• sampler::LogitsProcessor
• sampler::MultiSampler
• sampler::Sampler
• scheduler::AdmissionPolicy
• scheduler::AdvancedScheduler
• scheduler::Scheduler
• tensor::AsyncTensorOps
• tensor::TensorBatchOps
• tensor::TensorDataAccess
• tensor::TensorFactory
• tensor::TensorLike
• tensor::TensorMemoryManager
• tensor::TensorOps
• tokenizer::AsyncTokenizer
• tokenizer::IncrementalTokenizer
• tokenizer::TextProcessor
• tokenizer::Tokenizer
• tokenizer::TokenizerCapabilities
• tokenizer::TokenizerFactory
• tokenizer::TokenizerRegistry
• transformer::TransformerWeights

Functions

• tensor::utils::are_broadcastable
• tensor::utils::broadcast_shapes
• tensor::utils::matmul_output_shape

Type Aliases

• BlockId
• Result
• engine::HardwareConstraints
• engine::LatencyRequirements
• engine::RequestCharacteristics
• engine::SpeculationConfig
• scheduler::SchedulerConfig
• scheduler::SchedulerMetrics
• tensor::TensorRef