List of all items
Structs
- AddModelConfig
- AgentToolApproval
- AgentToolApprovalDecision
- AgentToolApprovalRequest
- AgentToolMetadata
- AgenticSessionStore
- AgenticToolCallRecord
- AnyMoeConfig
- AnyMoeLoader
- AnyMoePipeline
- ApproximateUserLocation
- AudioInput
- AutoLoader
- AutoLoaderBuilder
- AutoTuneRequest
- AutoTuneResult
- BuildInfo
- CalledFunction
- ChatCompletionChunkResponse
- ChatCompletionResponse
- ChatTemplate
- Choice
- ChunkChoice
- CodeExecutionApproval
- CodeExecutionApprovalRequest
- CodeExecutionConfig
- CompletionChoice
- CompletionChunkChoice
- CompletionChunkResponse
- CompletionResponse
- CpuInfo
- Delta
- DetokenizationRequest
- DeviceInfo
- DeviceLayerMapMetadata
- DeviceMapMetadata
- DiffusionGenerationParams
- DiffusionLoader
- DiffusionLoaderBuilder
- DoctorCheck
- DoctorReport
- DrySamplingParams
- EmbeddingLoader
- EmbeddingLoaderBuilder
- EmbeddingModelPaths
- EmbeddingSpecificConfig
- EngineConfig
- Function
- GGMLLoader
- GGMLLoaderBuilder
- GGMLSpecificConfig
- GGUFLoader
- GGUFLoaderBuilder
- GGUFSpecificConfig
- GemmaLoader
- Hanzo
- HanzoBuilder
- HanzoConfig
- HfConnectivityInfo
- Idefics2Loader
- ImageChoice
- ImageGenerationResponse
- IntervalLogger
- LLaVALoader
- LLaVANextLoader
- LayerDeviceMapper
- LayerTopology
- LlamaLoader
- LoaderBuilder
- LocalModelPaths
- Logprobs
- LoraAdapterPaths
- McpClient
- McpClientConfig
- McpServerConfig
- McpToolInfo
- MemoryInfo
- MemoryUsage
- MistralLoader
- MixtralLoader
- Modalities
- ModelGenerationDefaults
- ModelLoaderConfig
- MultimodalLoader
- MultimodalLoaderBuilder
- MultimodalSpecificConfig
- NormalLoader
- NormalLoaderBuilder
- NormalRequest
- NormalSpecificConfig
- Ordering
- PagedAttentionConfig
- Phi2Loader
- Phi3Loader
- Phi3VLoader
- Qwen2Loader
- ResponseLogprob
- ResponseMessage
- SamplingParams
- SandboxPolicy
- SearchFunctionParameters
- SearchResult
- SerializedSession
- SerializedVideo
- SpeechLoader
- SpeechPipeline
- Starcoder2Loader
- SystemInfo
- TokenizationRequest
- Tool
- ToolCallContext
- ToolCallResponse
- ToolCallbackWithTool
- TopLogprob
- Topology
- TuneCandidate
- UnloadedModelState
- Usage
- VideoInput
- WebSearchOptions
- disk_kv_cache::CacheHit
- disk_kv_cache::DiskKvCache
- disk_kv_cache::KvcHeader
- files::File
- files::FileSource
- files::FileStore
- files::RequestedFile
- layers::AvgPool2d
- layers::CausalMaskConfig
- layers::CausalMasker
- layers::Conv3dConfig
- layers::Conv3dNoBias
- layers::DeepSeekV2RopeConfig
- layers::DeepSeekV2RotaryEmbedding
- layers::DiaRotaryEmbedding
- layers::F32RmsNorm
- layers::FloatInfo
- layers::Gemma3RopeScalingConfig
- layers::Gemma3RotaryEmbedding
- layers::Gemma3nRopeScalingConfig
- layers::Gemma3nRotaryEmbedding
- layers::GemmaRmsNorm
- layers::GptOssRotaryEmbedding
- layers::Llama3RopeConfig
- layers::Llama3RotaryEmbedding
- layers::MatMul
- layers::Mlp
- layers::Phi4MMRopeScalingConfig
- layers::Phi4MMRotaryEmbedding
- layers::PhiRopeConfig
- layers::PhiRotaryEmbedding
- layers::QLinear
- layers::QRmsNorm
- layers::Qwen2VLRotaryEmbedding
- layers::Qwen2_5VLRotaryEmbedding
- layers::Qwen3VLRotaryEmbedding
- layers::ReflectionPad2d
- layers::RmsNorm
- layers::RotaryEmbedding
- layers::ScaledEmbedding
- layers::Sdpa
- layers::SmolLm3RopeConfig
- layers::SmolLm3RotaryEmbedding
- matformer::MatformerConfig
- matformer::MatformerSliceConfig
- matformer::Slice
- reasoning_parsers::harmony::HarmonyAccumulated
- reasoning_parsers::harmony::HarmonyContext
- reasoning_parsers::harmony::HarmonyDelta
- reasoning_parsers::harmony::HarmonyToolCall
- reasoning_parsers::tag_based::TagReasoningContext
- speculative::cache::PagedSpeculativeCacheAccess
- speculative::cache::PagedSpeculativeCacheGuard
- speculative::cache::SpeculativeCacheOutcome
- speculative::config::MtpConfig
- speculative::logging::SpeculativeAttachInfo
- speculative::proposer::SpeculativeProposal
- speculative::proposer::SpeculativeProposalBatch
- speculative::proposer::SpeculativeProposeBatchCtx
- speculative::verifier::VerificationOutcome
Enums
- AdapterPaths
- AgentPermission
- AgentToolApprovalHandler
- AgentToolKind
- AgentToolSource
- AgenticToolCallData
- AgenticToolCallPhase
- AnyMoeExpertType
- AutoDeviceMapParams
- CodeExecutionPermission
- Constraint
- DefaultSchedulerMethod
- DeviceMapSetting
- DiffusionLoaderType
- DoctorStatus
- EmbeddingLoaderType
- EngineInstruction
- FitStatus
- GGUFArchitecture
- HanzoError
- ImageGenerationResponseFormat
- IsqBits
- IsqOrganization
- IsqType
- McpServerSource
- MemoryGpuConfig
- ModelCategory
- ModelDType
- ModelKind
- ModelSelected
- ModelStatus
- MultimodalLoaderType
- NetworkMode
- NormalLoaderType
- PagedCacheType
- QualityTier
- ReasoningEffort
- Request
- RequestMessage
- Response
- ResponseErr
- ResponseOk
- SchedulerConfig
- SearchContextSize
- SearchEmbeddingModel
- SpeechGenerationConfig
- SpeechLoaderType
- StopTokens
- SupportedModality
- TokenSource
- ToolCallType
- ToolCallbackKind
- ToolChoice
- ToolOutput
- ToolType
- TuneProfile
- WebSearchUserLocation
- disk_kv_cache::SaveReason
- files::FileContent
- layers::Activation
- layers::DeepSeekV2RopeScaling
- layers::Gemma3ScaledRopeType
- layers::Gemma3nScaledRopeType
- layers::Llama3RopeType
- layers::Phi4MMScaledRopeType
- layers::PhiRopeScalingConfig
- layers::ScaledRopeType
- layers::SmolLm3RopeType
- reasoning_parsers::ReasoningMode
- reasoning_parsers::harmony::HarmonyChannel
- speculative::config::SpeculativeConfig
- speculative::logging::SpeculativeAttachKind
- speculative::proposer::SpeculativeKvCache
Traits
- CustomLogitsProcessor
- Loader
- ModelPaths
- MultimodalPromptPrefixer
- Pipeline
- TryIntoDType
- layers::GetFloatInfo
- layers::TensorInfExtend
- reasoning_parsers::ReasoningParser
- speculative::cache::SpeculativeCacheAccess
- speculative::cache::SpeculativeCacheGuard
- speculative::driver::SpeculativePipelineExt
- speculative::proposer::SpeculativeProposer
- speculative::target::SpeculativeTargetMixin
- speech_utils::Sample
Functions
- auto_tune
- check_hf_gated_access
- collect_system_info
- disk_kv_cache::key_for
- distributed::is_daemon
- distributed::nccl_daemon_replicator
- distributed::ring_daemon_replicator
- distributed::use_nccl
- distributed::use_ring
- expand_isq_value
- files::compose_tool_response_with_files
- files::format_from_name
- files::is_text_mime
- files::merge_required_outputs_into_args
- files::mime_for_format
- files::required_files_tool_addendum
- files::tool_file_to_file
- get_auto_device_map_params
- get_engine_terminate_flag
- get_model_dtype
- get_tgt_non_granular_index
- get_toml_selected_model_device_map_params
- get_toml_selected_model_dtype
- hf_home_dir
- hf_hub_cache_dir
- hf_token_path
- initialize_logging
- is_hf_hub_offline
- layers::batch_norm
- layers::clamp_for_f16
- layers::conv1d
- layers::conv1d_no_bias
- layers::conv2d
- layers::conv2d_no_bias
- layers::embedding
- layers::group_norm
- layers::layer_norm
- layers::linear
- layers::linear_b
- layers::linear_no_bias
- layers::q_rms_norm_rope
- layers::qk_rms_norm_mrope
- layers::qk_rms_norm_rope
- layers::repeat_kv
- paged_attn_supported
- parse_isq_value
- parse_uqff_shard
- probe_hf_repo_files
- reasoning_parsers::harmony::is_harmony_encoding_ready
- reasoning_parsers::harmony::is_harmony_template
- reasoning_parsers::harmony::prewarm_harmony_encoding
- reasoning_parsers::is_reasoning_template
- reasoning_parsers::tag_based::is_channel_tag_template
- reasoning_parsers::tag_based::is_think_tag_template
- reset_engine_terminate_flag
- resolve_uqff_shorthand
- run_doctor
- sample_frame_indices
- should_terminate_engine_sequences
- speculative::driver::try_sample_speculative_causal_gen
- speculative::logging::log_attach
- speculative::verifier::finish_verified_step
- speech_utils::write_pcm_as_wav
- using_flash_attn
Type Aliases
- AgentToolApprovalAsyncCallback
- AgentToolApprovalCallback
- AgentToolApprovalFuture
- AgentToolApprovalNotifier
- CodeExecutionApprovalCallback
- CodeExecutionApprovalNotifier
- LlguidanceGrammar
- MessageContent
- MultimodalToolCallback
- SearchCallback
- ToolCallback
- ToolCallbacks
- speculative::proposer::TargetTokenEmbedder
Statics
Constants
- DEFAULT_MAX_TOOL_ROUNDS
- GGUF_MULTI_FILE_DELIMITER
- HANZO_GIT_REVISION
- HF_HUB_OFFLINE_ENV
- MULTI_LORA_DELIMITER
- SYSTEM_FINGERPRINT
- UQFF_MULTI_FILE_DELIMITER
- files::DEFAULT_FILE_TTL
- files::MODEL_INLINE_BYTES
- files::READ_FILE_MAX_SLICE_CHARS
- files::WIRE_EMBED_LIMIT_BYTES
- reasoning_parsers::tag_based::CHANNEL_CLOSE_TAG
- reasoning_parsers::tag_based::CHANNEL_CLOSE_TOKEN
- reasoning_parsers::tag_based::CHANNEL_OPEN_TAG
- reasoning_parsers::tag_based::CHANNEL_OPEN_TOKEN
- reasoning_parsers::tag_based::THINK_CLOSE_TAG
- reasoning_parsers::tag_based::THINK_OPEN_TAG