Skip to main content

Module retrieval

Module retrieval 

Source
Expand description

Retrieval system for Vectorless document trees.

This module implements a hybrid retrieval architecture combining:

  • Adaptive Strategy Selection: Automatically chooses between keyword, semantic, and LLM strategies
  • Multi-Path Search: Beam search and MCTS for exploring multiple tree paths
  • Incremental Retrieval: Stops early when sufficient information is collected

§Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    RetrievalOrchestrator                         │
│                                                                  │
│  ┌─────────┐    ┌─────────┐    ┌─────────┐    ┌─────────┐      │
│  │ Analyze │───►│  Plan   │───►│ Search  │───►│  Evaluate  │      │
│  └─────────┘    └─────────┘    └─────────┘    └─────────┘      │
│                                     ▲              │             │
│                                     └──────────────┘             │
│                                    (NeedMoreData)               │
└─────────────────────────────────────────────────────────────────┘

§Pipeline Stages

StageDescription
AnalyzeStageQuery analysis (complexity, keywords, targets)
PlanStageStrategy and algorithm selection
SearchStageExecute tree search
EvaluateStageSufficiency checking

§Quick Start

use vectorless::retrieval::pipeline::{RetrievalOrchestrator, RetrievalStage};
use vectorless::retrieval::stages::{AnalyzeStage, PlanStage, SearchStage, EvaluateStage};

let orchestrator = RetrievalOrchestrator::new()
    .stage(AnalyzeStage::new())
    .stage(PlanStage::new())
    .stage(SearchStage::new())
    .stage(EvaluateStage::new());

let response = orchestrator.execute(tree, query, options).await?;

Re-exports§

pub use pipeline::CandidateNode;
pub use pipeline::ExecutionGroup;
pub use pipeline::FailurePolicy;
pub use pipeline::PipelineContext;
pub use pipeline::RetrievalMetrics;
pub use pipeline::RetrievalOrchestrator;
pub use pipeline::RetrievalStage;
pub use pipeline::SearchAlgorithm;
pub use pipeline::SearchConfig;
pub use pipeline::StageOutcome;
pub use pipeline::PipelineContext as StageContext;
pub use stages::AnalyzeStage;
pub use stages::EvaluateStage;
pub use stages::PlanStage;
pub use stages::SearchStage;
pub use strategy::CrossDocumentConfig;
pub use strategy::CrossDocumentStrategy;
pub use strategy::DocumentEntry;
pub use strategy::DocumentId;
pub use strategy::DocumentResult;
pub use strategy::HybridConfig;
pub use strategy::HybridStrategy;
pub use strategy::KeywordStrategy;
pub use strategy::LlmStrategy;
pub use strategy::MergeStrategy;
pub use strategy::PageRange;
pub use strategy::PageRangeConfig;
pub use strategy::PageRangeStrategy;
pub use strategy::RetrievalStrategy;
pub use strategy::SemanticStrategy;
pub use strategy::StrategyCapabilities;
pub use strategy::StrategyCost;
pub use search::BeamSearch;
pub use search::GreedySearch;
pub use search::SearchConfig as SearchAlgConfig;
pub use search::SearchResult;
pub use sufficiency::SufficiencyChecker;
pub use sufficiency::SufficiencyLevel;
pub use sufficiency::ThresholdChecker;
pub use complexity::ComplexityDetector;
pub use cache::PathCache;
pub use content::AggregationResult;
pub use content::AllocationResult;
pub use content::AllocationStrategy;
pub use content::BudgetAllocator;
pub use content::ContentAggregator;
pub use content::ContentAggregatorConfig;
pub use content::ContentChunk;
pub use content::ContentRelevance;
pub use content::OutputFormat;
pub use content::RelevanceScorer;
pub use content::ScoreComponents;
pub use content::ScoringStrategyConfig;
pub use content::SelectedContent;
pub use content::StructureBuilder;
pub use content::StructuredContent;
pub use pilot::NoopPilot;
pub use pilot::BudgetConfig;
pub use pilot::InterventionConfig;
pub use pilot::InterventionPoint;
pub use pilot::Pilot;
pub use pilot::PilotConfig;
pub use pilot::PilotDecision;
pub use pilot::PilotMode;
pub use pilot::RankedCandidate;
pub use pilot::SearchDirection;
pub use pilot::SearchState;

Modules§

cache
Caching for retrieval operations.
complexity
Query complexity detection.
content
Content aggregation module for retrieval results.
pilot
Pilot - The brain of the Retriever Pipeline.
pipeline
Retrieval pipeline infrastructure.
search
Search algorithms for tree traversal.
stages
Built-in retrieval pipeline stages.
strategy
Retrieval strategies for different query types.
sufficiency
Sufficiency checking for incremental retrieval.

Structs§

ContextBuilder
Context builder for assembling retrieval results.
DecompositionConfig
Configuration for query decomposition.
DecompositionResult
Result of query decomposition.
FollowedReference
Result of following a reference.
NavigationStep
A single navigation step in the search trace.
PipelineRetriever
Pipeline-based retriever using the stage architecture.
QueryDecomposer
Query decomposer for multi-turn retrieval.
ReferenceConfig
Configuration for reference following.
ReferenceExpansion
Reference expansion result for content aggregation.
ReferenceFollower
Reference follower for expanding content via cross-references.
ResultAggregator
Aggregator for combining sub-query results.
RetrievalContext
Context passed to strategies during retrieval.
RetrievalResult
A single retrieval result.
RetrievalStats
Statistics for a retrieval operation.
RetrieveOptions
Options for retrieval operations.
RetrieveResponse
Complete retrieval response.
SearchPath
Search path for multi-path algorithms.
SubQuery
Sub-query resulting from decomposition.
SubQueryResult
Result aggregator for multi-turn retrieval.

Enums§

NavigationDecision
Navigation decision at each step.
PruningStrategy
Pruning strategy for context building.
QueryComplexity
Query complexity level for adaptive strategy selection.
RetrieverError
Errors that can occur during retrieval.
Strategy
Strategy preference for retrieval.
StrategyPreference
Strategy preference for retrieval.
SubQueryComplexity
Complexity level for a sub-query.
SubQueryType
Type of sub-query.
SufficiencyLevel
Sufficiency level for incremental retrieval.
TokenEstimation
Token estimation mode.

Traits§

Retriever
Trait for document retrieval strategies.

Functions§

expand_with_references
Expand search results by following references.
format_for_llm
Format retrieval results for LLM consumption.
format_for_llm_async
Format retrieval results asynchronously.
format_tree_for_llm
Format a document tree for LLM consumption.
format_tree_for_llm_async
Format a document tree asynchronously.

Type Aliases§

RetrieverResult
Result type for retriever operations.