Crate sochdb_query 

SochDB Query Engine

SOCH-QL query language for TOON-native data.

SOCH-QL

SQL-like query language that returns results in TOON format:

SELECT id,name FROM users WHERE score > 80
→ users[2]{id,name}:
  1,Alice
  3,Charlie

Query Execution Pipeline (Task 6)

parse(sql) → SochQuery → validate → plan → execute → SochTable

Token reduction: 40-60% vs JSON (66% for typical queries)

CONTEXT SELECT (LLM-Native)

Priority-based context aggregation for LLM consumption:

CONTEXT SELECT
  FROM session('abc123')
  WITH TOKEN_LIMIT 4000
  SECTIONS (
    USER {query, preferences} PRIORITY 1,
    HISTORY {recent} PRIORITY 2,
    KNOWLEDGE {docs} PRIORITY 3
  )

Re-exports

pub use agent_context::AgentContext;

pub use agent_context::AgentPermissions;

pub use agent_context::AuditEntry;

pub use agent_context::AuditOperation;

pub use agent_context::AuditResult;

pub use agent_context::ContextError;

pub use agent_context::ContextValue;

pub use agent_context::DbPermissions;

pub use agent_context::FsPermissions;

pub use agent_context::OperationBudget;

pub use agent_context::PendingWrite;

pub use agent_context::ResourceType;

pub use agent_context::SessionId;

pub use agent_context::SessionManager;

pub use agent_context::TransactionScope;

pub use calc::BinaryOp;

pub use calc::CalcError;

pub use calc::Evaluator;

pub use calc::Expr;

pub use calc::Parser as CalcParser;

pub use calc::RowContext;

pub use calc::UnaryOp;

pub use calc::calculate;

pub use calc::parse_expr;

pub use context_query::ContextQueryError;

pub use context_query::ContextQueryParser;

pub use context_query::ContextQueryResult;

pub use context_query::ContextSection;

pub use context_query::ContextSelectQuery;

pub use context_query::HnswVectorIndex;

pub use context_query::SectionPriority;

pub use context_query::SectionResult;

pub use context_query::SimpleVectorIndex;

pub use context_query::VectorIndex;

pub use context_query::VectorIndexStats;

pub use context_query::VectorSearchResult;

pub use optimizer_integration::CacheStats;

pub use optimizer_integration::ExecutionPlan;

pub use optimizer_integration::ExecutionStep;

pub use optimizer_integration::OptimizedExecutor;

pub use optimizer_integration::OptimizedQueryPlan;

pub use optimizer_integration::PlanCache;

pub use optimizer_integration::StorageBackend;

pub use optimizer_integration::TableStats;

pub use plugin_table::PluginVirtualTable;

pub use plugin_table::VirtualColumnDef;

pub use plugin_table::VirtualColumnType;

pub use plugin_table::VirtualFilter;

pub use plugin_table::VirtualRow;

pub use plugin_table::VirtualTable;

pub use plugin_table::VirtualTableError;

pub use plugin_table::VirtualTableRegistry;

pub use plugin_table::VirtualTableSchema;

pub use plugin_table::VirtualTableStats;

pub use sql::BinaryOperator;

pub use sql::ColumnDef as SqlColumnDef;

pub use sql::CreateTableStmt;

pub use sql::DeleteStmt;

pub use sql::DropTableStmt;

pub use sql::Expr as SqlExpr;

pub use sql::InsertStmt;

pub use sql::JoinType;

pub use sql::Lexer;

pub use sql::OrderByItem as SqlOrderBy;

pub use sql::Parser as SqlParser;

pub use sql::SelectStmt;

pub use sql::Span;

pub use sql::SqlError;

pub use sql::SqlResult;

pub use sql::Statement;

pub use sql::Token;

pub use sql::TokenKind;

pub use sql::UnaryOperator;

pub use sql::UpdateStmt;

pub use token_budget::BudgetAllocation;

pub use token_budget::BudgetSection;

pub use token_budget::TokenBudgetConfig;

pub use token_budget::TokenBudgetEnforcer;

pub use token_budget::TokenEstimator;

pub use token_budget::TokenEstimatorConfig;

pub use token_budget::truncate_rows;

pub use token_budget::truncate_to_tokens;

pub use soch_ql::ColumnDef;

pub use soch_ql::ColumnType;

pub use soch_ql::ComparisonOp;

pub use soch_ql::Condition;

pub use soch_ql::CreateTableQuery;

pub use soch_ql::InsertQuery;

pub use soch_ql::LogicalOp;

pub use soch_ql::OrderBy;

pub use soch_ql::ParseError;

pub use soch_ql::SelectQuery;

pub use soch_ql::SortDirection;

pub use soch_ql::SochQlParser;

pub use soch_ql::SochQuery;

pub use soch_ql::SochResult;

pub use soch_ql::SochValue;

pub use soch_ql::WhereClause;

pub use soch_ql_executor::KeyRange;

pub use soch_ql_executor::Predicate;

pub use soch_ql_executor::PredicateCondition;

pub use soch_ql_executor::QueryPlan;

pub use soch_ql_executor::TokenReductionStats;

pub use soch_ql_executor::SochQlExecutor;

pub use soch_ql_executor::estimate_token_reduction;

pub use soch_ql_executor::execute_sochql;

pub use topk_executor::ColumnRef;

pub use topk_executor::ExecutionStrategy as TopKExecutionStrategy;

pub use topk_executor::IndexAwareTopK;

pub use topk_executor::OrderByColumn;

pub use topk_executor::OrderByLimitExecutor;

pub use topk_executor::OrderByLimitStats;

pub use topk_executor::OrderBySpec;

pub use topk_executor::SingleColumnTopK;

pub use topk_executor::SortDirection as TopKSortDirection;

pub use topk_executor::TopKHeap;

pub use streaming_context::RollingBudget;

pub use streaming_context::SectionChunk;

pub use streaming_context::StreamingConfig;

pub use streaming_context::StreamingContextExecutor;

pub use streaming_context::StreamingContextIter;

pub use embedding_provider::CachedEmbeddingProvider;

pub use embedding_provider::EmbeddingError;

pub use embedding_provider::EmbeddingProvider;

pub use embedding_provider::EmbeddingVectorIndex;

pub use embedding_provider::MockEmbeddingProvider;

pub use hybrid_retrieval::FusionMethod;

pub use hybrid_retrieval::HybridQuery;

pub use hybrid_retrieval::HybridQueryExecutor;

pub use hybrid_retrieval::LexicalIndex;

pub use hybrid_retrieval::MetadataFilter;

pub use temporal_decay::DecayCurve;

pub use temporal_decay::TemporalDecayConfig;

pub use temporal_decay::TemporalScorer;

pub use temporal_decay::TemporallyDecayedResult;

pub use memory_compaction::Abstraction;

pub use memory_compaction::CompactionStats;

pub use memory_compaction::Episode;

pub use memory_compaction::ExtractiveSummarizer;

pub use memory_compaction::HierarchicalMemory;

pub use memory_compaction::Summary;

pub use memory_compaction::Summarizer;

pub use exact_token_counter::ExactBudgetEnforcer;

pub use exact_token_counter::ExactTokenCounter;

pub use exact_token_counter::HeuristicTokenCounter;

pub use exact_token_counter::TokenCounter;

pub use semantic_triggers::EscalationLevel;

pub use semantic_triggers::EventSource;

pub use semantic_triggers::LogLevel;

pub use semantic_triggers::SemanticTrigger;

pub use semantic_triggers::TriggerAction;

pub use semantic_triggers::TriggerBuilder;

pub use semantic_triggers::TriggerError;

pub use semantic_triggers::TriggerEvent;

pub use semantic_triggers::TriggerIndex;

pub use semantic_triggers::TriggerMatch;

pub use semantic_triggers::TriggerStats;

pub use filter_ir::AuthCapabilities;

pub use filter_ir::AuthScope;

pub use filter_ir::Disjunction;

pub use filter_ir::FilterAtom;

pub use filter_ir::FilterBuilder;

pub use filter_ir::FilterIR;

pub use filter_ir::FilterValue;

pub use filter_ir::FilteredExecutor;

pub use namespace::Namespace;

pub use namespace::NamespaceError;

pub use namespace::NamespaceScope;

pub use namespace::QueryRequest;

pub use namespace::ScopedQuery;

pub use metadata_index::ConcurrentMetadataIndex;

pub use metadata_index::EqualityIndex;

pub use metadata_index::MetadataIndex;

pub use metadata_index::PostingSet;

pub use metadata_index::RangeIndex;

pub use candidate_gate::AllowedBitmap;

pub use candidate_gate::AllowedSet;

pub use candidate_gate::CandidateGate;

pub use candidate_gate::ExecutionStrategy;

pub use filtered_vector_search::FilterAwareSearch;

pub use filtered_vector_search::FilteredSearchConfig;

pub use filtered_vector_search::FilteredSearchResult;

pub use filtered_vector_search::FilteredSearchStrategy;

pub use filtered_vector_search::FilteredVectorStore;

pub use filtered_vector_search::ScoredResult;

pub use bm25_filtered::Bm25Params;

pub use bm25_filtered::DisjunctiveBm25Executor;

pub use bm25_filtered::FilteredBm25Executor;

pub use bm25_filtered::FilteredPhraseExecutor;

pub use bm25_filtered::InvertedIndex;

pub use bm25_filtered::PositionalIndex;

pub use bm25_filtered::PositionalPosting;

pub use bm25_filtered::PostingList;

pub use unified_fusion::Bm25Executor;

pub use unified_fusion::Bm25QuerySpec;

pub use unified_fusion::FilteredCandidates;

pub use unified_fusion::FusionConfig;

pub use unified_fusion::FusionEngine;

pub use unified_fusion::FusionMethod as UnifiedFusionMethod;

pub use unified_fusion::FusionResult;

pub use unified_fusion::Modality;

pub use unified_fusion::UnifiedHybridExecutor;

pub use unified_fusion::UnifiedHybridQuery;

pub use unified_fusion::VectorExecutor;

pub use unified_fusion::VectorQuerySpec;

pub use capability_token::AclTagIndex;

pub use capability_token::CapabilityToken;

pub use capability_token::TokenBuilder;

pub use capability_token::TokenCapabilities;

pub use capability_token::TokenError;

pub use capability_token::TokenSigner;

pub use capability_token::TokenValidator;

Modules

agent_context

Unified Agent Execution Context (Task 16)

bm25_filtered

BM25 Filter Pushdown via Posting-Set Intersection (Task 6)

calc

Calculator Expression Evaluator (Task 13)

candidate_gate

Unified Candidate Gate Interface (Task 4)

capability_token

Capability Tokens + ACLs (Task 8)

context_query

LLM-Native CONTEXT SELECT Query

cost_optimizer

Cost-Based Query Optimizer with Cardinality Estimation (Task 6)

embedding_provider

Automatic Embedding Generation (Task 2)

exact_token_counter

Exact Token Counting (Task 6)

filter_ir

Canonical Filter IR + Planner Pushdown Contract (Task 1)

filtered_vector_search

Filter-Aware Vector Search with Selectivity-Driven Fallback (Task 5)

hybrid_retrieval

Hybrid Retrieval Pipeline (Task 3)

memory_compaction

Hierarchical Memory Compaction (Task 5)

metadata_index

Metadata Index Primitives (Task 3)

namespace

Namespace-Scoped Query API (Task 2)

optimizer_integration

Query Optimizer Integration (Task 10)

plugin_table

Plugin-as-Table Integration

query_optimizer

Query Optimizer with Cost-Based Planning

semantic_triggers

Semantic Trigger Engine (Task 7)

simd_filter

SIMD Vectorized Query Filters

soch_ql

TOON Query Language (SOCH-QL)

soch_ql_executor

SOCH-QL Query Executor (Task 6)

sql

SQL Module - Complete SQL support for SochDB

streaming_context

Streaming Context Generation (Task 1)

temporal_decay

Temporal Decay Scoring (Task 4)

token_budget

Token Budget Enforcement

topk_executor

ORDER BY + LIMIT Optimization with Streaming Top-K

unified_fusion

Unified Hybrid Fusion with Mandatory Pre-Filtering (Task 7)

Macros

filter_ir

Create a filter IR from a simple DSL