Skip to main content

Module exec

Module exec 

Source
Expand description

The physical operator: a _search that streams its result page by page.

One _search can return at most index.max_result_window hits (10 000 by default), so a scan that simply set size would silently truncate any larger result. OpenSearchExec instead opens a scroll cursor and emits one RecordBatch per page until the result is exhausted or the plan’s fetch (a pushed-down LIMIT) is satisfied, so memory stays bounded by the page size and the result is complete. With more than one partition the scan is a sliced scroll: each DataFusion partition reads one slice (slice: { id, max }) in parallel, which is how providers over sharded stores fan out. A LIMIT or search_after read always uses one partition, so a limit is exact.

Structs§

OpenSearchExec
A scan of one index, streamed as pages of _source documents turned into record batches.