Expand description
The physical operator: a _search that streams its result page by page.
One _search can return at most index.max_result_window hits (10 000 by default), so a scan
that simply set size would silently truncate any larger result. OpenSearchExec instead
opens a scroll cursor and emits one RecordBatch per page until the result is exhausted or the
plan’s fetch (a pushed-down LIMIT) is satisfied, so memory stays bounded by the page size
and the result is complete. With more than one partition the scan is a sliced scroll: each
DataFusion partition reads one slice (slice: { id, max }) in parallel, which is how providers
over sharded stores fan out. A LIMIT or search_after read always uses one partition, so a
limit is exact.
Structs§
- Open
Search Exec - A scan of one index, streamed as pages of
_sourcedocuments turned into record batches.