Skip to main content

Module paths

Module paths 

Source
Expand description

Turning a name in FROM into a file location.

SELECT * FROM 'edges.parquet' names a file where a table would go. A name that reads as a location (it has a scheme, a path separator, an extension or a glob) is opened as a listing table; anything else is left to the catalog, so a misspelt table name still reports “table not found”.

DataFusion’s ListingTableUrl::parse expands a glob only in a scheme-less local path. table_url also expands one in a URL (s3://bucket/dir/*.parquet, file:///data/*.csv): the URL is split at the path segment where the glob begins, the part before it becomes the listing prefix, and the rest filters the listing.

Functions§

expand_tilde
~ or ~/… with the home directory put in for the ~.
looks_like_location
Whether a table name reads as a file location rather than a table name.
split_url_glob
For a URL with a scheme, split it at the start of the path segment that holds the first glob character: s3://b/data/*.parquet gives ("s3://b/data/", "*.parquet"). None for a local path (DataFusion expands its globs itself), a URL without a glob, and http(s) URLs, where ? starts a query and a server cannot be listed anyway.
table_url
The listing table URL for a location: a local path (with globs, as DataFusion expands them), or a URL whose path may contain a glob.