Struct graphannis::CorpusStorage

pub struct CorpusStorage { /* fields omitted */ }

A thread-safe API for managing corpora stored in a common location on the file system.

Multiple corpora can be part of a corpus storage and they are identified by their unique name. Corpora are loaded from disk into main memory on demand: An internal main memory cache is used to avoid re-loading a recently queried corpus from disk again.

Methods

impl CorpusStorage

pub fn with_cache_strategy(
    db_dir: &Path,
    cache_strategy: CacheStrategy,
    use_parallel_joins: bool
) -> Result<CorpusStorage>

Create a new instance with a maximum size for the internal corpus cache.

• db_dir - The path on the filesystem where the corpus storage content is located. Must be an existing directory.
• cache_strategy: A strategy for clearing the cache.
• use_parallel_joins - If true parallel joins are used by the system, using all available cores.

pub fn with_auto_cache_size(
    db_dir: &Path,
    use_parallel_joins: bool
) -> Result<CorpusStorage>

Create a new instance with a an automatic determined size of the internal corpus cache.

Currently, set the maximum cache size to 25% of the available/free memory at construction time. This behavior chan change in the future.

• db_dir - The path on the filesystem where the corpus storage content is located. Must be an existing directory.
• use_parallel_joins - If true parallel joins are used by the system, using all available cores.

pub fn list(&self) -> Result<Vec<CorpusInfo>>

List all available corpora in the corpus storage.

pub fn info(&self, corpus_name: &str) -> Result<CorpusInfo>

Return detailled information about a specific corpus with a given name (corpus_name).

pub fn import_from_fs(
    &self,
    path: &Path,
    format: ImportFormat,
    corpus_name: Option<String>
) -> Result<String>

Import a corpus from an external location on the file system into this corpus storage.

• path - The location on the file system where the corpus data is located.
• format - The format in which this corpus data is stored.
• corpus_name - Optionally override the name of the new corpus for file formats that already provide a corpus name.

pub fn delete(&self, corpus_name: &str) -> Result<bool>

Delete a corpus from this corpus storage. Returns true if the corpus was sucessfully deleted and false if no such corpus existed.

pub fn apply_update(
    &self,
    corpus_name: &str,
    update: &mut GraphUpdate
) -> Result<()>

Apply a sequence of updates (update parameter) to this graph for a corpus given by the corpus_name parameter.

It is ensured that the update process is atomic and that the changes are persisted to disk if the result is Ok.

pub fn preload(&self, corpus_name: &str) -> Result<()>

Preloads all annotation and graph storages from the disk into a main memory cache.

pub fn unload(&self, corpus_name: &str)

Unloads a corpus from the cache.

pub fn update_statistics(&self, corpus_name: &str) -> Result<()>

Update the graph and annotation statistics for a corpus given by corpus_name.

pub fn validate_query(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage
) -> Result<bool>

Parses a query and checks if it is valid.

• corpus_name - The name of the corpus the query would be executed on (needed because missing annotation names can be a semantic parser error).
• query - The query as string.
• query_language The query language of the query (e.g. AQL).

Returns true if valid and an error with the parser message if invalid.

pub fn plan(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage
) -> Result<String>

Returns a string representation of the execution plan for a query.

• corpus_name - The name of the corpus to execute the query on.
• query - The query as string.
• query_language The query language of the query (e.g. AQL).

pub fn count(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage
) -> Result<u64>

Count the number of results for a query.

• corpus_name - The name of the corpus to execute the query on.
• query - The query as string.
• query_language The query language of the query (e.g. AQL).

Returns the count as number.

pub fn count_extra(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage
) -> Result<CountExtra>

Count the number of results for a query and return both the total number of matches and also the number of documents in the result set.

• corpus_name - The name of the corpus to execute the query on.
• query - The query as string.
• query_language The query language of the query (e.g. AQL).

pub fn find(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage,
    offset: usize,
    limit: usize,
    order: ResultOrder
) -> Result<Vec<String>>

Find all results for a query and return the match ID for each result.

The query is paginated and an offset and limit can be specified.

• corpus_name - The name of the corpus to execute the query on.
• query - The query as string.
• query_language The query language of the query (e.g. AQL).
• offset - Skip the n first results, where n is the offset.
• limit - Return at most n matches, where n is the limit.
• order - Specify the order of the matches.

Returns a vector of match IDs, where each match ID consists of the matched node annotation identifiers separated by spaces. You can use the subgraph(...) method to get the subgraph for a single match described by the node annnotation identifiers.

pub fn subgraph(
    &self,
    corpus_name: &str,
    node_ids: Vec<String>,
    ctx_left: usize,
    ctx_right: usize
) -> Result<Graph>

Return the copy of a subgraph which includes the given list of node annotation identifiers, the nodes that cover the same token as the given nodes and all nodes that cover the token which are part of the defined context.

• corpus_name - The name of the corpus for which the subgraph should be generated from.
• node_ids - A set of node annotation identifiers describing the subgraph.
• ctx_left and ctx_right - Left and right context in token distance to be included in the subgraph.

pub fn subgraph_for_query(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage,
    component_type_filter: Option<ComponentType>
) -> Result<Graph>

Return the copy of a subgraph which includes all nodes matched by the given query.

• corpus_name - The name of the corpus for which the subgraph should be generated from.
• query - The query which defines included nodes.
• query_language - The query language of the query (e.g. AQL).
• component_type_filter - If set, only include edges of that belong to a component of the given type.

pub fn subcorpus_graph(
    &self,
    corpus_name: &str,
    corpus_ids: Vec<String>
) -> Result<Graph>

Return the copy of a subgraph which includes all nodes that belong to any of the given list of sub-corpus/document identifiers.

• corpus_name - The name of the corpus for which the subgraph should be generated from.
• corpus_ids - A set of sub-corpus/document identifiers describing the subgraph.

pub fn corpus_graph(&self, corpus_name: &str) -> Result<Graph>

Return the copy of the graph of the corpus structure given by corpus_name.

pub fn frequency(
    &self,
    corpus_name: &str,
    query: &str,
    query_language: QueryLanguage,
    definition: Vec<FrequencyDefEntry>
) -> Result<FrequencyTable<String>>

Execute a frequency query.

• corpus_name - The name of the corpus to execute the query on.
• query - The query as string.
• query_language The query language of the query (e.g. AQL).
• definition - A list of frequency query definitions.

Returns a frequency table of strings.

pub fn node_descriptions(
    &self,
    query: &str,
    query_language: QueryLanguage
) -> Result<Vec<QueryAttributeDescription>>

Parses a queryand return a list of descriptions for its nodes.

• query - The query to be analyzed.
• query_language - The query language of the query (e.g. AQL).

pub fn list_components(
    &self,
    corpus_name: &str,
    ctype: Option<ComponentType>,
    name: Option<&str>
) -> Vec<Component>

Returns a list of all components of a corpus given by corpus_name.

• ctype - Optionally filter by the component type.
• name - Optionally filter by the component name.

pub fn list_node_annotations(
    &self,
    corpus_name: &str,
    list_values: bool,
    only_most_frequent_values: bool
) -> Vec<Annotation>

Returns a list of all node annotations of a corpus given by corpus_name.

• list_values - If true include the possible values in the result.
• only_most_frequent_values - If both this argument and list_values are true, only return the most frequent value for each annotation name.

pub fn list_edge_annotations(
    &self,
    corpus_name: &str,
    component: &Component,
    list_values: bool,
    only_most_frequent_values: bool
) -> Vec<Annotation>

Returns a list of all edge annotations of a corpus given by corpus_name and the component.

• list_values - If true include the possible values in the result.
• only_most_frequent_values - If both this argument and list_values are true, only return the most frequent value for each annotation name.

Trait Implementations

impl Drop for CorpusStorage

fn drop(&mut self)

Executes the destructor for this type.