Struct GcsSource 

#[non_exhaustive]
pub struct GcsSource {
    pub input_uris: Vec<String>,
    pub data_schema: String,
    /* private fields */
}

Available on crate features completion-service or document-service or user-event-service only.

Cloud Storage location for input content.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

input_uris: Vec<String>

Required. Cloud Storage URIs to input files. Each URI can be up to 2000 characters long. URIs can match the full object path (for example, gs://bucket/directory/object.json) or a pattern matching one or more files, such as gs://bucket/directory/*.json.

A request can contain at most 100 files (or 100,000 files if data_schema is content). Each file can be up to 2 GB (or 100 MB if data_schema is content).

data_schema: String

The schema to use when parsing the data from the source.

Supported values for document imports:

• document (default): One JSON Document per line. Each document must have a valid Document.id.
• content: Unstructured data (e.g. PDF, HTML). Each file matched by input_uris becomes a document, with the ID set to the first 128 bits of SHA256(URI) encoded as a hex string.
• custom: One custom data JSON per row in arbitrary format that conforms to the defined Schema of the data store. This can only be used by the GENERIC Data Store vertical.
• csv: A CSV file with header conforming to the defined Schema of the data store. Each entry after the header is imported as a Document. This can only be used by the GENERIC Data Store vertical.

Supported values for user event imports:

• user_event (default): One JSON UserEvent per line.

Implementations

impl GcsSource

pub fn new() -> Self

pub fn set_input_uris<T, V>(self, v: T) -> Self

where T: IntoIterator<Item = V>, V: Into<String>,

Sets the value of input_uris.

Example

let x = GcsSource::new().set_input_uris(["a", "b", "c"]);

pub fn set_data_schema<T: Into<String>>(self, v: T) -> Self

Sets the value of data_schema.

Example

let x = GcsSource::new().set_data_schema("example");

Trait Implementations

impl Clone for GcsSource

fn clone(&self) -> GcsSource

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GcsSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GcsSource

fn default() -> GcsSource

Returns the “default value” for a type.

impl Message for GcsSource

fn typename() -> &'static str

The typename of this message.

impl PartialEq for GcsSource

fn eq(&self, other: &GcsSource) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for GcsSource