Struct ParquetSerDe 

#[non_exhaustive]
pub struct ParquetSerDe {
    pub block_size_bytes: Option<i32>,
    pub page_size_bytes: Option<i32>,
    pub compression: Option<ParquetCompression>,
    pub enable_dictionary_compression: Option<bool>,
    pub max_padding_bytes: Option<i32>,
    pub writer_version: Option<ParquetWriterVersion>,
}

A serializer to use for converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

block_size_bytes: Option<i32>

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Firehose uses this value for padding calculations.

page_size_bytes: Option<i32>

The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

compression: Option<ParquetCompression>

The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

enable_dictionary_compression: Option<bool>

Indicates whether to enable dictionary compression.

max_padding_bytes: Option<i32>

The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

writer_version: Option<ParquetWriterVersion>

Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

Implementations

impl ParquetSerDe

pub fn block_size_bytes(&self) -> Option<i32>

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Firehose uses this value for padding calculations.

pub fn page_size_bytes(&self) -> Option<i32>

The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

pub fn compression(&self) -> Option<&ParquetCompression>

The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

pub fn enable_dictionary_compression(&self) -> Option<bool>

Indicates whether to enable dictionary compression.

pub fn max_padding_bytes(&self) -> Option<i32>

The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

pub fn writer_version(&self) -> Option<&ParquetWriterVersion>

Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

impl ParquetSerDe

pub fn builder() -> ParquetSerDeBuilder

Creates a new builder-style object to manufacture ParquetSerDe.

Trait Implementations

impl Clone for ParquetSerDe

fn clone(&self) -> ParquetSerDe

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ParquetSerDe

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ParquetSerDe

fn eq(&self, other: &ParquetSerDe) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for ParquetSerDe