Struct SparkBatch

#[non_exhaustive]
pub struct SparkBatch {
    pub args: Vec<String>,
    pub jar_file_uris: Vec<String>,
    pub file_uris: Vec<String>,
    pub archive_uris: Vec<String>,
    pub driver: Option<Driver>,
    /* private fields */
}

A configuration for running an Apache Spark batch workload.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

args: Vec<String>

Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as --conf, since a collision can occur that causes an incorrect batch submission.

jar_file_uris: Vec<String>

Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.

file_uris: Vec<String>

Optional. HCFS URIs of files to be placed in the working directory of each executor.

archive_uris: Vec<String>

Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

driver: Option<Driver>

The specification of the main method to call to drive the Spark workload. Specify either the jar file that contains the main class or the main class name. To pass both a main jar and a main class in that jar, add the jar to jar_file_uris, and then specify the main class name in main_class.

Implementations

impl SparkBatch

pub fn new() -> Self

pub fn set_args<T, V>(self, v: T) -> Self

where T: IntoIterator<Item = V>, V: Into<String>,

Sets the value of args.

pub fn set_jar_file_uris<T, V>(self, v: T) -> Self

where T: IntoIterator<Item = V>, V: Into<String>,

Sets the value of jar_file_uris.

pub fn set_file_uris<T, V>(self, v: T) -> Self

where T: IntoIterator<Item = V>, V: Into<String>,

Sets the value of file_uris.

pub fn set_archive_uris<T, V>(self, v: T) -> Self

where T: IntoIterator<Item = V>, V: Into<String>,

Sets the value of archive_uris.

pub fn set_driver<T: Into<Option<Driver>>>(self, v: T) -> Self

Sets the value of driver.

Note that all the setters affecting driver are mutually exclusive.

pub fn main_jar_file_uri(&self) -> Option<&String>

The value of driver if it holds a MainJarFileUri, None if the field is not set or holds a different branch.

pub fn main_class(&self) -> Option<&String>

The value of driver if it holds a MainClass, None if the field is not set or holds a different branch.

pub fn set_main_jar_file_uri<T: Into<String>>(self, v: T) -> Self

Sets the value of driver to hold a MainJarFileUri.

Note that all the setters affecting driver are mutually exclusive.

pub fn set_main_class<T: Into<String>>(self, v: T) -> Self

Sets the value of driver to hold a MainClass.

Note that all the setters affecting driver are mutually exclusive.

Trait Implementations

impl Clone for SparkBatch

fn clone(&self) -> SparkBatch

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparkBatch

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SparkBatch

fn default() -> SparkBatch

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for SparkBatch

where SparkBatch: Default,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Message for SparkBatch

fn typename() -> &'static str

The typename of this message.

fn to_map(&self) -> Result<Map<String, Value>, AnyError>

where Self: Sized + Serialize,

Store the value into a JSON object.

fn from_map(map: &Map<String, Value>) -> Result<Self, AnyError>

where Self: DeserializeOwned,

Extract the value from a JSON object.

impl PartialEq for SparkBatch

fn eq(&self, other: &SparkBatch) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for SparkBatch

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for SparkBatch