Crate google_dataproc1 [−] [src]

This documentation was generated from dataproc crate version 1.0.3+20161102, where 20161102 is the exact revision of the dataproc:v1 schema built by the mako code generator v1.0.3.

Everything else about the dataproc v1 API can be found at the official documentation site. The original source code is on github.

Features

Handle the following Resources with ease from the central hub ...

projects
- regions clusters create, regions clusters delete, regions clusters diagnose, regions clusters get, regions clusters list, regions clusters patch, regions jobs cancel, regions jobs delete, regions jobs get, regions jobs list, regions jobs submit, regions operations cancel, regions operations delete, regions operations get and regions operations list

Not what you are looking for ? Find all other Google APIs in their Rust documentation index.

Structure of this Library

The API is structured into the following primary items:

Hub
- a central object to maintain state and allow accessing all Activities
- creates Method Builders which in turn allow access to individual Call Builders
Resources
- primary types that you can apply Activities to
- a collection of properties and Parts
- Parts
  - a collection of properties
  - never directly used in Activities
Activities
- operations to apply to Resources

All structures are marked with applicable traits to further categorize them and ease browsing.

Generally speaking, you can invoke Activities like this:

let r = hub.resource().activity(...).doit()

Or specifically ...

let r = hub.projects().regions_clusters_patch(...).doit()
let r = hub.projects().regions_clusters_create(...).doit()
let r = hub.projects().regions_operations_get(...).doit()
let r = hub.projects().regions_clusters_delete(...).doit()
let r = hub.projects().regions_clusters_diagnose(...).doit()

The resource() and activity(...) calls create builders. The second one dealing with Activities supports various methods to configure the impending operation (not shown here). It is made such that all required arguments have to be specified right away (i.e. (...)), whereas all optional ones can be build up as desired. The doit() method performs the actual communication with the server and returns the respective result.

Usage

Setting up your Project

To use this library, you would put the following lines into your Cargo.toml file:

[dependencies]
google-dataproc1 = "*"

A complete example

extern crate hyper;
extern crate yup_oauth2 as oauth2;
extern crate google_dataproc1 as dataproc1;
use dataproc1::Cluster;
use dataproc1::{Result, Error};
use std::default::Default;
use oauth2::{Authenticator, DefaultAuthenticatorDelegate, ApplicationSecret, MemoryStorage};
use dataproc1::Dataproc;
 
// Get an ApplicationSecret instance by some means. It contains the `client_id` and 
// `client_secret`, among other things.
let secret: ApplicationSecret = Default::default();
// Instantiate the authenticator. It will choose a suitable authentication flow for you, 
// unless you replace  `None` with the desired Flow.
// Provide your own `AuthenticatorDelegate` to adjust the way it operates and get feedback about 
// what's going on. You probably want to bring in your own `TokenStorage` to persist tokens and
// retrieve them from storage.
let auth = Authenticator::new(&secret, DefaultAuthenticatorDelegate,
                              hyper::Client::new(),
                              <MemoryStorage as Default>::default(), None);
let mut hub = Dataproc::new(hyper::Client::new(), auth);
// As the method needs a request, you would usually fill it with the desired information
// into the respective structure. Some of the parts shown here might not be applicable !
// Values shown here are possibly random and not representative !
let mut req = Cluster::default();
 
// You can configure optional parameters by calling the respective setters at will, and
// execute the final call using `doit()`.
// Values shown here are possibly random and not representative !
let result = hub.projects().regions_clusters_patch(req, "projectId", "region", "clusterName")
             .update_mask("et")
             .doit();
 
match result {
    Err(e) => match e {
        // The Error enum provides details about what exactly happened.
        // You can also just use its `Debug`, `Display` or `Error` traits
         Error::HttpError(_)
        |Error::MissingAPIKey
        |Error::MissingToken(_)
        |Error::Cancelled
        |Error::UploadSizeLimitExceeded(_, _)
        |Error::Failure(_)
        |Error::BadRequest(_)
        |Error::FieldClash(_)
        |Error::JsonDecodeError(_, _) => println!("{}", e),
    },
    Ok(res) => println!("Success: {:?}", res),
}

Handling Errors

All errors produced by the system are provided either as Result enumeration as return value of the doit() methods, or handed as possibly intermediate results to either the Hub Delegate, or the Authenticator Delegate.

When delegates handle errors or intermediate values, they may have a chance to instruct the system to retry. This makes the system potentially resilient to all kinds of errors.

Uploads and Downloads

If a method supports downloads, the response body, which is part of the Result, should be read by you to obtain the media. If such a method also supports a Response Result, it will return that by default. You can see it as meta-data for the actual media. To trigger a media download, you will have to set up the builder by making this call: .param("alt", "media").

Methods supporting uploads can do so using up to 2 different protocols: simple and resumable. The distinctiveness of each is represented by customized doit(...) methods, which are then named upload(...) and upload_resumable(...) respectively.

Customization and Callbacks

You may alter the way an doit() method is called by providing a delegate to the Method Builder before making the final doit() call. Respective methods will be called to provide progress information, as well as determine whether the system should retry on failure.

The delegate trait is default-implemented, allowing you to customize it with minimal effort.

Optional Parts in Server-Requests

All structures provided by this library are made to be enocodable and decodable via json. Optionals are used to indicate that partial requests are responses are valid. Most optionals are are considered Parts which are identifiable by name, which will be sent to the server to indicate either the set parts of the request or the desired parts in the response.

Builder Arguments

Using method builders, you are able to prepare an action call by repeatedly calling it's methods. These will always take a single argument, for which the following statements are true.

PODs are handed by copy
strings are passed as &str
request values are moved

Arguments will always be copied or cloned into the builder, to make them independent of their original life times.

Structs

CancelJobRequest	A request to cancel a job.
Cluster	Describes the identifying information, config, and status of a cluster of Google Compute Engine instances.
ClusterConfig	The cluster config.
ClusterMetrics	Contains cluster daemon metrics, such as HDFS and YARN stats. Beta Feature: This report is available for testing purposes only. It may be changed before final release.
ClusterStatus	The status of a cluster and its instances.
Dataproc	Central instance to access all Dataproc related resource activities
DefaultDelegate	A delegate with a conservative default implementation, which is used if no other delegate is set.
DiagnoseClusterRequest	A request to collect cluster diagnostic information.
DiskConfig	Specifies the config of disk options for a group of VM instances.
Empty	A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for `Empty` is empty JSON object `{}`.
ErrorResponse	A utility to represent detailed errors we might see in case there are BadRequests. The latter happen if the sent parameters or request structures are unsound
GceClusterConfig	Common config settings for resources of Google Compute Engine cluster instances, applicable to all instances in the cluster.
HadoopJob	A Cloud Dataproc job for running Apache Hadoop MapReduce jobs on Apache Hadoop YARN.
HiveJob	A Cloud Dataproc job for running Apache Hive queries on YARN.
InstanceGroupConfig	[Optional] The config settings for Google Compute Engine resources in an instance group, such as a master or worker group.
Job	A Cloud Dataproc job resource.
JobPlacement	Cloud Dataproc job config.
JobReference	Encapsulates the full scoping used to reference a job.
JobStatus	Cloud Dataproc job status.
ListClustersResponse	The list of all clusters in a project.
ListJobsResponse	A list of jobs in a project.
ListOperationsResponse	The response message for Operations.ListOperations.
LoggingConfig	The runtime logging config of the job.
ManagedGroupConfig	Specifies the resources used to actively manage an instance group.
MethodInfo	Contains information about an API request.
MultiPartReader	Provides a `Read` interface that converts multiple parts into the protocol identified by RFC2387. Note: This implementation is just as rich as it needs to be to perform uploads to google APIs, and might not be a fully-featured implementation.
NodeInitializationAction	Specifies an executable to run on a fully configured node and a timeout period for executable completion.
Operation	This resource represents a long-running operation that is the result of a network API call.
PigJob	A Cloud Dataproc job for running Apache Pig queries on YARN.
ProjectMethods	A builder providing access to all methods supported on project resources. It is not used directly, but through the `Dataproc` hub.
ProjectRegionClusterCreateCall	Creates a cluster in a project.
ProjectRegionClusterDeleteCall	Deletes a cluster in a project.
ProjectRegionClusterDiagnoseCall	Gets cluster diagnostic information. After the operation completes, the Operation.response field contains `DiagnoseClusterOutputLocation`.
ProjectRegionClusterGetCall	Gets the resource representation for a cluster in a project.
ProjectRegionClusterListCall	Lists all regions/{region}/clusters in a project.
ProjectRegionClusterPatchCall	Updates a cluster in a project.
ProjectRegionJobCancelCall	Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list or regions/{region}/jobs.get.
ProjectRegionJobDeleteCall	Deletes the job from the project. If the job is active, the delete fails, and the response returns `FAILED_PRECONDITION`.
ProjectRegionJobGetCall	Gets the resource representation for a job in a project.
ProjectRegionJobListCall	Lists regions/{region}/jobs in a project.
ProjectRegionJobSubmitCall	Submits a job to a cluster.
ProjectRegionOperationCancelCall	Starts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. If the server doesn't support this method, it returns `google.rpc.Code.UNIMPLEMENTED`. Clients can use Operations.GetOperation or other methods to check whether the cancellation succeeded or whether the operation completed despite cancellation. On successful cancellation, the operation is not deleted; instead, it becomes an operation with an Operation.error value with a google.rpc.Status.code of 1, corresponding to `Code.CANCELLED`.
ProjectRegionOperationDeleteCall	Deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation. If the server doesn't support this method, it returns `google.rpc.Code.UNIMPLEMENTED`.
ProjectRegionOperationGetCall	Gets the latest state of a long-running operation. Clients can use this method to poll the operation result at intervals as recommended by the API service.
ProjectRegionOperationListCall	Lists operations that match the specified filter in the request. If the server doesn't support this method, it returns `UNIMPLEMENTED`. NOTE: the `name` binding below allows API services to override the binding to use different resource name schemes, such as `users/*/operations`.
PySparkJob	A Cloud Dataproc job for running Apache PySpark applications on YARN.
QueryList	A list of queries to run on a cluster.
SoftwareConfig	Specifies the selection and config of software inside the cluster.
SparkJob	A Cloud Dataproc job for running Apache Spark applications on YARN.
SparkSqlJob	A Cloud Dataproc job for running Apache Spark SQL queries.
Status	The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. The error model is designed to be: - Simple to use and understand for most users - Flexible enough to meet unexpected needs # Overview The `Status` message contains three pieces of data: error code, error message, and error details. The error code should be an enum value of google.rpc.Code, but it may accept additional error codes if needed. The error message should be a developer-facing English message that helps developers understand and resolve the error. If a localized user-facing error message is needed, put the localized message in the error details or localize it in the client. The optional error details may contain arbitrary information about the error. There is a predefined set of error detail types in the package `google.rpc` which can be used for common error conditions. # Language mapping The `Status` message is the logical representation of the error model, but it is not necessarily the actual wire format. When the `Status` message is exposed in different client libraries and different wire protocols, it can be mapped differently. For example, it will likely be mapped to some exceptions in Java, but more likely mapped to some error codes in C. # Other uses The error model and the `Status` message can be used in a variety of environments, either with or without APIs, to provide a consistent developer experience across different environments. Example uses of this error model include: - Partial errors. If a service needs to return partial errors to the client, it may embed the `Status` in the normal response to indicate the partial errors. - Workflow errors. A typical workflow has multiple steps. Each step may have a `Status` message for error reporting purpose. - Batch operations. If a client uses batch request and batch response, the `Status` message should be used directly inside batch response, one for each error sub-response. - Asynchronous operations. If an API call embeds asynchronous operation results in its response, the status of those operations should be represented directly using the `Status` message. - Logging. If some API errors are stored in logs, the message `Status` could be used directly after any stripping needed for security/privacy reasons.
SubmitJobRequest	A request to submit a job.
YarnApplication	A YARN application created by a job. Application information is a subset of org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto. Beta Feature: This report is available for testing purposes only. It may be changed before final release.

Enums

Error
Scope	Identifies the an OAuth2 authorization scope. A scope is needed when requesting an authorization token.

Traits

CallBuilder	Identifies types which represent builders for a particular resource method
Delegate	A trait specifying functionality to help controlling any request performed by the API. The trait has a conservative default implementation.
Hub	Identifies the Hub. There is only one per library, this trait is supposed to make intended use more explicit. The hub allows to access all resource methods more easily.
MethodsBuilder	Identifies types for building methods of a particular resource type
NestedType	Identifies types which are only used by other types internally. They have no special meaning, this trait just marks them for completeness.
Part	Identifies types which are only used as part of other types, which usually are carrying the `Resource` trait.
ReadSeek	A utility to specify reader types which provide seeking capabilities too
RequestValue	Identifies types which are used in API requests.
Resource	Identifies types which can be inserted and deleted. Types with this trait are most commonly used by clients of this API.
ResponseResult	Identifies types which are used in API responses.
ToParts	A trait for all types that can convert themselves into a parts string

Functions

remove_json_null_values

Type Definitions

Result

A universal result type used as return for all calls.