Skip to main content

HttpSinkConfig

Struct HttpSinkConfig 

Source
pub struct HttpSinkConfig {
    pub url: String,
    pub method: Method,
    pub headers: HeaderMap,
    pub auth: AuthSpec<HttpSinkAuth>,
    pub batch_mode: HttpBatchMode,
    pub max_retries: usize,
    pub concurrency: usize,
    pub batch_size: usize,
}
Expand description

Configuration for the HTTP sink connector.

Fields§

§url: String

Target endpoint URL.

§method: Method

HTTP method (default: POST).

§headers: HeaderMap

Additional request headers.

§auth: AuthSpec<HttpSinkAuth>

Authentication: either inline ({ type, config }) or a { ref: <name> } pointer to a shared provider in the CLI’s top-level auth: catalog.

§batch_mode: HttpBatchMode

How to batch records in requests.

§max_retries: usize

Number of retries on transient failures (default: 0).

§concurrency: usize

Maximum number of concurrent requests in Individual mode (default: 10).

§batch_size: usize

Maximum number of records sent per outbound HTTP request. Defaults to DEFAULT_BATCH_SIZE (1000).

Interpretation depends on batch_mode:

  • In HttpBatchMode::Array mode, write_batch re-chunks the incoming slice into batch_size-row chunks and issues one POST request per chunk, with each request body a JSON array of up to batch_size records. batch_size = 0 is the “no batching” sentinel: the entire upstream StreamPage is forwarded as a single JSON array — useful when the source already chunks to a size the destination endpoint accepts.
  • In HttpBatchMode::Individual mode the sink already sends one request per record, so batch_size has no effect on wire framing; the field is accepted for config-shape parity with other sinks and validated via faucet_core::validate_batch_size at load time.

Recommended value for HTTP POST endpoints that accept arrays: match the destination’s documented batch limit (commonly 100–1000 records per request).

Implementations§

Source§

impl HttpSinkConfig

Source

pub fn new(url: impl Into<String>) -> Self

Create a new config with the given URL and sensible defaults.

Source

pub fn method(self, method: Method) -> Self

Set the HTTP method.

Source

pub fn headers(self, headers: HeaderMap) -> Self

Set additional request headers.

Source

pub fn auth(self, auth: HttpSinkAuth) -> Self

Set the authentication method (inline).

Source

pub fn batch_mode(self, mode: HttpBatchMode) -> Self

Set the batch mode.

Source

pub fn max_retries(self, retries: usize) -> Self

Set the maximum number of retries.

Source

pub fn concurrency(self, concurrency: usize) -> Self

Set the maximum number of concurrent requests in Individual mode.

Source

pub fn with_batch_size(self, batch_size: usize) -> Self

Set the maximum number of records sent per outbound HTTP request.

In HttpBatchMode::Array mode this controls how many records are packed into each POST body. In HttpBatchMode::Individual mode it has no effect on wire framing (one request per record either way) and is accepted only for parity.

Pass 0 to opt out of re-chunking in Array mode — the entire records slice handed to write_batch is sent as a single JSON array, preserving upstream StreamPage framing.

Trait Implementations§

Source§

impl Clone for HttpSinkConfig

Source§

fn clone(&self) -> HttpSinkConfig

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for HttpSinkConfig

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl<'de> Deserialize<'de> for HttpSinkConfig

Source§

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>
where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer. Read more
Source§

impl JsonSchema for HttpSinkConfig

Source§

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema. Read more
Source§

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type. Read more
Source§

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type. Read more
Source§

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword. Read more
Source§

impl Serialize for HttpSinkConfig

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

impl<T> DeserializeOwned for T
where T: for<'de> Deserialize<'de>,