pub struct HttpSinkConfig {
pub url: String,
pub method: Method,
pub headers: HeaderMap,
pub auth: AuthSpec<HttpSinkAuth>,
pub batch_mode: HttpBatchMode,
pub max_retries: usize,
pub concurrency: usize,
pub batch_size: usize,
}Expand description
Configuration for the HTTP sink connector.
Fields§
§url: StringTarget endpoint URL.
method: MethodHTTP method (default: POST).
headers: HeaderMapAdditional request headers.
auth: AuthSpec<HttpSinkAuth>Authentication: either inline ({ type, config }) or a { ref: <name> }
pointer to a shared provider in the CLI’s top-level auth: catalog.
batch_mode: HttpBatchModeHow to batch records in requests.
max_retries: usizeNumber of retries on transient failures (default: 0).
concurrency: usizeMaximum number of concurrent requests in Individual mode (default: 10).
batch_size: usizeMaximum number of records sent per outbound HTTP request. Defaults to
DEFAULT_BATCH_SIZE (1000).
Interpretation depends on batch_mode:
- In
HttpBatchMode::Arraymode,write_batchre-chunks the incoming slice intobatch_size-row chunks and issues one POST request per chunk, with each request body a JSON array of up tobatch_sizerecords.batch_size = 0is the “no batching” sentinel: the entire upstreamStreamPageis forwarded as a single JSON array — useful when the source already chunks to a size the destination endpoint accepts. - In
HttpBatchMode::Individualmode the sink already sends one request per record, sobatch_sizehas no effect on wire framing; the field is accepted for config-shape parity with other sinks and validated viafaucet_core::validate_batch_sizeat load time.
Recommended value for HTTP POST endpoints that accept arrays: match the destination’s documented batch limit (commonly 100–1000 records per request).
Implementations§
Source§impl HttpSinkConfig
impl HttpSinkConfig
Sourcepub fn new(url: impl Into<String>) -> Self
pub fn new(url: impl Into<String>) -> Self
Create a new config with the given URL and sensible defaults.
Sourcepub fn auth(self, auth: HttpSinkAuth) -> Self
pub fn auth(self, auth: HttpSinkAuth) -> Self
Set the authentication method (inline).
Sourcepub fn batch_mode(self, mode: HttpBatchMode) -> Self
pub fn batch_mode(self, mode: HttpBatchMode) -> Self
Set the batch mode.
Sourcepub fn max_retries(self, retries: usize) -> Self
pub fn max_retries(self, retries: usize) -> Self
Set the maximum number of retries.
Sourcepub fn concurrency(self, concurrency: usize) -> Self
pub fn concurrency(self, concurrency: usize) -> Self
Set the maximum number of concurrent requests in Individual mode.
Sourcepub fn with_batch_size(self, batch_size: usize) -> Self
pub fn with_batch_size(self, batch_size: usize) -> Self
Set the maximum number of records sent per outbound HTTP request.
In HttpBatchMode::Array mode this controls how many records are
packed into each POST body. In HttpBatchMode::Individual mode it
has no effect on wire framing (one request per record either way)
and is accepted only for parity.
Pass 0 to opt out of re-chunking in Array mode — the entire
records slice handed to write_batch is sent as a single JSON
array, preserving upstream StreamPage framing.
Trait Implementations§
Source§impl Clone for HttpSinkConfig
impl Clone for HttpSinkConfig
Source§fn clone(&self) -> HttpSinkConfig
fn clone(&self) -> HttpSinkConfig
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for HttpSinkConfig
impl Debug for HttpSinkConfig
Source§impl<'de> Deserialize<'de> for HttpSinkConfig
impl<'de> Deserialize<'de> for HttpSinkConfig
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl JsonSchema for HttpSinkConfig
impl JsonSchema for HttpSinkConfig
Source§fn schema_id() -> Cow<'static, str>
fn schema_id() -> Cow<'static, str>
Source§fn json_schema(generator: &mut SchemaGenerator) -> Schema
fn json_schema(generator: &mut SchemaGenerator) -> Schema
Source§fn inline_schema() -> bool
fn inline_schema() -> bool
$ref keyword. Read more