Crate magnetar

Expand description

Apache Pulsar client driver for Rust.

Public façade for the magnetar workspace. Re-exports the sans-io core (magnetar_proto) plus the selected runtime engine, and provides an ergonomic PulsarClient entry point that wires the protocol layer to the tokio engine by default.

use magnetar::{OutgoingMessage, PulsarClient};

let client = PulsarClient::builder()
    .service_url("pulsar://localhost:6650")
    .build()
    .await?;

let producer = client.producer("persistent://public/default/orders").create().await?;
producer
    .send(OutgoingMessage::with_payload(b"hello".as_slice()).into())
    .await?;

let consumer = client
    .consumer("persistent://public/default/orders")
    .subscription("worker")
    .subscribe()
    .await?;
let msg = consumer.receive().await?;
consumer.ack(msg.message_id).await?;

§Feature flags

tokio (default): pull in the tokio engine.
moonpool: pull in the moonpool engine.
admin: re-export [magnetar_admin] under [admin] for the REST admin client.
auth-oauth2, auth-sasl, auth-athenz: pluggable auth providers.
encryption: PIP-4 end-to-end encryption.
opentelemetry (default off): inject/extract W3C traceparent/tracestate into message properties (ADR-0053).

Re-exports§

pub use magnetar_proto as proto;
pub use magnetar_runtime_tokio as runtime_tokio;

Structs§

Backoff: Truncated-exponential backoff with deterministic jitter.
ClientBuilder: Builder for PulsarClient.
ConnectionConfig: Connection configuration.
ConsumerBuilder: Builder for a consumer.
ConsumerEventListenerHandle: Owns the background poller task driving a ConsumerEventListener. Structurally identical to MessageListenerHandle — dropping the handle aborts the poller; Self::close awaits a clean stop.
ConsumerHandle: A consumer id, allocated by the Connection when a subscription opens.
IncomingMessage: Convenience alias for an incoming message handed back to the caller.
JavaStringHashHasher: Pick the partition with java_string_hash (Java String.hashCode() semantics), then hash % partitions. Falls back to round-robin when no key is set.
MemoryLimit: Java parity: configured global publish memory budget. Stored verbatim on crate::ClientBuilder and exposed to consumers via PulsarClient::memory_limit.
MessageBuilder: Producer-bound counterpart to OutgoingMessage. Mirrors Java’s TypedMessageBuilder — the producer is captured at construction so the terminal send() has no extra argument.
MessageId: A logical message identifier (ledger / entry / batch / partition).
MessageListenerHandle: Owns the background poller task driving a push-mode consumer. Mirrors the lifetime semantics of crate::TableView’s drain task: dropping the handle aborts the poller; Self::close awaits a clean stop.
MultiTopicsConsumer: Multi-topics consumer. Each contained consumer subscribes to one topic; receive() returns the next message across the whole set.
MultiTopicsConsumerBuilder: Builder for MultiTopicsConsumer. Mirrors org.apache.pulsar.client.api.ConsumerBuilder at the multi-topic layer.
MultiTopicsMessage: A message yielded by MultiTopicsConsumer::receive, carrying the topic it came from.
Murmur3HashHasher: Pick the partition by hashing the message’s UTF-8-encoded partition key with murmur3_32_hash (Apache Pulsar Murmur3_32Hash, seed 0), then hash % partitions. Falls back to round-robin via OutgoingMessage::key being None or empty.
NoEncryption: Zero-sized stub for any future engine that genuinely cannot wire real encryption. Both shipped engines (TokioEngine and [MoonpoolEngine]) now resolve their MessageEncryptorApi::Encryptor / MessageDecryptorApi::Decryptor to their own runtime’s Arc<dyn …MessageEncryptor> / …MessageDecryptor, so NoEncryption is no longer used by either. It is retained as the documented opt-out type an engine can hand to the façade to signal “encryption not supported on this engine” — the builders’ generic .create() / .subscribe() paths ignore the encryptor field regardless.
OperationRetryConfig: Configuration for lookup, partition-metadata, producer-open, and subscribe retries.
OutgoingMessage: Convenience alias for outgoing application messages.
PartitionedConsumerBuilder: Builder for a partition-aware consumer.
PartitionedMessageBuilder: Partitioned-producer-bound counterpart to crate::MessageBuilder. Same chained setters; the terminal .send().await resolves the partition and dispatches.
PartitionedProducer: Partition-aware producer.
PartitionedProducerBuilder: Builder for PartitionedProducer. Mirrors Java’s ProducerBuilder at the partitioned layer.
PatternConsumer: Regex-pattern consumer. Holds one consumer per matching topic and reconciles the set against PIP-145 deltas on update().
PatternConsumerBuilder: Builder for PatternConsumer. Mirrors Java’s PulsarClient#newConsumer().topicsPattern(...).
PatternMessage: A message yielded by PatternConsumer::receive, carrying the topic it came from.
ProducerBuilder: Builder for a producer.
ProducerHandle: A producer id, allocated by the Connection when a producer opens.
PulsarClient: High-level Pulsar client, generic over the runtime Engine.
Reader: Reader handle — a non-durable consumer that reads from a topic without persisting an acknowledgement cursor. Use a reader for: log replay, message inspection, batch ETL, or anywhere you want at-most-once delivery semantics that the broker doesn’t track.
ReaderBuilder: Builder for a Reader.
ReconcileReport: Outcome of a single PatternConsumer::update reconciliation cycle.
RequestId: A protocol-level request id, monotonically increasing per connection.
SequenceId: A monotonic per-producer publish sequence id.
SupervisorConfig: Configuration for the auto-reconnect supervisor.
TableView: Compacted-topic key/value view.
TableViewBuilder: Builder for a TableView. Mirrors org.apache.pulsar.client.api.TableViewBuilder.
TokioEngine: Zero-sized marker for the tokio production engine. Default E on crate::PulsarClient<E>.
TopicListChange: PIP-145 TopicListChanged delta surfaced through BrokerMetadataApi::poll_topic_list_change. Façade-side analogue of the per-runtime TopicListChange structs — each runtime impl converts its own delta into this engine-agnostic shape so generic surfaces (PatternConsumer<C>::update) can reconcile without touching runtime-specific types.
Transaction: A live Pulsar transaction token. Holds the broker-assigned magnetar_proto::TxnId.
TypedConsumer: A schema-aware consumer. Wraps a consumer and decodes every received payload with the configured schema before returning to the caller.
TypedConsumerBuilder: Builder for a TypedConsumer.
TypedMessage: A decoded message yielded by TypedConsumer::receive.
TypedMessageBuilder: Schema-aware counterpart to crate::MessageBuilder. Captures a &TypedProducer and lets callers chain Java-style: producer.new_message().key(..).value(&typed).send(). The schema runs on .send(&value) so we don’t pay the encode cost on values that get dropped mid-build (a logic error caught by the borrow checker, but cheap to be defensive about).
TypedProducer: A schema-aware producer. Wraps a producer and applies the configured schema to every outbound value.
TypedProducerBuilder: Builder for a TypedProducer. The schema is required; the topic comes from the parent PulsarClient::typed_producer entry point.
TypedTableView: Schema-aware TableView. Wraps a raw TableView plus an Arc<S> and exposes typed accessors that decode the payload on demand. Mirrors Java’s pulsar.tableView(Schema) shape.
TypedTableViewBuilder: Builder for a TypedTableView. Mirrors Java’s schema-aware pulsar.tableViewBuilder(Schema) shape.

Enums§

ConnectionEvent: A semantic event surfaced by the state machine.
ConsumerEvent: Event surfaced by a push-delivery ConsumerEventListener (issue #348). Mirrors Java ConsumerEventListener#becameActive(Consumer, int) / becameInactive(Consumer, int) — the Failover subscription active-consumer transitions. magnetar drops the partitionId argument (single-topic consumers only; a partitioned consumer’s per-partition event listener is attached per child, so the topic/partition is already implicit in which consumer’s listener fired).
MemoryLimitPolicy: Java parity: org.apache.pulsar.client.api.MemoryLimitPolicy.
MessageRoutingMode: How a PartitionedProducer picks the partition for an outgoing message.
OpOutcome: Result of consuming a pending op via Connection::take_outcome.
ProtocolError: Errors that the Connection state machine can surface.
PulsarError: Top-level errors surfaced by the façade.
SeekTarget: Per-topic seek target supplied by the closure passed to crate::MultiTopicsConsumer::seek_per_partition (and the equivalent on crate::PartitionedConsumer). Mirrors Java’s Consumer#seek(Function<String, Object>), where the function returns either a MessageId or a Long publish-time millis-since-epoch.
TxnState: Result of committing or aborting a Transaction. Re-exported from magnetar-proto. Lifecycle of a transaction tracked by TxnClient.

Traits§

AuthProvider: Synchronous, sans-io authentication provider.
BrokerMetadataApi: Engine-side broker metadata lookups used by crate::PartitionedConsumerBuilder and crate::PatternConsumerBuilder (alongside other partition-aware surfaces). Each runtime implements this on its concrete Client type.
ConsumerApi: Pulsar consumer wire surface — implemented by each runtime on its Consumer type. Foundational alongside ProducerApi per ADR-0026 §D1.
ConsumerInterceptor: Java ConsumerInterceptor SPI. Plug receive-side hooks behind Consumer::receive to inspect / mutate incoming messages and observe ack outcomes. Mirrors org.apache.pulsar.client.api.interceptor.ConsumerInterceptor:
CreateProducerApi: Engine-side producer-creation surface used by ProducerBuilder<E> and PartitionedProducer<E>. Same shape as SubscribeApi for the producer side.
Engine: Marker trait labelling a runtime engine. Implementations select the concrete storage type (Self::ClientState) that backs the engine’s branch of crate::PulsarClient<E>.
MessageDecryptorApi: Engine-side message-decryptor selection. Mirror of MessageEncryptorApi for the consume path. Implemented on the engine marker.
MessageEncryptorApi: Engine-side message-encryptor selection. Each engine declares its own concrete encryptor type; the façade’s ProducerBuilder stores Option<E::Encryptor> (engine-typed) instead of an Arc<dyn magnetar_runtime_tokio::MessageEncryptor> (tokio-locked).
MessageRouter: Plug a user-provided routing function in front of MessageRoutingMode. Mirrors Java’s MessageRouter SPI — when set on the builder, the function decides the partition for every outgoing message; the configured MessageRoutingMode is ignored. Use this for affinity routing rules (geo, tenant, schema-keyed) that don’t fit the partition-key-hash mould.
ProducerApi: Pulsar producer wire surface — implemented by each runtime on its Producer type. Foundational for the seven dependent façade lifts (Reader, TypedSchemas, MultiTopicsConsumer, PartitionedProducer, PartitionedConsumer, PatternConsumer, TableView) per ADR-0026 §D1.
ProducerExt: Extension trait that gives magnetar_runtime_tokio::Producer the Java-symmetric producer.new_message().key(..).value(..).send().await entry point.
ProducerInterceptor: Java ProducerInterceptor SPI. Plug pipeline hooks in front of Producer::send to inspect, mutate, or react to outgoing messages. Mirrors the Java org.apache.pulsar.client.api.interceptor.ProducerInterceptor interface — eligible gates whether the interceptor runs for a given message, before_send runs first (mutating the OutgoingMessage), and on_send_acknowledgement fires after the broker acks the publish (or the send errors out).
SubscribeApi: Engine-side subscribe surface used by ConsumerBuilder<E> and the other consumer-spawning façade surfaces (MultiTopicsConsumer, PatternConsumer, Reader). Each runtime implements this on its concrete Client type with the runtime-specific Consumer type surfaced via the associated Consumer type.
TransactionApi: Pulsar transactions (PIP-31) — implemented by each runtime on its Client type. Phase 1 of the D1 lift train.
WrapperReceiver: A wrapper consumer’s receive() surface, abstracted for the wrapper poller.

Functions§

ack_cumulative_with_interceptors: Cumulative ack variant of ack_with_interceptors. Notifies via on_acknowledge_cumulative instead of on_acknowledge.
ack_with_interceptors: Ack via consumer and notify every interceptor of the outcome. Mirrors Java’s post-ack callback chain. Returns whatever the runtime ack returned, mapped into a PulsarError.
java_string_hash: Bit-for-bit port of String.hashCode() & Integer.MAX_VALUE. Iterates over UTF-16 code units (matching Java’s char) so non-BMP code points hash identically to the JDK. ASCII strings short-circuit through the byte path.
murmur3_32_hash: Bit-for-bit port of Apache Pulsar’s Murmur3_32Hash.makeHash(byte[]) (Murmur3_32Hash.java). Used by Murmur3HashHasher so cross-language consumers (Java, C++, Go) see identical routing for the same key.
receive_with_interceptors: Receive the next message via consumer, running every ConsumerInterceptor in interceptors against the payload before it is returned. Mirrors Java’s interceptor chain on the receive path — every interceptor’s before_consume runs in order on a single progressively-mutated message.
send_with_interceptors: Send msg through producer, running every eligible ProducerInterceptor in interceptors in order. Mirrors Java’s interceptor-chain semantics: eligible is evaluated against the original message, before_send runs in order on a single message the chain progressively mutates, and on_send_acknowledgement fires on every eligible interceptor regardless of whether the broker accepted the publish.
spawn_consumer_event_listener: Attach a ConsumerEventListener to an already-subscribed consumer, returning the owning ConsumerEventListenerHandle. The poller drives consumer.next_active_change() in a loop and invokes listener once per transition — ConsumerEvent::BecameActive for Ok(true), ConsumerEvent::BecameInactive for Ok(false) — stopping cleanly the first time the future resolves Err (closed / terminally-disconnected consumer).
spawn_message_listener: Attach a push-delivery listener to an already-subscribed consumer, returning the owning MessageListenerHandle. The poller drives consumer.receive() and invokes listener once per message, sequentially and in order, with no auto-ack — the callback acks explicitly.
spawn_wrapper_message_listener: Spawn a push-delivery poller over a wrapper consumer, returning the owning MessageListenerHandle. The poller drives receiver.wrapper_receive() and invokes listener(topic, &msg) once per message, sequentially and in order, with no auto-ack — the callback acks explicitly via the wrapper’s topic-routed ack (ack(topic, id)).

Type Aliases§

ConsumerEventListener: Callback fired for every Failover active/standby transition observed by a push-mode consumer event listener. Mirrors MessageListener’s shape — a synchronous callback, invoked sequentially from the poller task. Mirrors Java ConsumerEventListener#becameActive / #becameInactive, collapsed into one callback taking ConsumerEvent (no consumer/partition argument — hold a clone of your consumer in the closure if you need to act on it, the same convention MessageListener uses).
MessageListener: Callback fired for every message delivered to a push-mode consumer.
OpenProducerFut: Helper alias: CreateProducerApi::open_producer future return type.
PartitionedConsumer: Partition-aware consumer. Effectively a crate::MultiTopicsConsumer whose topic list was auto-discovered from a partitioned topic.
ReceiveBatchFut: Helper alias: ConsumerApi::receive_batch / receive_batch_with_bytes_cap future return type.
ReceiveOptFut: Helper alias: ConsumerApi::receive_with_timeout future return type.
SubscribeFut: Helper alias: SubscribeApi::subscribe future return type.
TableViewListener: Callback fired for every mutation applied to the table view.
TypedMessageListener: Schema-aware push-delivery callback (Java ConsumerBuilder<T>#messageListener). Fired once per delivered message with the decoded TypedMessage. Like the raw crate::MessageListener, it runs inside the poller task — sequentially, in order — and must ack explicitly (the poller never auto-acks). Register it via TypedConsumerBuilder::message_listener and subscribe via TypedConsumerBuilder::subscribe_with_listener.
WatchTopicListFut: Helper alias: BrokerMetadataApi::watch_topic_list future return type.
WrapperMessageListener: Callback fired for every message delivered to a push-mode wrapper consumer (crate::MultiTopicsConsumer, crate::PartitionedConsumer, crate::PatternConsumer).

Crate magnetar

Crate magnetar Copy item path

§Feature flags

Re-exports§

Structs§

Enums§

Traits§

Functions§

Type Aliases§

Crate magnetar