Module flink 

An Apache Flink JobManager, optionally paired with a companion TaskManager via FlinkContainer::with_task_manager for a real session cluster that can actually run jobs (a bare JobManager has zero task slots and can only accept/reject submissions, never execute them).

Topology

A real Flink session cluster is two processes that must find each other over a persistent bidirectional RPC connection (Pekko/Akka remoting): the TaskManager dials the JobManager’s RPC port (6123) at boot and stays connected — registration is not a one-shot request/response, it is the first message on a connection the TaskManager keeps open for as long as it runs, carrying heartbeats and slot offers/task deployments both ways for the cluster’s whole lifetime. FlinkContainer::with_task_manager puts both containers on one internally-created rightsize::Network, aliases this JobManager as "jobmanager", and sets FLINK_PROPERTIES=jobmanager.rpc.address: jobmanager on both containers (the image’s own env-driven config mechanism) — not just the TaskManager. Verified directly: setting it on the TaskManager alone leaves the JobManager’s own Pekko actor system bound under its container hostname rather than the alias, so every registration attempt from the TaskManager (correctly dialing pekko.tcp://flink@jobmanager:6123) gets silently dropped as a non-local recipient (dropping message ... arriving at [pekko.tcp://flink@jobmanager:6123] inbound addresses are [pekko.tcp://flink@<container-id>:6123]) — the JobManager must be told its own address is the alias too.

Backend support — full on docker, JobManager-only on msb

Docker: verified end-to-end — the TaskManager registers with the JobManager (Successful registration at resource manager ... in its own log) and GET /taskmanagers on the JobManager’s REST port shows one slot-bearing TM within seconds of both containers starting.

microsandbox: FlinkContainer::with_task_manager returns rightsize::RightsizeError::UnsupportedByBackend before ever booting anything. The actual blocker is more basic than a Pekko/tunnel incompatibility: msb’s install_network_links requires nc/busybox inside the consumer image to serve the tunnel’s in-guest listener, and the official flink:1.20.5 image is a bare JRE + Flink install with neither — the attempt failed immediately with UnsupportedByBackend: network links (no nc/busybox in consumer image 'flink:1.20.5'), thrown from the msb backend’s own nc prerequisite probe before a single byte of Pekko traffic could be exchanged. Whether Pekko’s persistent-connection RPC registration would work over the tunnel’s single-connection-at-a-time model was never reached or tested — the missing nc/busybox prerequisite stops the attempt before that question is even in play. A bare JobManager (REST /overview only, no TM) works fine on msb — it needs no network-link emulation at all, just the ordinary published-port HTTP path — so this module still supports msb for JobManager-only use; only FlinkContainer::with_task_manager is gated.

Memory — JVM, ladder applies to both roles

A JobManager settles around ~310 MiB RSS and a TaskManager around ~375 MiB RSS at rest on docker with no cap (docker stats, real boot) — both comfortably over msb’s ~450 MB default individually, and this module runs the JobManager on msb too (see above), so with_memory_limit(1024) is this module’s default for both roles, matching the family’s established single-JVM floor (crate::keycloak::KeycloakContainer, crate::neo4j::Neo4jContainer).

No control characters were found in the image’s baked env (checked via docker image inspect).

Structs

FlinkContainer

An Apache Flink JobManager, with an optional companion TaskManager (see FlinkContainer::with_task_manager).

FlinkGuard

The running guard for a FlinkContainer.