Skip to main content

Module spark

Module spark 

Source
Expand description

Apache Spark monitoring REST API server.

Serves the Spark monitoring endpoints a client / the History Server UI hits (/api/v1/applications, .../{id}/jobs, /stages, /executors). The LLM roleplays the application’s control plane, inventing applications/jobs/stages/executors per request.

Static vs LLM-driven:

  • GET /api/v1/version — mechanical version banner, answered statically (no LLM).
  • Unrecognised paths get a static 404 (plain text), no LLM call.
  • applications, jobs, stages, executors — LLM-driven.

Fail-closed: on an LLM error, or when the model produces no spark_response, the server answers 503/500 with a JSON error object — never a success-shaped empty array (which a client cannot distinguish from a genuinely empty application list).

Modules§

actions
Apache Spark monitoring REST API (/api/v1/...) protocol actions and event types.

Structs§

SparkServer
Apache Spark monitoring REST server.

Constants§

MAX_REQUEST_BYTES
Largest request body this server will buffer.