# Codegen design notes: resource-shaped vs. generic APIs
This document explains how `olai-codegen` handles APIs that don't fit the standard
resource/CRUD shape, the "flat binding lowering" that makes them work, and the
larger refactor we deliberately deferred.
## Background: the resource bias
The pipeline (`parsing → analysis → codegen`) was originally built around
`google.api.resource` + standard CRUD (AIP-131…135). Methods are classified into
`RequestType::{List, Create, Get, Update, Delete, Custom(Pattern)}`
(`analysis/types.rs`), and services that manage a resource get convenient,
resource-scoped clients.
This convenience is intentional and language-specific:
- **Rust** uses a resource-scoped client (e.g. `CatalogClient` bound to a name) plus
per-method **builders** (`builder.rs`) with `with_*()` setters and `IntoFuture`.
Builders are the idiomatic Rust shape.
- **Python / Node / TypeScript** bindings wrap that client but should read
idiomatically *for their language* — named params, defaults, keyword args — rather
than mechanically mirroring the Rust builders.
The aim is easy-to-use, idiomatic code in **every** language. The resource scoping is
a convenience layer, not a requirement.
## The problem: non-resource APIs
APIs that don't fit the resource mold broke the bindings:
- **Entity tag assignments** — resource-less, keyed by a composite path
`(entity_type, entity_name, tag_key)`.
- **Temporary credentials** — resource-less factory RPCs.
- **Delta-commit-style RPCs** — custom verbs that return `google.protobuf.Empty`.
The Rust client (`client.rs`) already handled all of these (it passes every param
and builds the URL per call). The breakage was isolated to the **language bindings**,
which modeled every service as a resource-scoped client and:
1. **Dropped path params** (`inner_resource_client_call` filtered out
`is_path_param()`), expecting a scoped client to supply them. Resource-less
services have no scoped client, so composite-key methods lost their keys.
2. **Mishandled `Empty` returns** — `PyUnityCatalogResult` was emitted without its
`<()>` type argument (uncompilable).
The previous mitigation, `ServiceHandler::skip_bindings()`, simply emitted **no**
Python/Node bindings for such services. That has been removed.
## The fix: flat binding lowering
The model (per the original design intent): resource-less services are treated like
**collection APIs that live on the root client**. There is no resource to bind to, so
their methods sit on the aggregate/root client and accept every param directly.
This is captured by `BindingMode` (`codegen/mod.rs`):
| `Scoped` | service has `google.api.resource` | unchanged: per-service scoped client (`get`/`update`/`delete`), collection methods + resource accessor on the root client; path params supplied by the scoped client |
| `Flat` | resource-less service | **all** methods emitted on the root client, passing **every** param incl. path params; no scoped client/module |
`ServiceHandler::is_resource_scoped()` / `binding_mode()` select the mode. In `Flat`
mode the binding emitters:
- do not filter out path params (`resource_method_parameters` / the `*_method_parameters`
helpers and `inner_resource_client_call` keep them);
- call the **base** client method (`MethodPlan::base_method_ident`), not the scoped
`get`/`update` verb;
- route get/update/delete/custom onto the root client (`flat_client_method`).
The lowering reuses the `RequestParam` list that analysis already computed from the
URL template (path/query/body, in template order) — it does **not** invent a parallel
resource-based param derivation.
`Empty` returns are handled per-emitter from `MethodHandler::output_type()` being
`None`: the Python emitter (`PyOutput::Unit`) returns `PyResult<()>` from blocking methods
and resolves awaitables to `None`; the TS/Node emitters likewise yield `Promise<void>` /
`napi::Result<()>`.
### Generated Rust aggregate root client
The language bindings call a Rust **aggregate root client** (e.g. `UnityCatalogClient`)
via `self.client.<method>(...)`. That client used to be hand-written in the consuming
crate and went stale whenever a new service/method was added. It is now **generated**
(`codegen/aggregate.rs`) into the client output as a top-level `client.rs`, named from
`BindingsConfig::aggregate_client_name`. It is emitted only when `bindings` is set.
The aggregate is generated by **mirroring the Python/Node aggregate dispatch**
(`python::bindings::collection_client_struct` / `generate_resource_accessor_method`),
so the Rust method surface and the binding call sites agree **by construction**:
- one field per service holding the generated low-level per-service client
(`crate::codegen::<base_path>::<ClientType>`), built in `new(client, base_url)`;
- **scoped** services contribute their collection methods (`list_*`, `create_*`,
factory RPCs) as builder constructors, plus a resource accessor
`<singular>(params…) -> <Singular>Client` (and `<singular>_from_full_name` when the
resource has a composite `name_field`);
- **flat** (resource-less) services contribute *every* method as a flat builder
constructor passing all params including path params.
The aggregate is **mechanical**: bespoke ergonomic helpers (custom stream signatures,
delegation shortcuts, low-level `*_client()` accessors) stay hand-written in a separate
`impl` block in the consuming crate, which re-exports the generated aggregate.
**Consuming-crate conventions the generated aggregate relies on:**
- Hand-written **scoped clients** (`CatalogClient`, `SchemaClient`, …) must follow one
constructor convention: `<Singular>Client::new(<accessor params…>, <service client>)`,
and `<Singular>Client::new_from_full_name(full_name, <service client>)` when the
resource uses a composite `name_field`. (At time of writing, unitycatalog-rs's
`TableClient` was missing `new_from_full_name` — the lone outlier to normalize.)
- Scoped clients must be re-exported at the crate root (the aggregate references them as
`crate::<Singular>Client`).
- The consuming crate's `lib.rs` should `pub use crate::codegen::client::<Aggregate>`
and keep only bespoke extension methods, rather than hand-defining the aggregate.
### Method naming (annotation-driven)
`MethodHandler::binding_method_name[_str]()` prefers the gnostic
`gnostic.openapi.v3.operation` `operation_id` (snake-cased) for the emitted binding
method name, falling back to the proto-derived name. This affects only the *emitted*
binding name; the underlying Rust client call still uses `base_method_ident`, so the
handler/client API is unchanged. The same source is used in Python, Node-NAPI, and
TypeScript so the three layers stay name-consistent (NAPI snake→camel matches the TS
camelization). We deliberately do **not** use gnostic `tags` for grouping — in
practice tags mirror the service name, which we already know.
## URL-template parsing (correctness)
The `google.api.http` template is the authoritative path→field map:
`extract_request_fields` (`analysis/mod.rs`) walks the template's placeholder names
and matches each against request fields by name. The parser (`parsing/http.rs`) now
handles gRPC-transcoding placeholder syntax:
- **Segment bindings** `{name=catalogs/*}` — the `=pattern` suffix is stripped; the
field bound is `name`. **Supported.**
- **Dotted nested paths** `{table.catalog_name}` — **not yet supported**; the
placeholder is preserved verbatim (so the URL stays intact) and is not surfaced as a
param, avoiding broken identifiers downstream. Full support would require nested
field resolution in analysis, `request.a.b` access in the Rust client URL builder,
and matching binding signatures. Tag-assignment/credential APIs use flat composite
keys, not dotted paths, so this is a low-priority gap.
## Known limitations / current behavior
- **Resource-less standard verbs are classified `Custom`.** `RequestType` derivation
requires `google.api.resource` matching for List/Get/Delete, so a resource-less
`ListTagAssignments`/`DeleteTagAssignment` is `Custom`, not `List`/`Delete`. The flat
emitters treat these as unbound operations returning the full response message (e.g.
the binding returns `ListTagAssignmentsResponse` rather than a streamed item list).
This is correct and usable; whether to give resource-less collections streamed
semantics is a future refinement.
- Dotted nested path placeholders (above).
## IR enrichment (done)
The "deferred refactor" below has largely landed. Method *intent* used to be re-derived
from the CRUD `RequestType` enum (plus method-name heuristics) at ~15 emit sites across the
Rust emitters and the three language bindings, and those derivations had drifted. Intent is
now computed **once** in `analysis/` and consumed by every emitter:
- **Derived intent fields on `MethodPlan`** (`analysis/types.rs`), computed in `analyze_method` /
`analyze_service`:
- `MethodShape { CollectionScoped, InstanceScoped, Unbound }` — where a method sits relative
to a resource, folding the old `is_collection_method()` + `BindingMode` logic into one field.
- `has_request_body`, `needs_request_parts`, `scoped_verb` — scalars that replace the
`RequestType` re-matching in `client.rs` (body attach), `server.rs` (extractor selection +
`RequestPartsExt` import), and `MethodPlan::resource_client_method` (scoped verb).
- **`EmitShape { List, Create, GetUpdate, Delete }`** (`analysis::emit_shape`) is the single
source for the per-binding emit dispatch. Python/Node/TypeScript each used to re-`match
request_type` to pick a list/create/get-update/delete shape; they now branch on
`MethodHandler::emit_shape()`, with emittability guards (`is_scoped_instance_method`, the
collection-shape check on the root client) preserving the exact prior per-client surface.
- **Name heuristics in one place.** The `List`/`Generate` proto-name conventions live only in
`analysis::is_collection_method`; emitters call it rather than re-implementing the prefix checks.
- **Resource accessor params on `ServicePlan`.** `resource_accessor_params` is derived once
(Phase 2 of `analyze_metadata`) and rendered by all four accessor emitters (Rust aggregate +
Python + Node + TypeScript) via the shared `AccessorSpec`, replacing the old
`derive_resource_accessor_params` that was called at four emit sites with a now-converged
nesting rule (`params.len() > 1`).
- **Shared param filtering.** `MethodHandler::param_plan{,_split}` apply the drop-path / drop-
`page_token` filtering once; the per-language emitters render the result and layer only their
language-required transforms (pyo3 signature, NAPI capability filter + `Buffer`, TS camelCase).
**Bug fixed in passing:** the accessor heuristic used to scoop standard AIP-132 pagination fields
(`page_token`, `max_results`, …) into a flat resource's accessor params (e.g.
`catalog(page_token, catalog_name)`). The single derivation now excludes them via
`analysis::is_standard_list_field` (shared with the resource-enum path), so a flat resource yields
`catalog(catalog_name)`.
**Determinism fix:** `analyze_metadata` now sorts services by name before building plans;
previously the `HashMap` iteration order made emitted module declarations non-reproducible.
**Regression oracle:** `tests/golden_integration.rs` snapshots the full generated tree (Rust +
Python + Node + TypeScript) for the proto fixtures and diffs byte-for-byte against committed
golden files. Re-bless intended output changes with
`BLESS=1 cargo test -p olai-codegen --test golden_integration`.
### Evaluated, not done
- **`OutputShape` enum.** Considered, but the Empty/response/streamed-item handling is already DRY:
every path funnels through `MethodHandler::output_message()` (the single `Empty` detection point)
via `output_type()` / `has_response` / `list_output_field()`. There is
no re-derived logic to collapse, so an enum would be churn without benefit.
- **Classify resource-less standard verbs.** A resource-less `ListTagAssignments` is still
`RequestType::Custom` (List/Get/Delete classification requires `google.api.resource`), and the
flat emitters return the full response message rather than a streamed item list. Giving
resource-less collections streamed semantics remains a future refinement.
- **Reduce `ManagedResource` to a presentation concern** — `MethodShape` makes the scoped/flat
decision explicit, but `ManagedResource` is still the load-bearing source of resource presence;
fully demoting it to presentation is not yet done.
- **Config escape hatch (`service_overrides`)** — still not needed; annotations (resource, http,
gnostic operation) cover every case seen so far. Don't build it speculatively.