Expand description
The Python CST as the other end of a rule. Spec §3.2.
The counterpart of crate::adapter::java, crate::adapter::cpp
and crate::adapter::c. Reading two of them side by side is the
point: where they differ, the difference is the language’s, and it is
written down here rather than spread through a renderer.
Values are resolved from the model, never from the overlay — the same rule the sibling materialisers follow and for the same reason (spec §2.3).
§Python is the hardest of the five, and this is why
The other four target languages state a width, a length and a
signedness in the declaration itself. Python states none of them. A
long is 32 bits and a Python int is arbitrary precision; an
IDL octet and an IDL unsigned long long are the same Python type;
long a[3] and sequence<long> a are both “a thing you can index”.
A backend that wrote x: int and stopped would produce a file that
type-checks and a value the wire cannot hold, with nothing between
the two to notice.
So this column is arranged around one question: for each IDL fact Python cannot enforce, where is it written down? There are exactly two answers used here and no third.
- The Python type, where Python can enforce it. Mutability
(
listagainsttuple), arity (tuple[T, T, T]), absence (typing.Optional[T]), and which class a member holds. A static checker rejects a program that gets these wrong. typing.Annotatedmetadata, where it cannot. The width, the signedness, the bound, the digit count, the character width. A checker passes these through untouched; they are read back at run time withtyping.get_type_hints(cls, include_extras=True), which is what the codec pass will do.
Nothing is left implicit. x: int never appears in generated output.
§The integer mapping
| IDL | Python | metadata |
|---|---|---|
boolean | bool | — |
char | str | _IdlText(bits=8, bound=1) |
wchar | str | _IdlText(bits=16, bound=1) |
int8 | int | _IdlInt(bits=8, signed=True) |
octet, uint8 | int | _IdlInt(bits=8, signed=False) |
short | int | _IdlInt(bits=16, signed=True) |
unsigned short | int | _IdlInt(bits=16, signed=False) |
long | int | _IdlInt(bits=32, signed=True) |
unsigned long | int | _IdlInt(bits=32, signed=False) |
long long | int | _IdlInt(bits=64, signed=True) |
unsigned long long | int | _IdlInt(bits=64, signed=False) |
float | float | _IdlFloat(bits=32) |
double | float | _IdlFloat(bits=64) |
long double | str | _IdlFloat(bits=128) |
Each of those is written once as a module-level alias — _Int32,
_UInt64 — so a field reads x: _Int32 and the width is one hop
away rather than inline on every line.
An alias exists exactly where the type’s whole description is fixed.
A string<7> and a string<9> are not the same type, so the string
family is written inline instead — including the unbounded one, so
that string and string<7> have one spelling between them rather
than a name for the common case and a form for the rest.
Where this departs from every other column. Java has to decide
which of its signed types is wide enough to hold an unsigned IDL
value, and unsigned long long ends up a long carrying a bit
pattern rather than a number. Python has no such problem and so
makes no such trade: every one of the ten integers is a Python int
holding the number, and only the metadata differs. That is the one
place where Python’s lack of widths is an advantage, and it is worth
saying because everywhere else it is the cost.
§long double is a str and never a float
A Python float is a C double — 53 bits of significand. An IDL
long double is a binary128 with 113. Writing one into a float
loses sixty bits at the assignment, before any codec runs, with
nothing to say it happened. That is the silent-substitution class
this project is arranged against, and it is the same defect the
three-column gate found in the C column when long double reached
double there.
So the member holds the decimal text, exactly as fixed<P,S>
does and for the same reason: it is the one Python type that holds
the value the source wrote without rounding it. The Java column
reaches for org.zerodds.wire.Float128 instead, which this column
cannot: a generated Python module here imports nothing but
dataclasses, enum and typing, so a runtime class would make
every generated file depend on a package that is not part of this
layer.
§The shapes
| IDL | Python |
|---|---|
string<N>, wstring<N> | str, with the width and bound in the metadata |
fixed<P,S> | str, holding the decimal |
sequence<T> | list[T] |
sequence<T,N> | list[T], with the bound in the metadata |
T[N] | tuple[T, …] — N arguments, not an ellipsis |
T[N][M] | tuple[tuple[T, …M], …N], outer dimension outermost |
map<K,V> | list[tuple[K, V]] |
union | a dataclass with _d and one optional field per case |
enum | enum.IntEnum; a member of it is an int |
bitmask | enum.IntFlag; a member of it is an int |
bitset | a dataclass of one field per bitfield |
@optional T m | typing.Optional[T], defaulting to None |
typedef | nothing; a comment, and the target’s type at each use |
module | a class the declarations nest in |
§sequence<T> is a list and T[N] is a tuple
Not a style choice. An IDL array has a length that is part of its
type and a sequence does not, and those are the two Python built-ins
that say exactly that: a list can be appended to and a tuple
cannot. The array’s length goes further — tuple[int, int, int]
names three elements, and a static checker rejects a two-element
literal — which makes the length the one IDL fact in this whole
column that Python’s own type system enforces. Writing
tuple[int, ...] and putting the length in the metadata would give
that up for nothing.
The cost is stated rather than hidden: long a[1000] produces an
annotation with a thousand arguments in it. Nothing truncates it and
there is no threshold at which the spelling changes, because a
threshold would mean the same IDL construct had two Python forms
depending on a number nobody chose.
§map<K,V> is a list of pairs and not a dict
Two independent reasons, either of which decides it.
A dict requires hashable keys. A generated struct is a
@dataclasses.dataclass with eq=True, which sets __hash__ to
None — so dict[Point, int] is a type that cannot hold anything,
and map<Point, long> is ordinary IDL.
And the entry order is a policy axis. MapEntryOrder exists to
measure which order a peer puts the entries in, and a container that
decided an order of its own would make the axis measure the container.
A list of pairs keeps the sample’s order where the sample put it.
The Java column reaches the same answer through an array of a
generated entry class; Python needs no generated class because
tuple[K, V] already is one.
§@optional is typing.Optional[T] and None is the absence
One field, not a field beside a flag. XTypes 1.3 §7.4.1.2.3 makes
absence the value, not a zero, and None is Python’s. A
bool _has_m beside an m would be two things that can disagree,
which is what the C column has to carry because C has no None.
A non-optional member is emitted with no default at all, so constructing an aggregate without it is an error rather than a silently zero-filled sample.
§An enumeration is an enum.IntEnum and a member of it is an int
Both halves matter.
The declaration is a real enum.IntEnum because a Python enum
member is an int — Colour.RED == 0 is true and
Colour.RED + 1 is an int. Java cannot do this, because a Java
enum constant’s identity is its ordinal and @value(-1) is not an
ordinal, which is why the Java column emits a class of int
constants and loses the type.
The member is nonetheless an int and not the enumeration:
Colour(7) raises ValueError, and an @appendable writer that
added an enumerator sends exactly that. A field typed Colour could
only refuse the sample. So the member is
typing.Annotated[int, _IdlEnum(declared=Colour, bits=32)] — the
runtime type is what the wire can deliver, and which enumeration it
is stays readable, in the metadata, where a codec can get at it.
A bitmask is the same arrangement with enum.IntFlag.
§A bitset states the bit width and not a holder
bitfield<3> low is three bits. C and Java have to round that up to
the narrowest integer that holds it, because they have no three-bit
type; Python has no fixed-width type at all, so nothing is rounded
and the metadata states three. The declared holder of
bitfield<1, boolean> names a range and not a wire form, and the
width is what the packing uses — the same reading the Java column
gives.
§A typedef declares nothing
Python has type aliases, and this column does not use one, for a
reason that is Python’s own: an assignment is evaluated eagerly.
Position = a.Point at module level runs when the module is
imported, so it has to come after class a, and a typedef in one
IDL module naming a type in another has no position in the nesting
at which that is true. The alias would be a NameError at import,
for exactly the cross-module cases and no others. So a typedef is
transparent — the use takes the target’s type — and a comment
carries the name and what it resolved to, which is what the Java
column does and for a different reason.
§Modules are nested classes
Python’s own unit of scope is the module, and one IDL file becomes one Python module, so the file’s top level is already the global IDL scope; nothing needs an outer class the way the Java column does. A nested IDL module becomes a class, which is the only Python construct that nests, carries a name and can be referred to through a dotted path.
Every reference to a generated type is written from the module root
down — a.b.Point. Unlike Java there is no capture hazard in doing
so: typing.get_type_hints resolves an annotation against the
module globals, where a is bound, and a member named a is not
in those globals. The escape below covers the case that does bite.
§Identifiers
Through zerodds_lang_cst::python::escape_name, which appends an
underscore — PEP 8’s own rule for this case. It moves three kinds of
name: a keyword, a soft keyword, and a name the generated module
itself binds (int, list, typing). The third is the one that is
invisible without it: an @optional member named int becomes a
class attribute set to None, and typing.get_type_hints resolves
against the class namespace before the module’s, so every int in
that class would then resolve to None.
The IDL name is not lost. A renamed field carries it in
dataclasses.field(metadata={"idl": …}), which is machine-readable;
every other renamed declaration carries it in a comment on the line
above. And two IDL names that would escape onto one Python name in
one scope are refused rather than letting the second take the first’s
declaration.
§What is refused
any, ValueBase, an object reference and an annotation type have
no Python data form here and are named as
MaterialiseError::NoLanguageForm rather than left out of the
class. Python would happily take object for all four, which is
exactly why it is not written: a member typed object is a field no
codec can write, from a generator that reported success.
Functions§
- declared_
path - The path a generated declaration of
idis reached by. - primitive_
alias - The alias an IDL primitive takes.
- python_
member_ type - The Python annotation of a value in a position that has to hold one, refusing the constructs Python cannot hold.
- python_
module - Build the Python module a rule application produced.
- type_
tag - The name a codec function takes for the type it codes.