Skip to main content

Module python

Module python 

Source
Expand description

The Python CST as the other end of a rule. Spec §3.2.

The counterpart of crate::adapter::java, crate::adapter::cpp and crate::adapter::c. Reading two of them side by side is the point: where they differ, the difference is the language’s, and it is written down here rather than spread through a renderer.

Values are resolved from the model, never from the overlay — the same rule the sibling materialisers follow and for the same reason (spec §2.3).

§Python is the hardest of the five, and this is why

The other four target languages state a width, a length and a signedness in the declaration itself. Python states none of them. A long is 32 bits and a Python int is arbitrary precision; an IDL octet and an IDL unsigned long long are the same Python type; long a[3] and sequence<long> a are both “a thing you can index”. A backend that wrote x: int and stopped would produce a file that type-checks and a value the wire cannot hold, with nothing between the two to notice.

So this column is arranged around one question: for each IDL fact Python cannot enforce, where is it written down? There are exactly two answers used here and no third.

  1. The Python type, where Python can enforce it. Mutability (list against tuple), arity (tuple[T, T, T]), absence (typing.Optional[T]), and which class a member holds. A static checker rejects a program that gets these wrong.
  2. typing.Annotated metadata, where it cannot. The width, the signedness, the bound, the digit count, the character width. A checker passes these through untouched; they are read back at run time with typing.get_type_hints(cls, include_extras=True), which is what the codec pass will do.

Nothing is left implicit. x: int never appears in generated output.

§The integer mapping

IDLPythonmetadata
booleanbool—
charstr_IdlText(bits=8, bound=1)
wcharstr_IdlText(bits=16, bound=1)
int8int_IdlInt(bits=8, signed=True)
octet, uint8int_IdlInt(bits=8, signed=False)
shortint_IdlInt(bits=16, signed=True)
unsigned shortint_IdlInt(bits=16, signed=False)
longint_IdlInt(bits=32, signed=True)
unsigned longint_IdlInt(bits=32, signed=False)
long longint_IdlInt(bits=64, signed=True)
unsigned long longint_IdlInt(bits=64, signed=False)
floatfloat_IdlFloat(bits=32)
doublefloat_IdlFloat(bits=64)
long doublestr_IdlFloat(bits=128)

Each of those is written once as a module-level alias — _Int32, _UInt64 — so a field reads x: _Int32 and the width is one hop away rather than inline on every line.

An alias exists exactly where the type’s whole description is fixed. A string<7> and a string<9> are not the same type, so the string family is written inline instead — including the unbounded one, so that string and string<7> have one spelling between them rather than a name for the common case and a form for the rest.

Where this departs from every other column. Java has to decide which of its signed types is wide enough to hold an unsigned IDL value, and unsigned long long ends up a long carrying a bit pattern rather than a number. Python has no such problem and so makes no such trade: every one of the ten integers is a Python int holding the number, and only the metadata differs. That is the one place where Python’s lack of widths is an advantage, and it is worth saying because everywhere else it is the cost.

§long double is a str and never a float

A Python float is a C double — 53 bits of significand. An IDL long double is a binary128 with 113. Writing one into a float loses sixty bits at the assignment, before any codec runs, with nothing to say it happened. That is the silent-substitution class this project is arranged against, and it is the same defect the three-column gate found in the C column when long double reached double there.

So the member holds the decimal text, exactly as fixed<P,S> does and for the same reason: it is the one Python type that holds the value the source wrote without rounding it. The Java column reaches for org.zerodds.wire.Float128 instead, which this column cannot: a generated Python module here imports nothing but dataclasses, enum and typing, so a runtime class would make every generated file depend on a package that is not part of this layer.

§The shapes

IDLPython
string<N>, wstring<N>str, with the width and bound in the metadata
fixed<P,S>str, holding the decimal
sequence<T>list[T]
sequence<T,N>list[T], with the bound in the metadata
T[N]tuple[T, …] — N arguments, not an ellipsis
T[N][M]tuple[tuple[T, …M], …N], outer dimension outermost
map<K,V>list[tuple[K, V]]
uniona dataclass with _d and one optional field per case
enumenum.IntEnum; a member of it is an int
bitmaskenum.IntFlag; a member of it is an int
bitseta dataclass of one field per bitfield
@optional T mtyping.Optional[T], defaulting to None
typedefnothing; a comment, and the target’s type at each use
modulea class the declarations nest in

§sequence<T> is a list and T[N] is a tuple

Not a style choice. An IDL array has a length that is part of its type and a sequence does not, and those are the two Python built-ins that say exactly that: a list can be appended to and a tuple cannot. The array’s length goes further — tuple[int, int, int] names three elements, and a static checker rejects a two-element literal — which makes the length the one IDL fact in this whole column that Python’s own type system enforces. Writing tuple[int, ...] and putting the length in the metadata would give that up for nothing.

The cost is stated rather than hidden: long a[1000] produces an annotation with a thousand arguments in it. Nothing truncates it and there is no threshold at which the spelling changes, because a threshold would mean the same IDL construct had two Python forms depending on a number nobody chose.

§map<K,V> is a list of pairs and not a dict

Two independent reasons, either of which decides it.

A dict requires hashable keys. A generated struct is a @dataclasses.dataclass with eq=True, which sets __hash__ to None — so dict[Point, int] is a type that cannot hold anything, and map<Point, long> is ordinary IDL.

And the entry order is a policy axis. MapEntryOrder exists to measure which order a peer puts the entries in, and a container that decided an order of its own would make the axis measure the container. A list of pairs keeps the sample’s order where the sample put it. The Java column reaches the same answer through an array of a generated entry class; Python needs no generated class because tuple[K, V] already is one.

§@optional is typing.Optional[T] and None is the absence

One field, not a field beside a flag. XTypes 1.3 §7.4.1.2.3 makes absence the value, not a zero, and None is Python’s. A bool _has_m beside an m would be two things that can disagree, which is what the C column has to carry because C has no None.

A non-optional member is emitted with no default at all, so constructing an aggregate without it is an error rather than a silently zero-filled sample.

§An enumeration is an enum.IntEnum and a member of it is an int

Both halves matter.

The declaration is a real enum.IntEnum because a Python enum member is an int — Colour.RED == 0 is true and Colour.RED + 1 is an int. Java cannot do this, because a Java enum constant’s identity is its ordinal and @value(-1) is not an ordinal, which is why the Java column emits a class of int constants and loses the type.

The member is nonetheless an int and not the enumeration: Colour(7) raises ValueError, and an @appendable writer that added an enumerator sends exactly that. A field typed Colour could only refuse the sample. So the member is typing.Annotated[int, _IdlEnum(declared=Colour, bits=32)] — the runtime type is what the wire can deliver, and which enumeration it is stays readable, in the metadata, where a codec can get at it.

A bitmask is the same arrangement with enum.IntFlag.

§A bitset states the bit width and not a holder

bitfield<3> low is three bits. C and Java have to round that up to the narrowest integer that holds it, because they have no three-bit type; Python has no fixed-width type at all, so nothing is rounded and the metadata states three. The declared holder of bitfield<1, boolean> names a range and not a wire form, and the width is what the packing uses — the same reading the Java column gives.

§A typedef declares nothing

Python has type aliases, and this column does not use one, for a reason that is Python’s own: an assignment is evaluated eagerly. Position = a.Point at module level runs when the module is imported, so it has to come after class a, and a typedef in one IDL module naming a type in another has no position in the nesting at which that is true. The alias would be a NameError at import, for exactly the cross-module cases and no others. So a typedef is transparent — the use takes the target’s type — and a comment carries the name and what it resolved to, which is what the Java column does and for a different reason.

§Modules are nested classes

Python’s own unit of scope is the module, and one IDL file becomes one Python module, so the file’s top level is already the global IDL scope; nothing needs an outer class the way the Java column does. A nested IDL module becomes a class, which is the only Python construct that nests, carries a name and can be referred to through a dotted path.

Every reference to a generated type is written from the module root down — a.b.Point. Unlike Java there is no capture hazard in doing so: typing.get_type_hints resolves an annotation against the module globals, where a is bound, and a member named a is not in those globals. The escape below covers the case that does bite.

§Identifiers

Through zerodds_lang_cst::python::escape_name, which appends an underscore — PEP 8’s own rule for this case. It moves three kinds of name: a keyword, a soft keyword, and a name the generated module itself binds (int, list, typing). The third is the one that is invisible without it: an @optional member named int becomes a class attribute set to None, and typing.get_type_hints resolves against the class namespace before the module’s, so every int in that class would then resolve to None.

The IDL name is not lost. A renamed field carries it in dataclasses.field(metadata={"idl": …}), which is machine-readable; every other renamed declaration carries it in a comment on the line above. And two IDL names that would escape onto one Python name in one scope are refused rather than letting the second take the first’s declaration.

§What is refused

any, ValueBase, an object reference and an annotation type have no Python data form here and are named as MaterialiseError::NoLanguageForm rather than left out of the class. Python would happily take object for all four, which is exactly why it is not written: a member typed object is a field no codec can write, from a generator that reported success.

Functions§

declared_path
The path a generated declaration of id is reached by.
primitive_alias
The alias an IDL primitive takes.
python_member_type
The Python annotation of a value in a position that has to hold one, refusing the constructs Python cannot hold.
python_module
Build the Python module a rule application produced.
type_tag
The name a codec function takes for the type it codes.