AotOperator

AotOperator(
    op: AirOperator,
    model: AirModel,
    platform: SocPlatform,
    prefix: str = 'aot',
    attributes: dict[str, Any] | None = None,
)

Base class for the AOT lowering of a single AIR operator.

Provisional: importable, but outside the compatibility promise; a minor release can change or remove it.

One instance wraps one AirOperator and drives it through the per-operator half of the conversion: resolve() validates the operator against the target platform and materializes any scratch or constant tensors, plan() reports what the memory planner must reserve, and emit() renders the operator’s C source. Subclasses implement those steps for one TYPE and are registered into RegistryContext.aot_operator_classes (see register_default_aot_operators for the built-ins).

The upper-case class attributes below are the declarative contract the rest of the pipeline reads. AotMeta rejects reassignment of an upper-case name that a class declares in its own body, so a subclass that inherits SUPPORTED_DTYPES without redeclaring it can still have it assigned at runtime.

Parameters

Parameters of AotOperator
Name	Type	Default	Description
`op`	`AirOperator`	Required	The AIR operator to wrap.
`model`	`AirModel`	Required	The AIR model.
`platform`	`SocPlatform`	Required	The target platform for code generation.
`prefix`	`str`	'aot'	Prefix for generated code files. Defaults to "aot".
`attributes`	`dict[str, Any] \| None`	None	Attributes for template values.

Raises

Errors raised by AotOperator
Type	Description
`InvalidConfigError`	If ``ATTRIBUTES`` rejects the merged attributes, such as a required attribute that no rule sets.

TYPE: AirOpType = AirOpType.OPERATOR

AIR operator type this class lowers, and its registry key.

CORE_CALL: tuple[str, str | None] | None = None

CAPABILITIES: OpCapability = OpCapability.NONE

Declared optimization contract (interning, static reordering, shared init); see OpCapability.

USES_CMSIS_FLOAT_KERNEL: bool = False

Whether the float lowering dispatches to a gated ns-cmsis-nn float kernel.

CMSIS_FLOAT_KERNEL_DTYPES: frozenset = frozenset({np.float16, np.float32})

Float dtypes that dependency applies to.

MIN_CMSIS_NN_VERSION_FLOAT: tuple[int, int, int] | None = None

Minimum ns-cmsis-nn version the float lowering requires, as (major, minor, patch), or None for the module-wide floor.

kernel: KernelRecord | None = None

The :class:KernelRecord chosen among the operator’s candidate ns-cmsis-nn entries, or None. The plan records its entry, so a subclass must not use the name for anything else.

mve_weight_sums: bool = False

Whether the layer reads precomputed weight sums, which fold its bias and which ns-cmsis-nn reads only on MVE builds; see :attr:requires_mve_build.

READS_FIRST_QUANT_CHANNEL: bool = False

Whether the kernels read the first channel of a per-channel input or output by design (SVDF), so the input and output check skips its one-entry rule.

resolved_kernel_plan: ResolvedKernelPlan | None

Resolved backend implementation, or None for an unmigrated operator.

requires_mve_build: bool

Whether this layer’s plan only runs on an MVE build.

True for a resolved backend plan requiring MVE or MVE-only weight sums, which also hold its bias; the module’s common.h then stops any build without ARM_MATH_MVEI. A selected entry that cannot run without MVE carries its own guard (KernelRecord.unsupported_build) instead.

CTX_BUF_USAGE: CtxBufUsage | None = None

Declared cmsis_nn_context::buf contract; see CtxBufUsage.

CTX_BUF_BACKING_TENSOR_NAMES: tuple[str, ...] = ()

SUPPORTED_DTYPES: tuple[type, ...] | None = None

Tensor dtypes the operator accepts, or None when the operator imposes no dtype gate of its own.

RESTRICTIONS: tuple[str, ...] = ()

User-facing sentences describing the limits validate() enforces.

ACCEPTS_EMPTY_TENSORS: frozenset[type] = frozenset()

MOVES_DATA_ONLY: bool = False

Whether the lowering only moves, compares or selects stored values without rescaling, so the operator may forward a zero scale (see :func:zero_scale_tensor_ids). A subclass of a built-in mover inherits True; one that rescales must set False.

zero_scale_ids: frozenset[TensorId] = frozenset()

kInputTensorIndex: int = 0

kOutputTensorIndex: int = 0

kWeightsTensorName: str = 'weights'

kMultiplierTensorName: str = 'multiplier'

kShiftTensorName: str = 'shift'

ATTRIBUTES: type[OperatorAttributes] = OperatorAttributes

op: AirOperator = op

model: AirModel = model

platform: SocPlatform = platform

prefix: str = prefix

attributes: OperatorAttributes = self.ATTRIBUTES(**attributes or {})

requested_attribute_keys: frozenset[str] = frozenset(attributes or {})

resolved_knobs: dict[str, str] = default_knob_values()

schedule: ScheduleMode = ScheduleMode.TABLE

id: str

Return the operator ID.

name: str

Return the operator name.

code_in_itcm: bool

Whether this operator’s run code is placed in ITCM.

True only when code_placement is ITCM and the target has ITCM; the operator handler warns about requests that cannot be honored.

code_placement_macro: str

Name of the <prefix>_platform.h macro that places run code in ITCM.

input_tensors: list[AirTensor]

Return the input tensors.

output_tensors: list[AirTensor]

Return the output tensors.

local_tensors: list[AirTensor]

Get all local tensors used by the operator.

QUANTIZED_FILTER_CHANNEL_AXIS: int | None = None

QUANTIZED_FILTER_ANY_CHANNEL_LABEL: bool = False

has_init: bool

Whether this operator emits a per-instance _init function.

Operators whose initialization is a no-op may opt out by returning False. When an operator opts out, it must not emit an _init function/declaration in its templates, and the dispatch table records a NULL init slot (the model init loop skips the call while preserving the init callback contract). Defaults to True so existing operators keep emitting their own init unchanged.

capabilities: OpCapability

Return the operator’s effective optimization capabilities.

The effective set is the declared :attr:CAPABILITIES widened by the capabilities implied by the operator’s runtime behavior, so the legacy seams remain the source of truth during migration and no operator has to declare a capability twice:

not self.has_init implies :attr:OpCapability.STATELESS.
A non-empty :meth:shared_kernel_helpers or :meth:shared_activation_helpers implies :attr:OpCapability.DESCRIPTOR_DRIVEN and :attr:OpCapability.SHARED_KERNEL.

Because some seams are dtype-dependent (e.g. fully-connected only routes through a shared kernel for int8 per-channel quantization), this is an instance property rather than a class constant: it reflects the concrete resolved operator. It performs no model mutation and does not change emitted code.

direct_dispatch: tuple[str, str] | None

Return the direct shared-kernel call target for static scheduling.

Under static scheduling the model schedule can call a descriptor driven operator’s shared kernel helper directly, eliding the thin per-node _run thunk (the main .text reclaim described in RFC 0003 Stage 4). This is only valid for operators whose _run body is exactly return helper(ctx, &desc); – i.e. operators that route through a single shared kernel helper with the uniform helper(ctx, const desc_t *) signature (ADD/MUL/per-channel int8 FULLY_CONNECTED). Shared activation helpers marshal flattened arguments and keep mutable file-scope state, so they are intentionally excluded and keep their _run thunk.

kernel_plan_on(capabilities: frozenset[SocCapability]) -> ResolvedKernelPlan | None

The plan this resolved layer would get on a target with capabilities, or None without a plan.

Planning is pure: the layer and its model are left unchanged.

set_knobs(values: dict[str, str]) -> None

Set the resolved optimization knob values before resolve().

Parameters

Parameters of set_knobs
Name	Type	Default	Description
`values`	`dict[str, str]`	Required	Knob name to resolved value, for every knob.

Raises

Errors raised by set_knobs
Type	Description
`ValueError`	When a knob is missing or unknown, or a value is ``auto`` or not one of the knob's values.

knob_applicability(name: str) -> tuple[bool, str | None] | None

Whether a knob’s alternatives would change this operator’s code.

Operators that implement a knob override this; the base operator has no knobs.

Parameters

Parameters of knob_applicability
Name	Type	Default	Description
`name`	`str`	Required	The knob's name.

Returns

Returns of knob_applicability
Type	Description
`tuple[bool, str \| None] \| None`	tuple[bool, str \| None] \| None: ``(applicable, reason when not)``,
`tuple[bool, str \| None] \| None`	or None when the knob does not exist for this operator.

fixed_choices() -> dict[str, tuple[str, FixedSetBy]]

Choices that change numerics or speed but are not knobs yet.

Returns

Returns of fixed_choices
Type	Description
`dict[str, tuple[str, FixedSetBy]]`	dict[str, tuple[str, FixedSetBy]]: Choice name to ``(value, set_by)``,
`dict[str, tuple[str, FixedSetBy]]`	where ``set_by`` is ``attribute``, ``default``, ``heuristic`` or
`dict[str, tuple[str, FixedSetBy]]`	``knob``.

typed_options(options_type: type[_OptionsT]) -> _OptionsT

Return self.op.options checked against the operator’s options class.

AirOperator.options is typed as the AirOperatorOptions marker base; each AOT operator knows its concrete class and reads through this so field access is type-checked and a mismatch (a mis-registered parser, a hand-built graph) fails with a clear error.

Parameters

Parameters of typed_options
Name	Type	Default	Description
`options_type`	`type[_OptionsT]`	Required	The ``AirXxxOptions`` class this operator expects.

Returns

Returns of typed_options
Type	Description
`_OptionsT`	The options object, typed as ``options_type``.

Raises

Errors raised by typed_options
Type	Description
`TypeError`	If the operator carries options of a different class.

float_io_dtypes() -> set[type]

Numpy float scalar types present on this operator’s IO tensors.

Returns

Returns of float_io_dtypes
Type	Description
`set[type]`	set[type]: The subset of ``{np.float16, np.float32}`` referenced by
`set[type]`	the operator's input or output tensors.

cmsis_float_dependencies() -> set[type]

Float dtypes for which this operator needs the ns-cmsis-nn float API.

A returned dtype means the emitted C references the float-only arm_nnfunctions_flt.h API (a gated arm_*_f32 / arm_*_f16 kernel or the float16_t / float32_t types it introduces) for that precision, so the ns-cmsis-nn build must enable ARM_NN_ENABLE_F32 / ARM_NN_ENABLE_F16.

The base implementation derives the set from :attr:USES_CMSIS_FLOAT_KERNEL: operators that dispatch to a gated float kernel advertise every float dtype on their IO; everything else (integer kernels, byte movers, portable conversions) advertises none. Operators with a per-instance dependency override this method.

Returns

Returns of cmsis_float_dependencies
Type	Description
`set[type]`	set[type]: Subset of ``{np.float16, np.float32}`` requiring the
`set[type]`	ns-cmsis-nn float API.

cmsis_nn_version_requirement() -> tuple[int, int, int] | None

Minimum ns-cmsis-nn version this operator instance requires.

The base implementation applies :attr:MIN_CMSIS_NN_VERSION_FLOAT only when the operator actually takes its float path, reusing the same :meth:cmsis_float_dependencies contract that drives the float header include and the ARM_NN_ENABLE_F32 / ARM_NN_ENABLE_F16 build defines. An int8 ABS therefore imposes no floor of its own while an fp32 ABS does; a declaration only raises a module’s requirement when it is above the repository-wide CMSIS_NN_VERSION.

Operators with an unconditional floor, or one that varies by something other than float dispatch, override this.

Returns

Returns of cmsis_nn_version_requirement
Type	Description
`tuple[int, int, int] \| None`	tuple[int, int, int] \| None: ``(major, minor, patch)``, or
`tuple[int, int, int] \| None`	``None`` when the module-wide floor already suffices.

get_local_tensor(name: str) -> AirTensor

Get a local tensor by name.

tensor_zero_point(tensor: AirTensor, index: int = 0) -> int

staticmethod

Return a tensor’s quantization zero point, defaulting to 0.

Parameters

Parameters of tensor_zero_point
Name	Type	Default	Description
`tensor`	`AirTensor`	Required	The tensor to inspect.
`index`	`int`	0	The zero-point index to read (per-tensor quant uses 0).

Returns

Returns of tensor_zero_point
Type	Description
`int`	The integer zero point, or 0 when the tensor is unquantized.

require_symmetric_filter(tensor: AirTensor, label: str, hint: str | None = None) -> None

staticmethod

Reject a quantized tensor whose zero points are not all 0.

The convolution, depthwise and transpose-convolution kernels take no filter offset, the int16 kernels apply no offsets, and per-channel fully connected and LSTM weights are symmetric by definition, so a nonzero zero point there would be dropped from the result.

Parameters

Parameters of require_symmetric_filter
Name	Type	Default	Description
`tensor`	`AirTensor`	Required	The tensor to check.
`label`	`str`	Required	What the tensor is, such as ``CONV_2D quantized filter``; the error starts with it.
`hint`	`str \| None`	None	The remedy to show, when the operator has a more specific one.

Raises

Errors raised by require_symmetric_filter
Type	Description
`ConfigValueError`	If any weight zero point is nonzero.

intern_constant_array(
    name: str,
    data: np.ndarray,
    *,
    role: AirTensorKind = AirTensorKind.CONSTANT,
    alignment: int | None = None,
) -> TensorId

Promote an operator-local constant array into a model tensor.

This routes operator metadata (e.g. per-channel quantization tables) through the same machinery as ordinary constants: memory planning, duplicate-constant interning (storage dedup), cold/staged residency, and uniform tensor-pointer resolution. The tensor is registered under op.named_tensors[name] so templates can reference it via ctx->tensor_ptrs[...] like any other tensor.

The operation is idempotent: a previously interned tensor with the same deterministic id is replaced.

Parameters

Parameters of intern_constant_array
Name	Type	Default	Description
`name`	`str`	Required	Local tensor name; also used to build the deterministic id.
`data`	`np.ndarray`	Required	Array payload, stored as a contiguous copy.
`role`	`AirTensorKind`	AirTensorKind.CONSTANT	Tensor kind/bucket. Defaults to ``CONSTANT``.
`alignment`	`int \| None`	None	Optional alignment hint in bytes.

Returns

Returns of intern_constant_array
Value	Type	Description
`TensorId`	`TensorId`	The id of the registered tensor.

require_filter_channel_label(weights: AirTensor, channel_axis: int, channels: int) -> None

Reject per-channel weight scales labelled with an axis the kernels do not read them along.

The kernels read one weight scale per output channel, in channel order, along channel_axis; an operator with :attr:QUANTIZED_FILTER_ANY_CHANNEL_LABEL also accepts another axis of that length. A single scale is per tensor whatever its label.

Parameters

Parameters of require_filter_channel_label
Name	Type	Default	Description
`weights`	`AirTensor`	Required	The weights tensor.
`channel_axis`	`int`	Required	The weights axis that holds the output channels.
`channels`	`int`	Required	The number of output channels.

Raises

Errors raised by require_filter_channel_label
Type	Description
`ConfigValueError`	If the weights carry more than one scale on another axis.

validate()

Validate the operator configuration.

resolve()

Public entry point—only runs once, even if called repeatedly.

This method validates the AIR operator and performs any model mutations needed.

Raises

Errors raised by resolve
Type	Description
`ConfigValueError`	If an input, output or named tensor breaks the shared quantization or element rules, or the operator's own validation rejects the model.
`MissingQuantizationError`	If a tensor read as quantized lacks a scale, or a quantized tensor carries a scale and no zero point; also a ``ConfigValueError``.
`ValueError`	If the operator declares that a kernel reads through a ``cmsis_nn_context`` buffer but allocated no backing for it (see :meth:`_assert_ctx_buf_backed`).

require_input_scale(*indices: int = ()) -> None

Reject the inputs at indices when every one has scale 0, for an operator that needs one nonzero.

The shared rule lets an input that is a constant, or data moved unchanged from one, have scale exactly 0. An operator whose arithmetic divides by an input’s scale, takes its log, or builds a table from it calls this first; one that uses the larger of two scales passes both. Only int8, uint8 and int16 inputs count; a float input’s quantization metadata is never read.

Raises

Errors raised by require_input_scale
Type	Description
`ConfigValueError`	If every quantized input at ``indices`` has a first scale of 0.

on_resolve()

Override this in subclasses to do the actual work.

on_placed(memory_plan) -> None

Adjust to where the memory plan put this operator’s tensors.

Runs after memory planning and before the render plan packs the constants. An override may change constant bytes but never a tensor’s size or placement, which tensors an operator binds, the model’s inputs and outputs, or the operator order: the plan is final, and apply_placement checks it. An override must be idempotent: a caller that builds the render plan on demand may run the hook again.

Parameters

Parameters of on_placed
Name	Default	Description
`memory_plan`	Required	The memory plan.

ctx_buf_required() -> bool

Whether this concrete operator’s lowering dereferences ctx->buf.

Only meaningful for operators declaring :attr:CtxBufUsage.CONDITIONAL, which MUST override this. Evaluate it against the same inputs the operator’s scratch sizing uses (dtype, options, platform capability) so the two can never disagree: the guard exists precisely to catch the case where sizing says “0 bytes” and the dispatched kernel says “I read that buffer”.

Returns

Returns of ctx_buf_required
Type	Description
`bool`	``True`` if at least one kernel this operator can dispatch reads or
`bool`	writes through ``ctx->buf``.

Raises

Errors raised by ctx_buf_required
Type	Description
`NotImplementedError`	If called on an operator that did not declare :attr:`CtxBufUsage.CONDITIONAL` and override this method.

plan()

Operator planning step.

shared_kernel_helpers() -> list[KernelHelper]

Return the shared operator-kernel helpers this operator requires.

This mirrors :meth:shared_activation_helpers but for general operator kernels (e.g. elementwise ADD/MUL or FULLY_CONNECTED). An operator that routes its _run body through a shared runtime helper (instead of an inlined per-node call site) declares the helper descriptors here so the code generator emits each distinct helper exactly once. Helpers are keyed by signature, so distinct operator variants map to distinct helpers and never collide. The base implementation requires no shared kernels.

Returns

Returns of shared_kernel_helpers
Type	Description
`list[KernelHelper]`	A list of :class:`KernelHelper` descriptors (see
`list[KernelHelper]`	mod:`helia_aot.aot.operators.kernel_runtime`). Empty for operators
`list[KernelHelper]`	that do not use a shared kernel helper.

shared_activation_helpers() -> list[dict[str, object]]

Return the shared LUT-activation helpers this operator requires.

Operators that emit their hot loop via a shared runtime helper (rather than an inlined per-node body) declare the helper descriptors here so the code generator can emit each distinct helper exactly once. The base implementation requires no shared helpers.

Returns

Returns of shared_activation_helpers
Type	Description
`list[dict[str, object]]`	A list of helper descriptor dicts (see
`list[dict[str, object]]`	func:`helia_aot.aot.operators.activation_runtime.activation_lut_helper`).
`list[dict[str, object]]`	Empty for operators that do not use a shared activation helper.

kernel_calls() -> dict[str, str] | None

Return the CORE calls the generated source makes, by role; templates emit exactly these names.

The calls come from the resolved plan, shared helpers or :attr:CORE_CALL. An empty mapping is an operator that calls no CORE entry; None denotes unknown coverage. These are direct generated calls, not transitive linked dependencies.

compute_values() -> dict[str, Any]

Compute the values for the operator source code template.

Args:

Returns

Returns of compute_values
Type	Description
`dict[str, Any]`	dict[str, Any]: Dictionary of values for the operator template.

print_info(verbose: int = 0)

Debug print the template values for the operator.

emit(save_path: Path)

Generate the source code for the operator.

This method should be overridden by subclasses.

Parameters

Parameters of emit
Name	Type	Default	Description
`save_path`	`Path`	Required	Path to save the generated source code.