Operators

Use the operator catalog to identify likely compatibility problems, then run conversion to check the actual graph. Support depends on the operator, tensor roles and types, shape, attributes and target; a model name alone cannot answer whether it lowers.

Browse the operator catalog→

Look up supported operators, tensor data types, kernel dependencies and restrictions before converting your model.

What “supported” means

Three steps must succeed for a remaining node:

1. The LiteRT parser produces an AIR operator with a canonical key such as CONV_2D.
2. That key resolves to a registered operator class.
3. The class accepts the node’s tensor and target conditions and prepares its lowering.

A lowering can be a kernel call, inline arithmetic, a copy, an alias or an accelerator invocation. The operator catalog summarizes the default registry’s types, dependencies and restrictions; a row is not a guarantee of every type/shape pairing or an optimized arithmetic path.

Candidate selection also depends on the target and operator implementation. How heliaAOT works explains the selection stage. The kernel library’s target build remains part of deployment.

Checking a model before you convert

1. Compare the model’s operator list and tensor roles with the catalog.
2. Read helia-aot platforms show <target> for the hardware assumptions.
3. Convert with the intended configuration and inspect any typed error.

There is no separate per-model coverage-report command. Conversion checks each remaining node before emission, after applicable graph rewrites. Success proves lowering with that configuration; build and output comparison establish the later contracts. No board is needed for this check.

The restrictions every model meets

The input must be a LiteRT flatbuffer and every output extent must resolve at conversion time. A runtime-dependent reshape target cannot be allocated statically. Dynamic signatures that resolve from concrete inputs are a different case from unresolved runtime shapes.

No operator takes an INT4 tensor: a model with INT4 weights is rejected, naming the first operator that uses one. Re-export it with int8 weights; INT4 support is tracked in #701.

A tensor with dimensions and more than one scale must name a quantized dimension it has, as LiteRT requires to load it; some exports give a rank-1 bias dimension 3.

Read requirements for each tensor role. An int16 LSTM cell state does not imply int16 activations, and index/mask tensors can differ from data tensors. The precision guide explains these distinctions.

Every int8, uint8 or int16 tensor an operator takes as an input or produces as an output, including a constant input such as a DEQUANTIZE source, must be quantized per tensor: one scale and at most one zero point. The emitted code reads only the first scale and zero point of such a tensor. A tensor an operator takes or produces, weights and other parameters included, that has a scale must also carry a zero point, whatever its dtype, as LiteRT requires to load it. An int8, uint8 or int16 tensor with no scale, and a tensor of any other dtype, such as a float tensor carrying a calibration range, is unquantized whatever it carries: its zero point is read as 0, as LiteRT’s kernels read it. The zero point of every int8, uint8, int16 or int32 tensor, including the model’s inputs and outputs and an operator’s weights, state or other parameters such as a PRELU alpha, must also lie inside its dtype’s range (-128..127, 0..255, -32768..32767 or the int32 range). An int8, uint8 or int16 zero point becomes an int32 offset or constant; nothing reads an int32 tensor’s zero point, which is held to the int32 range only. The generated I/O table lists the first scale and zero point of each int8, uint8 or int16 model input and output that has a scale, and scale 1 and zero point 0 for any other tensor, int32 included, since LiteRT’s kernels read no quantization of such a tensor. LiteRT’s interpreter still reports the scale and zero point of a tensor with a scale, whatever its dtype, and scale 0 and zero point 0 for a tensor without one, whatever zero point it carries. Every scale of an int8, uint8 or int16 tensor must be finite and positive once narrowed to float32; a LiteRT file stores float32 scales, but a model built through the Python API can carry wider values. A constant may have scale exactly 0, as an all-zero weight channel does. So may data moved unchanged from such a constant: the output of an operator that only moves, compares or selects stored values, when that operator is its only writer, the output is not a variable, and every int8, uint8 or int16 input it moves has scale 0 and is such a constant or such data. The operators that forward a zero scale are BATCH_TO_SPACE_ND, BROADCAST_TO, CONCATENATION, DEPTH_TO_SPACE, DILATE, DYNAMIC_UPDATE_SLICE, EXPAND_DIMS, GATHER, GATHER_ND, MAXIMUM, MINIMUM, MIRROR_PAD, PACK, PAD, RESHAPE, RESIZE_NEAREST_NEIGHBOR, REVERSE_SEQUENCE, REVERSE_V2, SELECT_V2, SLICE, SPACE_TO_BATCH_ND, SPACE_TO_DEPTH, SPLIT, SQUEEZE, STRIDED_SLICE, TILE, TRANSPOSE and UNPACK; the padding ones write padding as a raw stored value. PADV2 and SPLIT_V forward too, as they are lowered to PAD and SPLIT. Moved zero-scale data may feed an operator input, a parameter such as a PRELU alpha, or weights; an operator that precomputes from its weights at conversion, such as int8 CONV_2D and DEPTHWISE_CONV_2D on MVE targets or UNIDIRECTIONAL_SEQUENCE_LSTM, rejects moved weights. Any other operator output and the model’s inputs and outputs may not have scale 0, and an operator whose arithmetic needs a nonzero input scale rejects a zero one. Without test.golden_data, test.enabled generates the expected outputs with LiteRT, whose kernels reject or abort on a zero input scale, so a model with such a tensor needs its test outputs from test.golden_data. Operators can require more: int16 zero points of 0 for kernels that take no offset, for example. Per-channel weights, state and parameters otherwise follow each operator’s own restrictions, and SVDF reads the first channel of a per-channel tensor as its reference kernels do.

An operator input or output with no elements is rejected unless the operator accepts empty tensors of that dtype: CONCATENATION, PACK, UNPACK, SPLIT, SQUEEZE, RESHAPE and EXPAND_DIMS in float16 and float32, and SQRT, AVERAGE_POOL_2D and MAX_POOL_2D in float32. A pool accepts an empty height or width only when its output is empty there; its batch and channel counts must match the input’s and be nonzero.

An output shape the model declares fully static must be the shape the operator computes from its inputs and options. Shape propagation, which runs before any transform or memory planning, checks each declaration as soon as the operator’s inputs are known, and rejects a model whose declaration differs rather than converting it with the declared size. A LiteRT output with no recorded rank is read as a rank-0 scalar and checked as one. Convolution and pool output heights and widths follow LiteRT’s SAME or VALID output size, and a transposed convolution’s output is its output_shape input.

TRANSPOSE_CONV checks its declared output itself as well. Its input and output channels must match its weights, as LiteRT’s contract states. Its bias, like a CONV_2D or DEPTHWISE_CONV_2D bias, must hold at least one value per output channel, since the kernels read one per channel; a quantized DEPTHWISE_CONV_2D bias must hold exactly one. Otherwise a longer bias converts, and LiteRT’s default interpreter runs it the same way. A model with fewer output channels than its weights is rejected: LiteRT checks that only in debug builds, and its kernels then disagree on the result. A run-time output_shape must hold the declared output shape: the generated module computes the declared shape and does not read the value.

An int8 TRANSPOSE_CONV runs one of two ways. With both strides at most 2 and more input channels than the TRANSPOSE_REVERSE_CONV threshold (16 by default), the transform, enabled by default, rewrites it as a convolution, which has no output-size limit beyond the shape rule but needs constant weights. Otherwise the int8 kernel computes at most (input - 1) * stride + max(kernel, stride) rows and columns, so an output larger than that, padding included, is rejected. A Keras padding='valid' export exceeds it when output_padding is above max(stride - kernel, 0); Keras padding='same' exports stay within it. A shorter output converts, even one that leaves an input row’s whole filter outside it. Float32 converts every output size.

A shape rule also rejects operands no valid model has, as LiteRT does when it checks the operator: inputs that do not broadcast (ADD, SUB, MUL, DIV, MAXIMUM, MINIMUM, SQUARED_DIFFERENCE, the comparisons and SELECT_V2), PACK inputs of different shapes, DILATE dilations that are not rank 1, and a SCATTER_ND shape with a negative value.

Some outputs are not checked this way and rely on the operator’s own checks. A rule that cannot compute a shape, such as a RESHAPE target or FILL dims computed at run time, checks nothing. These operators have no shape rule: ARG_MAX, ARG_MIN, ASSIGN_VARIABLE, ETHOS_U, GATHER_ND, GRU, READ_VARIABLE, RESIZE_BILINEAR, RESIZE_NEAREST_NEIGHBOR, SHAPE, SQUEEZE, SVDF, UNIDIRECTIONAL_SEQUENCE_LSTM, UNPACK and WHERE, whose output the importer derives from its condition.

Every dimension of every tensor, and the product of a tensor’s nonzero dimensions, must fit int32, empty tensors included. Declared shapes are checked before any shape rule runs, and inferred shapes after propagation.

Restrictions can also involve ranks, constant weights, symmetric zero points, activation functions, broadcasting and signed kernel dimension/context ranges. The catalog is a summary; converter validators remain authoritative. Products can overflow a kernel range even when each dimension fits. Use the error’s node and tensor details to locate the unsupported form.

Stateful models

Stateful integration has two distinct contracts. Determine which the exported model uses before writing its streaming loop or golden fixture.

Operator/path	State contract
LSTM and SVDF internal state	Persistent variable tensors survive model_run; initialization resets the internal state. Check each operator’s dtype and shape restrictions.
Supported rolled GRU	Explicit initial-state input and final-state output; the caller carries state. Persistent GRU state tensors are rejected.
Resource variables	Read/write operations use persistent storage according to the graph’s variable contract.

The recurrent catalog entries describe supported data types: LSTM and SVDF have int8 and float paths with role-specific conditions; GRU is FP16, batch size one, with reset_after=True. Check the complete row for weights, sequence layout, state and rank constraints.

Where the state lives

Internal variable state is whole-program-live writable storage in a persistent arena. aot_model_init resets persistent slots and runs operator initialization; aot_model_run retains the resulting state for the next call. The normal sequence is:

1. Initialize once for a new independent sequence and check the status.
2. Copy the next input window and run; check status before consuming output.
3. Repeat input/run for subsequent windows without reinitializing.
4. Initialize again when deliberately resetting the sequence.

For explicitly exposed state, copy the previous state output to the corresponding state input, using the model’s I/O order and matching shape/dtype. Use memmove when buffers may overlap. Reset by supplying the intended initial-state values before the next independent sequence.

Bindings and generated arena buffers are module-global. Serialize calls and state updates; two context structs are not independent stateful instances. The residency report shows persistent storage, while explicit GRU state must be understood from the model I/O contract. Memory covers arena ownership and sharing.

The rolled GRU

A Keras export can represent the GRU as a WHILE loop across LiteRT subgraphs. The default fuse_rolled_gru LiteRT model hook recognizes the supported cell equations and fuses that form before conversion to AIR. It is distinct from AIR graph transforms applied later.

The supported GRU has sequence and initial-state inputs, plus sequence and final-state outputs. State tensors must be ordinary graph I/O, not persistent LiteRT variables. Consult the model-level I/O order: it need not match the operator’s local tensor order.

Python registry customizers can replace or remove the hook, but removing it does not make the raw loop lowerable. A different export or corresponding custom parser/lowering is then required.

Carrying state through a test run

For internal persistent state, repeated test iterations retain state without test.state_feedback. Non-state inputs are refreshed from the same stimulus on each iteration; this is not a stream of different input windows.

For explicit state I/O, test.state_feedback contains zero-based [output_position, input_position] pairs. Positions refer to the model’s output and input lists, not flatbuffer tensor IDs. For a model whose final state is output 0 and initial state is input 1:

test:
  enabled: true
  golden_data: golden.npz
  num_iterations: 4
  state_feedback:
    - [0, 1]

Use this mapping only after checking your model’s order. The converter rejects out-of-range positions, shape/dtype mismatches and multiple outputs assigned to the same input. The harness seeds the state input on iteration zero and then carries output into it with memmove; other inputs are refreshed each iteration.

With a supplied NPZ, output_N must describe the final iteration under that same stimulus, initial state, count and carry mapping. The converter does not rerun supplied golden data to repair a mismatched protocol. Without supplied golden data, its host reference loop uses the configured feedback/count.

For a real sequence of different windows, use an application harness that feeds those windows and checks the required outputs. The generated repeated-stimulus test alone does not establish streaming accuracy. See Testing and automation for the test contract.

When an operator is missing

First consider a supported upstream export or graph formulation. If custom lowering is needed, supply a LiteRT parser and AOT operator class using the same canonical key. Register them through a Python registry context; Custom operators walks through that process.

When the kernel is not the one you want

There is no configuration field selecting an arbitrary kernel name. Selection uses operator conditions, types and platform information. Changing capabilities requires a complete custom target or a separately registered Python platform; capability fields on a registered target name are ignored. Keep those declarations truthful and record the kernel library’s CPU/build flags; see Targets.

A Python customizer can replace an operator class with allow_override=True. Retain validation and scratch sizing when changing emission, and test the actual numerics. See replacing a built-in operator.

Rewriting the graph before it is lowered

AIR transforms run after loading and before operator resolution. Configure them in the transforms block described in Configuring a conversion, or register a custom transform under its NAME in a Python registry context. A custom transform that sets OPTIONS to a pydantic model or pydantic dataclass, or to NoOptions when it takes none, has its option keys, and while it is enabled its option values, checked like the built-in ones. The default is NoOptions, so omitting OPTIONS rejects all option keys. Set OPTIONS = None to accept free-form options that the transform checks itself; its errors propagate as they are. Any other OPTIONS is a TypeError. The operator list is the execution order: a transform must leave every operator after the producers of the tensors it reads (constants, persistent state and model inputs excepted), or the conversion stops and names the transform.

LiteRT model hooks, including rolled-GRU fusion, run earlier over the source model. Choose the extension stage that still contains the structure you need; do not treat a LiteRT multi-subgraph hook and an AIR transform as interchangeable.

INT32 comparisons and selection

EQUAL, NOT_EQUAL, GREATER, GREATER_EQUAL, LESS and LESS_EQUAL support unquantized INT32 inputs and Boolean outputs. Scalar through rank-four operands use singleton broadcasting. The broadcast output count must fit INT32_MAX / 4.

SELECT_V2 accepts a Boolean condition and INT32 x/y tensors with singleton broadcasting through rank eight. All dimensions must be positive, and tensor byte counts must fit INT32_MAX. Comparisons preserve full-width signed values; selection copies values exactly. Neither path needs float support or scratch. Existing INT8/INT16 routes retain their quantized comparison and data-selection behavior.

These routes use the supported CORE dependency described in the installation guide.

Floating-point division

DIV lowers FP32 and FP16 operands to heliaCORE’s direct division kernels. Equal layouts use a flat call; scalar through rank-four broadcasting uses compact, left-padded NHWC operands without scratch or expanded tensors. Operand order is preserved, including a scalar numerator. Integer DIV is not supported.

Both operands and the output must share a dtype and have positive static extents; each tensor’s byte count must fit INT32_MAX. FP16 requires an M55-class target. NONE, RELU, RELU_N1_TO_1 and RELU6 use the same dtype-specific activation bounds as float ADD/MUL: unbounded ends are represented by finite dtype limits. NaNs propagate through the heliaCORE clamp. FP16 rounds the quotient to half storage before activation. No reciprocal-multiply approximation is emitted.

The existing heliaCORE FP16 clamp converts negative zero to positive zero with RELU and RELU6; LiteRT’s FP32 reference retains the zero sign. FP16 fused division therefore does not promise bitwise LiteRT parity for signed zero. LiteRT does not execute native FP16 DIV; finite FP16 reference checks use exactly widened FP32 inputs and round the reference output to half precision.

heliaCORE’s CMake integration supplies the required Clang/Armclang -fno-fast-math option for DIV sources; manual heliaCORE builds must preserve it.

Floating-point NEG

NEG supports FP32 and FP16 tensors, including scalars and higher ranks, with identical input/output shapes and positive static dimensions. Tensor byte counts must fit INT32_MAX. FP16 requires a Cortex-M55-class target. The generated layer calls heliaCORE directly without scratch or arithmetic loops.

Negation toggles the IEEE sign bit, preserving subnormal values and NaN payload bits and exchanging positive and negative zero. Integer NEG is unsupported.

Floating-point squared difference

SQUARED_DIFFERENCE supports FP32 and FP16 operands with equal layouts and positive static dimensions, including scalars. Leading singleton axes can differ through rank four; higher ranks require identical shapes. Float broadcasting is rejected. Existing integer broadcasting remains supported.

Core computes a rounded subtraction followed by a rounded multiplication at the selected precision. NaN payloads are unspecified. Explicit MVE uses its architectural floating-point flushing behavior, which can differ from scalar gradual underflow for subnormals. From Core 7.47.0 the FP32 entry redoes on the scalar FPU each lane whose square MVE flushed, so at round-to-nearest it gives the scalar result, subnormal squares included; earlier releases, below the 7.47.0 minimum, flush them. FP16 requires a Cortex-M55-class platform. Each tensor byte count must fit INT32_MAX. The generated layer calls the flat Core entry without scratch or activation clamp.

These routes use the supported CORE dependency described in the installation guide.