Skip to content
heliaAOT
Examples
HELIA HUB

Recurrent state

Convert a stateful LSTM and inspect its persistent arena. The generated module retains recurrent state between calls to aot_model_run() and resets it during model initialization.

Field Value
Model one UNIDIRECTIONAL_SEQUENCE_LSTM node, int8 activations, int16 cell state
Target apollo510_evb
Shows LiteRT variable tensors as persistent arenas, and what test.state_feedback is for
CI Conversion and host compilation are declared in the examples CI job

The model is built by make_model.py: a single LSTM node whose state tensors are LiteRT variable tensors, the state carry this example is about.

Prerequisites

Section titled “Prerequisites”

Use a repository checkout and activate its environment after uv sync --frozen --group ci. The model-building script imports the repository’s Python helpers; an isolated CLI installation alone is not enough. See Running examples.

Run it

Section titled “Run it”
Terminal window
./run.sh

A second run stops with Output path ... already exists, because the first run’s module is still in out/. Run ./run.sh --force to replace it.

config.yaml asks for nothing about state. The model’s variable tensors are what produce it.

What to look at

Section titled “What to look at”

The arena summary grows a third line:

 Region   Kind                          Size    Capacity
 dtcm     scratch                       70 B    496.0 KB
 dtcm     persistent                    24 B    496.0 KB
 dtcm     constant (cold) · 20          80 B    496.0 KB
          tensors

Scratch is reused within a run. Persistent is not: the output state and the cell state are LiteRT variable tensors, and the module keeps them across runs so a streaming caller can hand it one window at a time. The residency report in out/lstm/aot_residency.json names both tensors and the arena they landed in.

The generated module has shared arena storage: two context structs do not carry independent state. Serialize calls to one logical module instance. model_init() resets persistent storage, while subsequent successful model_run() calls carry it. For separate streams, use separately generated modules with distinct prefixes and independent persistent storage, or design and validate explicit state save/restore; a second struct alone is insufficient.

The other shape of state

Section titled “The other shape of state”

The following is a configuration fragment for a different model with explicit state I/O, not a command to add to this LSTM recipe. This recipe does not create golden.npz or expose the illustrated feedback pair.

Some recurrent models do not hold state internally: they expose the initial state as an input and the final state as an output, and the caller is expected to feed one back into the other. For those, test.state_feedback tells the generated test harness to do exactly that, as a list of output-index to input-index pairs:

test:
  enabled: true
  golden_data: golden.npz
  num_iterations: 4
  state_feedback: [[0, 1]]

The harness then seeds the state once and carries each run’s output into the next run’s input. The NPZ supplies one input_N array per input and one output_N array per output, with the model’s dtype and shape. It does not contain a per-iteration sequence: output_N must represent the final result after num_iterations with the same reset/carry policy. Intermediate outputs are not compared. Choose indices for that model’s real I/O. Without feedback, the harness refreshes ordinary inputs from the fixture; an internal variable-state model can still carry state. Compare against a trusted oracle using the same reset/carry policy.

Next

Section titled “Next”