# Plans and reports A conversion writes two JSON files next to the module. `_plan.json` holds every choice: the value of each knob for each operator and the kernel entry each layer calls. `_report.json` explains the plan and never changes it. Their JSON schemas ship with the documentation data as `plan-schema.json` and `report-schema.json`. ## The plan | Field | What it records | | --- | --- | | `schema_version` | The file's schema; see [Schema versions](https://ambiqai.github.io/helia-aot/guide/plans-and-reports/#schema-versions). | | `generator.helia_aot`, `generator.cmsis_nn_required` | The heliaAOT release that wrote the plan and the ns-cmsis-nn version the module requires. | | `model.file`, `model.sha256`, `model.subgraph` | The model the plan was made from. | | `module_prefix` | The module's symbol prefix. | | `platform.name`, `platform.cpu`, `platform.capabilities`, `platform.supports_fp16` | The target. The capabilities are what the release that wrote the plan knew of it; a later release can remove one as well as add one. | | `optimization.goal`, `optimization.allow_approximate` | The goal and the approximation gate. | | `operators[]` | One entry per operator, below. | | `tensors[]` | One entry per constant the operators read, below. | | `schedule` | The transfer schedule, below; `source: none` with no units when nothing streams. | | `optimization.knobs` | Reserved and empty today, for model-level choices. | ### Operator ids Each operator has a stable `id`, `":"`, taken from the name the model gives its first output tensor, so it survives edits that add or remove other operators. `index` keeps its position in execution order and `node_id` its AIR node id. - Repeated names get `#1`, `#2` in operator order. An operator without a name falls back to `":@"`; `id_source` says which applied. - A tensor name can itself contain `:` (for example `0:StatefulPartitionedCall_1:0`), so a parser splits these ids on their first colon only. An id with `id_source: node_id` is the opaque AIR node id and is not split. - `operators[].id` rules match the stable id as well as the node id, so a plan's ids can be written back as rules. - A stable id that would also select another operator (another operator's node id, or a stable id spelled twice) is not matched; the plan then uses the node id as `id`, with `id_source: node_id`, and the conversion warns. ### Per-operator fields | Field | What it records | | --- | --- | | each knob | The value used, the value requested, `resolved_by` (`explicit`, or `auto_table` when `auto` chose it from the goal; `auto_heuristic` and `plan` are reserved), whether the value is approximate where it applies, and the `source`: `kind` is `operator` (with `rule`, the position, type and id of the winning `operators` entry), `model` (set under `optimization`, even to `auto`), or `default` (`plan` is reserved). | | `kernel.entry` | The ns-cmsis-nn entry chosen, for layers that choose among candidates (convolution, depthwise, transpose convolution, packed int8 fully connected, precise packed FP16 fully connected); the report repeats it under `entry`. Average and max pooling record their selected int8, int16, FP16 or FP32 entry. Pooling, convolution, depthwise, transpose convolution, fully connected, batch matmul and SVDF record their single-entry paths explicitly, FP16, FP32 and ordinary fully connected entries included; other unmigrated single-entry layers still have a `null` entry. Selection follows the rules and knobs; a plan cannot set it yet. | | `kernel.provider` | An opaque string naming the built-in CORE planning rules, for pooling, convolution, depthwise, transpose convolution, fully connected, batch matmul and SVDF. Optional and absent from older plans; compare it with another plan's, do not parse it. | | `kernel.candidates` | Every entry whose rules match the layer on its target, in precedence order; `entry` is the first one the knobs allow, so under `kernel: generic` a shape-specialized entry stays listed ahead of the generic entry the layer calls, and a plan diff shows which entries a release added or removed. Convolution, depthwise and transpose convolution list their matching records; fully connected lists the packed int8 entry when its gate holds and the precise packed FP16 entry when the weights are packed, ahead of the plain entry; the depthwise FP16 entry `accumulation: fast` selects is listed where it applies. Null for pooling, batch matmul and SVDF, whose providers do not enumerate alternatives, and in older plans. The list is a fact about the release that wrote the plan: a later release can add an entry or change a rule. | | `kernel.requantization` | For int8 and int16 convolution, depthwise and fully connected layers: `rshift_only` when every output channel has a shift below 0 or a multiplier of 0, so the emitted multipliers and shifts allow `arm_nn_requantize_channels_rshift_only`'s route, and `general` otherwise. The int8 MVE kernels read it to skip the left shift and the rounding fixup, and it gates the packed fully connected entry; for depthwise, int16 and targets without MVE it describes the tables and no kernel reads it. Null for float layers, other operators and older plans. | | `kernel.local_scratch_bytes` | For operators with a recorded entry, the actual resolved local tensors, excluding output tensors, constant weight sums and arena alignment padding. Not the model's total scratch-arena requirement. Older plans can omit it; operators without a recorded entry stay unrecorded. | | `fixed` | Choices that change numerics or speed but are not knobs yet, with what set them (`attribute`, `default`, `heuristic`, `knob` or `placement`; `plan` is reserved): RSQRT `lut_mode` (INT16 only), HARD_SWISH `use_lut` and `compat_variant` (INT8), and the float `weight_layout` of 1x1 convolutions, conv1d and fully connected layers, set by the knob for fast packed FP16 fully connected, by placement for eligible batch-one FP32 or precise FP16 fully connected, and by a heuristic otherwise, including precise multirow FC and FP16 convolutions. | ### Per-constant fields `tensors[]` lists every constant the operators read, by storage root (a constant interned onto another's storage is listed under that root), in the order of its first reader. Plans from heliaAOT 1.3 and earlier write the list empty. | Field | What it records | | --- | --- | | `id`, `storage_id` | The id `memory.tensors` rules match; for a constant repacked at conversion, the id it was packed from, with the storage tensor's own id in `storage_id`. | | `requested`, `residency` | The residency the merged rules ask for (`direct`, `pin` or `stream`, or null), and the one the constant got. A `stream` rule permits streaming: the constant is `stream` when a schedule unit names it and `direct` otherwise. | | `memory`, `destination_memory` | Where the bytes are stored, and for `pin` the memory the kernels read the copy from. | | `bytes`, `alignment` | The size, and the alignment the layout pads the constant to, so a planner computes unit sizes and pool fit with the same rule heliaAOT applies. | | `unit`, `pool_offset` | For `stream`, the transfer unit and the constant's offset in the stream pool; null otherwise. | ### The transfer schedule The `schedule` section carries the transfer schedule of a conversion whose constants stream (`residency: stream` and `memory.stream_pool`; see [Memory placement](https://ambiqai.github.io/helia-aot/guide/memory-placement/#residency)). heliaAOT never searches for a schedule: a planner writes one into a plan, the conversion reads that plan with `optimization.plan`, checks the schedule and writes it back with its derived fields. A conversion with a schedule copies each unit into the pool before its first reader. This release emits `cpu` units, which the module copies itself, and refuses a `dma` unit. | Field | Kind | What it records | | --- | --- | --- | | `source` | input | Required: `plan` with at least one unit, or `none` with no units when nothing streams. | | `pool` | derived | The stream pool's `memory`, `bytes` and `alignment`, present whenever `memory.stream_pool` is configured. | | `units[].id` | input | The unit's position in the list, which is issue order: by the position of `issue_after` (null first), then by id. | | `units[].tensors` | input | The stored copies the unit copies, each named by its `tensors[].storage_id` when the entry has one, else its `tensors[].id`, in the order the conversion's `tensors[]` lists them (by first reader, then in that reader's order); each streamed copy is in exactly one unit, and only a constant whose rules permit `stream` may be in one. | | `units[].pool_offset` | input | The unit's offset in the pool, a multiple of its span alignment (the largest `alignment` among its tensors), with `pool_offset + bytes` within `pool.bytes`. | | `units[].issue_after` | input | The `operators[].id` after whose run the copy is issued, or null for the start of the run; it must come before the unit's first reader. | | `units[].engine` | input | `dma` or `cpu`, among the engines the source and pool memories offer on the target (MRAM to DTCM offers `cpu`; PSRAM to DTCM offers both). This release emits `cpu` units only and refuses `dma`. | | `units[].bytes` | derived | The span size: tensor k starts at the next multiple of its `alignment` from the end of tensor k-1, and `bytes` is the end of the last one. | | `units[].source_memory`, `units[].source_offset` | derived | The one memory the unit's constants are stored in, and the span's offset in that memory's constant arena, since the memory planner lays a unit out as one span on both sides of the transfer. A stated `source_offset` is ignored. | | `units[].wait_before`, `units[].last_reader` | derived | The first and last operator, by position, that reads a constant of the unit. | A unit is live from its issue point to the end of its last reader's run, so two units may share pool bytes only when their live ranges do not meet. The conversion rejects a schedule, naming the unit or tensor, when two units live at once overlap in pool bytes, a unit does not fit the pool or does not start at a multiple of its span alignment, a unit is not issued before its first reader, a unit streams a constant whose rules do not permit `stream` or one the report says cannot be staged (read at init, pinned, exposed as a model output, or sharing storage through an alias), a unit's engine is not offered or its constants are stored in two memories, a derived field is stated differently, or the schedule holds more than 65535 units. A constant whose rules permit `stream` and that no unit names stays `direct`, and the conversion names it in a warning when it could have streamed. A permitted constant that cannot be staged (read at init, a model output, storage shared through an alias, or fixed in place) never streams; a separate warning names it with its reason, as the report's `constants[].reason` records it. The report's `stream_pool.layout` lists each unit's span in the pool and `pieces` splits every unit at multiples of the pool memory's DMA boundary counted from the pool base; a provider receives a `dma` unit in those pieces, and the module copies a `cpu` unit in one copy. ### Reading a plan as input `optimization.plan` names a plan to read before the conversion starts: in a configuration file the path resolves relative to that file, on the command line relative to the working directory, and either way the validated configuration stores it absolute. The plan's `schedule` section is applied, so it is read strictly: a field heliaAOT does not know under `schedule`, `schedule.pool` or `schedule.units[]` is rejected, naming it, because a later release adds there only fields it applies. Unknown fields elsewhere in the plan are ignored, so a tool annotates the plan outside `schedule`. ```text optimization.plan is not an optimization plan this release reads: schedule.units.0: unknown field(s) ['pool'] under schedule, which this release would have to apply. hint: The field comes from a later heliaAOT release or a planner written for one: convert with that release, or remove the field. ``` For every other section the plan must describe the conversion the configuration produces, and the conversion rejects the plan naming the first field that differs and both values: - `model.sha256` and `model.subgraph` must match the model (a plan whose `sha256` is null is rejected), and `platform.name` the target; - `operators[]` are compared by position: `node_id`, `op_type`, and the `value` of every knob and fixed choice both sides name; - `tensors[]` are compared by `id` and `storage_id` when the plan lists any (an entry listed twice is rejected): a constant on one side only is a rejection, and `requested`, `residency`, `memory`, `destination_memory`, `bytes` and `alignment` must match, memory names in any case, except `residency` for a constant whose rules permit `stream`, where `direct` and `stream` both match because the schedule decides between them. A null `memory` in the plan is not compared; a null `requested` (no rule) or `destination_memory` (not staged) is a value and must match. `generator`, `module_prefix`, the `optimization` section, `operators[].id`, `id_source` and `kernel`, `tensors[].unit` and `pool_offset`, unknown fields outside `schedule` and the report are not compared, so a plan written by another heliaAOT release is accepted when the comparison passes and its `schedule` holds only fields this release knows. A plan whose `schedule.source` is `none` replays the conversion that wrote it, and a plan heliaAOT wrote with a schedule reads back after only its `schedule` is edited. ## The report `_report.json` carries each operator's MACs, the alternatives of each knob and whether they apply on this target and why not, the measurements for this target and CPU keyed by setting, the constant placement facts, the hints the results print, the `folded_variables` list, the constants each operator reads with whether they could be staged into a stream pool, the stream pool and the schedule's pieces. Its content may change between releases, except the planner facts of RFC 0005 (each operator's `constants[]`, and the `memory`, `bytes` and `alignment` of `stream_pool`), which are stable within schema version 1; the pool's configured alignment and layout and the pieces are explanatory. The choices are in the plan. The optional HTML report (`documentation.html`) shows the plan's `kernel.local_scratch_bytes` as the local-scratch column of its execution table. - A hint appears only when its alternative applies and a single setting would select it. A knob hint lists the operators where the knob applies that do not run the value `auto` picks for `goal: latency` with `allow_approximate: true` (for accumulation, operators running `precise` where the `fast` alternative applies), leaving out operators an `operators` rule pins. - The placement hint needs constants outside TCM that all come from one cold memory and fit in the free DTCM. - Share of MACs counts convolution, depthwise and fully connected layers; it is not a share of run time unless you profile the model. ## Schema versions Both files carry a `schema_version`. Adding an optional field, a knob, a knob value or an enum member keeps it; renaming or removing a field, or changing what one means, increments it. - `OptimizationPlan` and `OptimizationReport` in `helia_aot.optimization` read both files back as typed models by that rule: they ignore a field they do not know, keep a knob, knob value or member they do not know as the string written, and reject only a file with another `schema_version`. - The plan's `schedule` section is the exception when a conversion reads the plan as input (`optimization.plan`): heliaAOT applies it, so it rejects a field it does not know under `schedule`, `schedule.pool` or `schedule.units[]` (validating with `context=APPLIED` does the same). A release adds a field there only when it applies it, keeping `schema_version`; derived facts go outside `schedule`, and an applied field outside it comes only with a new `schema_version`. - A script that uses them reads the files of older and newer releases with the same `schema_version`. - Plans and reports from heliaAOT 0.30 and earlier name the target's CPU `core`; they still read, as `cpu`. - Under its `json_schema` key, each exported schema lists every field, knob, value and member the release that exported it writes, and allows no other field (`additionalProperties: false`) except in one free-form object: a report hint's `facts`. - Validate a file against the schema of the release that wrote it (`generator.helia_aot`) rather than pinning one; a newer file need not validate against an older schema. - [Compatibility](https://ambiqai.github.io/helia-aot/reference/compatibility/#machine-readable-files) says what a release promises about each file. ## Generated CORE calls The report's `kernel_calls` records the CORE functions every operator's generated source calls, including batch matmul's operand transposes, SVDF's kernel-sum call for runtime feature weights and the shared ADD/MUL helpers. Each `calls` entry holds a `symbol` and the `operator_ids` from the plan; symbols are sorted and attribution follows execution order. - An operator that calls no CORE function, such as DILATE or an Ethos-U command stream, records none. A call compiled only under `ARM_MATH_MVEI`, such as REVERSE_V2's row copy, is recorded on every target. Initialization calls are included when emitted; offline precomputation is not a generated call. - The records use the same selected symbols as code generation. They list direct generated calls, including calls inside shared generated helpers, not the functions those CORE entries call internally or the final linked image. `unrecorded_operator_ids` lists operators without call coverage, so a partial list is not the module's complete dependencies. - Older reports have no `kernel_calls` field, read as null, which differs from an empty, covered call list. The generated HTML report shows the same records and coverage gaps. - The record changes no selection, scratch sizing or numerical behavior, and it is not an upstream kernel manifest. ## FP16 overflow thresholds A packed FP16 convolution on `arm_convolve_1x1_f16` or `arm_convolve_1_x_n_f16` sums each output in float16 lanes: the bias and the first 32 taps, then each later block of 32 taps, each folded into float32. A block partial that reaches 65504 overflows whatever a later block subtracts, so unnormalized data on a layer fed straight from a model input can overflow where the standard layout's shorter sums would not. The report's `fp16_overflow` lists each such layer whose input is a model input read through value-preserving operators only (RESHAPE, SQUEEZE, EXPAND_DIMS, TRANSPOSE, SLICE, STRIDED_SLICE, and PAD with a zero fill value). Each entry names the operator, its entry, the model input, the operator types on the path, and `threshold`, the input magnitude below which no sign pattern can make any lane block reach 65504, allowing for float16 rounding. - After `n` round-to-nearest fused multiply-adds a block partial is at most `(1 + g)` times the exact sum of magnitudes, with `g = n·u / (1 − n·u)` and `u = 2^-11` (`n` is the block's taps, plus one for the bias in block 0). - The threshold is `(65504 / (1 + g) − |bias|) / sum|w|` over the first block and `65504 / (1 + g) / sum|w|` over each later one, minimized over output channels and blocks, with the `channel`, `block`, `weight_sum` and `bias` attaining it. - At larger magnitudes the worst sign pattern overflows within about 2% above the threshold; typical data is far below it. - The threshold describes the lane-block artifact only: below it no lane block overflows, but the layer's float16 output can still overflow when its exact value exceeds the float16 range (a reduction longer than 32 taps is narrowed from float32 to float16 at the output), as on any float16 kernel. - A layer on another entry, with standard weights, or fed through any other operator, a PAD with another fill value or a constant is not listed, whatever its exposure. The list is a fact about the weights; it changes no generated code and no conversion decision.