Plans and reports

A conversion writes two JSON files next to the module. <prefix>_plan.json holds every choice: the value of each knob for each operator and the kernel entry each layer calls. <prefix>_report.json explains the plan and never changes it. Their JSON schemas ship with the documentation data as plan-schema.json and report-schema.json.

The plan

Field	What it records
schema_version	The file’s schema; see Schema versions.
generator.helia_aot, generator.cmsis_nn_required	The heliaAOT release that wrote the plan and the ns-cmsis-nn version the module requires.
model.file, model.sha256, model.subgraph	The model the plan was made from.
module_prefix	The module’s symbol prefix.
platform.name, platform.cpu, platform.capabilities, platform.supports_fp16	The target. The capabilities are what the release that wrote the plan knew of it; a later release can remove one as well as add one.
optimization.goal, optimization.allow_approximate	The goal and the approximation gate.
operators[]	One entry per operator, below.
tensors[]	One entry per constant the operators read, below.
schedule	The transfer schedule, below; source: none with no units when nothing streams.
optimization.knobs	Reserved and empty today, for model-level choices.

Operator ids

Each operator has a stable id, "<subgraph>:<output tensor name>", taken from the name the model gives its first output tensor, so it survives edits that add or remove other operators. index keeps its position in execution order and node_id its AIR node id.

• Repeated names get #1, #2 in operator order. An operator without a name falls back to "<subgraph>:<op_type>@<position>"; id_source says which applied.
• A tensor name can itself contain : (for example 0:StatefulPartitionedCall_1:0), so a parser splits these ids on their first colon only. An id with id_source: node_id is the opaque AIR node id and is not split.
• operators[].id rules match the stable id as well as the node id, so a plan’s ids can be written back as rules.
• A stable id that would also select another operator (another operator’s node id, or a stable id spelled twice) is not matched; the plan then uses the node id as id, with id_source: node_id, and the conversion warns.

Per-operator fields

Field	What it records
each knob	The value used, the value requested, resolved_by (explicit, or auto_table when auto chose it from the goal; auto_heuristic and plan are reserved), whether the value is approximate where it applies, and the source: kind is operator (with rule, the position, type and id of the winning operators entry), model (set under optimization, even to auto), or default (plan is reserved).
kernel.entry	The ns-cmsis-nn entry chosen, for layers that choose among candidates (convolution, depthwise, transpose convolution, packed int8 fully connected, precise packed FP16 fully connected); the report repeats it under entry. Average and max pooling record their selected int8, int16, FP16 or FP32 entry. Pooling, convolution, depthwise, transpose convolution, fully connected, batch matmul and SVDF record their single-entry paths explicitly, FP16, FP32 and ordinary fully connected entries included; other unmigrated single-entry layers still have a null entry. Selection follows the rules and knobs; a plan cannot set it yet.
kernel.provider	An opaque string naming the built-in CORE planning rules, for pooling, convolution, depthwise, transpose convolution, fully connected, batch matmul and SVDF. Optional and absent from older plans; compare it with another plan’s, do not parse it.
kernel.candidates	Every entry whose rules match the layer on its target, in precedence order; entry is the first one the knobs allow, so under kernel: generic a shape-specialized entry stays listed ahead of the generic entry the layer calls, and a plan diff shows which entries a release added or removed. Convolution, depthwise and transpose convolution list their matching records; fully connected lists the packed int8 entry when its gate holds and the precise packed FP16 entry when the weights are packed, ahead of the plain entry; the depthwise FP16 entry accumulation: fast selects is listed where it applies. Null for pooling, batch matmul and SVDF, whose providers do not enumerate alternatives, and in older plans. The list is a fact about the release that wrote the plan: a later release can add an entry or change a rule.
kernel.requantization	For int8 and int16 convolution, depthwise and fully connected layers: rshift_only when every output channel has a shift below 0 or a multiplier of 0, so the emitted multipliers and shifts allow arm_nn_requantize_channels_rshift_only’s route, and general otherwise. The int8 MVE kernels read it to skip the left shift and the rounding fixup, and it gates the packed fully connected entry; for depthwise, int16 and targets without MVE it describes the tables and no kernel reads it. Null for float layers, other operators and older plans.
kernel.local_scratch_bytes	For operators with a recorded entry, the actual resolved local tensors, excluding output tensors, constant weight sums and arena alignment padding. Not the model’s total scratch-arena requirement. Older plans can omit it; operators without a recorded entry stay unrecorded.
fixed	Choices that change numerics or speed but are not knobs yet, with what set them (attribute, default, heuristic, knob or placement; plan is reserved): RSQRT lut_mode (INT16 only), HARD_SWISH use_lut and compat_variant (INT8), and the float weight_layout of 1x1 convolutions, conv1d and fully connected layers, set by the knob for fast packed FP16 fully connected, by placement for eligible batch-one FP32 or precise FP16 fully connected, and by a heuristic otherwise, including precise multirow FC and FP16 convolutions.

Per-constant fields

tensors[] lists every constant the operators read, by storage root (a constant interned onto another’s storage is listed under that root), in the order of its first reader. Plans from heliaAOT 1.3 and earlier write the list empty.

Field	What it records
id, storage_id	The id memory.tensors rules match; for a constant repacked at conversion, the id it was packed from, with the storage tensor’s own id in storage_id.
requested, residency	The residency the merged rules ask for (direct, pin or stream, or null), and the one the constant got. A stream rule permits streaming: the constant is stream when a schedule unit names it and direct otherwise.
memory, destination_memory	Where the bytes are stored, and for pin the memory the kernels read the copy from.
bytes, alignment	The size, and the alignment the layout pads the constant to, so a planner computes unit sizes and pool fit with the same rule heliaAOT applies.
unit, pool_offset	For stream, the transfer unit and the constant’s offset in the stream pool; null otherwise.

The transfer schedule

The schedule section carries the transfer schedule of a conversion whose constants stream (residency: stream and memory.stream_pool; see Memory placement). heliaAOT never searches for a schedule: a planner writes one into a plan, the conversion reads that plan with optimization.plan, checks the schedule and writes it back with its derived fields. A conversion with a schedule copies each unit into the pool before its first reader. This release emits cpu units, which the module copies itself, and refuses a dma unit.

Field	Kind	What it records
source	input	Required: plan with at least one unit, or none with no units when nothing streams.
pool	derived	The stream pool’s memory, bytes and alignment, present whenever memory.stream_pool is configured.
units[].id	input	The unit’s position in the list, which is issue order: by the position of issue_after (null first), then by id.
units[].tensors	input	The stored copies the unit copies, each named by its tensors[].storage_id when the entry has one, else its tensors[].id, in the order the conversion’s tensors[] lists them (by first reader, then in that reader’s order); each streamed copy is in exactly one unit, and only a constant whose rules permit stream may be in one.
units[].pool_offset	input	The unit’s offset in the pool, a multiple of its span alignment (the largest alignment among its tensors), with pool_offset + bytes within pool.bytes.
units[].issue_after	input	The operators[].id after whose run the copy is issued, or null for the start of the run; it must come before the unit’s first reader.
units[].engine	input	dma or cpu, among the engines the source and pool memories offer on the target (MRAM to DTCM offers cpu; PSRAM to DTCM offers both). This release emits cpu units only and refuses dma.
units[].bytes	derived	The span size: tensor k starts at the next multiple of its alignment from the end of tensor k-1, and bytes is the end of the last one.
units[].source_memory, units[].source_offset	derived	The one memory the unit’s constants are stored in, and the span’s offset in that memory’s constant arena, since the memory planner lays a unit out as one span on both sides of the transfer. A stated source_offset is ignored.
units[].wait_before, units[].last_reader	derived	The first and last operator, by position, that reads a constant of the unit.

A unit is live from its issue point to the end of its last reader’s run, so two units may share pool bytes only when their live ranges do not meet. The conversion rejects a schedule, naming the unit or tensor, when two units live at once overlap in pool bytes, a unit does not fit the pool or does not start at a multiple of its span alignment, a unit is not issued before its first reader, a unit streams a constant whose rules do not permit stream or one the report says cannot be staged (read at init, pinned, exposed as a model output, or sharing storage through an alias), a unit’s engine is not offered or its constants are stored in two memories, a derived field is stated differently, or the schedule holds more than 65535 units. A constant whose rules permit stream and that no unit names stays direct, and the conversion names it in a warning when it could have streamed. A permitted constant that cannot be staged (read at init, a model output, storage shared through an alias, or fixed in place) never streams; a separate warning names it with its reason, as the report’s constants[].reason records it. The report’s stream_pool.layout lists each unit’s span in the pool and pieces splits every unit at multiples of the pool memory’s DMA boundary counted from the pool base; a provider receives a dma unit in those pieces, and the module copies a cpu unit in one copy.

Reading a plan as input

optimization.plan names a plan to read before the conversion starts: in a configuration file the path resolves relative to that file, on the command line relative to the working directory, and either way the validated configuration stores it absolute. The plan’s schedule section is applied, so it is read strictly: a field heliaAOT does not know under schedule, schedule.pool or schedule.units[] is rejected, naming it, because a later release adds there only fields it applies. Unknown fields elsewhere in the plan are ignored, so a tool annotates the plan outside schedule.

optimization.plan <path> is not an optimization plan this release reads: schedule.units.0: unknown field(s) ['pool'] under schedule, which this release would have to apply.
hint: The field comes from a later heliaAOT release or a planner written for one: convert with that release, or remove the field.

For every other section the plan must describe the conversion the configuration produces, and the conversion rejects the plan naming the first field that differs and both values:

• model.sha256 and model.subgraph must match the model (a plan whose sha256 is null is rejected), and platform.name the target;
• operators[] are compared by position: node_id, op_type, and the value of every knob and fixed choice both sides name;
• tensors[] are compared by id and storage_id when the plan lists any (an entry listed twice is rejected): a constant on one side only is a rejection, and requested, residency, memory, destination_memory, bytes and alignment must match, memory names in any case, except residency for a constant whose rules permit stream, where direct and stream both match because the schedule decides between them. A null memory in the plan is not compared; a null requested (no rule) or destination_memory (not staged) is a value and must match.

generator, module_prefix, the optimization section, operators[].id, id_source and kernel, tensors[].unit and pool_offset, unknown fields outside schedule and the report are not compared, so a plan written by another heliaAOT release is accepted when the comparison passes and its schedule holds only fields this release knows. A plan whose schedule.source is none replays the conversion that wrote it, and a plan heliaAOT wrote with a schedule reads back after only its schedule is edited.

The report

<prefix>_report.json carries each operator’s MACs, the alternatives of each knob and whether they apply on this target and why not, the measurements for this target and CPU keyed by setting, the constant placement facts, the hints the results print, the folded_variables list, the constants each operator reads with whether they could be staged into a stream pool, the stream pool and the schedule’s pieces. Its content may change between releases, except the planner facts of RFC 0005 (each operator’s constants[], and the memory, bytes and alignment of stream_pool), which are stable within schema version 1; the pool’s configured alignment and layout and the pieces are explanatory. The choices are in the plan. The optional HTML report (documentation.html) shows the plan’s kernel.local_scratch_bytes as the local-scratch column of its execution table.

• A hint appears only when its alternative applies and a single setting would select it. A knob hint lists the operators where the knob applies that do not run the value auto picks for goal: latency with allow_approximate: true (for accumulation, operators running precise where the fast alternative applies), leaving out operators an operators rule pins.
• The placement hint needs constants outside TCM that all come from one cold memory and fit in the free DTCM.
• Share of MACs counts convolution, depthwise and fully connected layers; it is not a share of run time unless you profile the model.

Schema versions

Both files carry a schema_version. Adding an optional field, a knob, a knob value or an enum member keeps it; renaming or removing a field, or changing what one means, increments it.

• OptimizationPlan and OptimizationReport in helia_aot.optimization read both files back as typed models by that rule: they ignore a field they do not know, keep a knob, knob value or member they do not know as the string written, and reject only a file with another schema_version.
• The plan’s schedule section is the exception when a conversion reads the plan as input (optimization.plan): heliaAOT applies it, so it rejects a field it does not know under schedule, schedule.pool or schedule.units[] (validating with context=APPLIED does the same). A release adds a field there only when it applies it, keeping schema_version; derived facts go outside schedule, and an applied field outside it comes only with a new schema_version.
• A script that uses them reads the files of older and newer releases with the same schema_version.
• Plans and reports from heliaAOT 0.30 and earlier name the target’s CPU core; they still read, as cpu.
• Under its json_schema key, each exported schema lists every field, knob, value and member the release that exported it writes, and allows no other field (additionalProperties: false) except in one free-form object: a report hint’s facts.
• Validate a file against the schema of the release that wrote it (generator.helia_aot) rather than pinning one; a newer file need not validate against an older schema.
• Compatibility says what a release promises about each file.

Generated CORE calls

The report’s kernel_calls records the CORE functions every operator’s generated source calls, including batch matmul’s operand transposes, SVDF’s kernel-sum call for runtime feature weights and the shared ADD/MUL helpers. Each calls entry holds a symbol and the operator_ids from the plan; symbols are sorted and attribution follows execution order.

• An operator that calls no CORE function, such as DILATE or an Ethos-U command stream, records none. A call compiled only under ARM_MATH_MVEI, such as REVERSE_V2’s row copy, is recorded on every target. Initialization calls are included when emitted; offline precomputation is not a generated call.
• The records use the same selected symbols as code generation. They list direct generated calls, including calls inside shared generated helpers, not the functions those CORE entries call internally or the final linked image. unrecorded_operator_ids lists operators without call coverage, so a partial list is not the module’s complete dependencies.
• Older reports have no kernel_calls field, read as null, which differs from an empty, covered call list. The generated HTML report shows the same records and coverage gaps.
• The record changes no selection, scratch sizing or numerical behavior, and it is not an upstream kernel manifest.

FP16 overflow thresholds

A packed FP16 convolution on arm_convolve_1x1_f16 or arm_convolve_1_x_n_f16 sums each output in float16 lanes: the bias and the first 32 taps, then each later block of 32 taps, each folded into float32.

A block partial that reaches 65504 overflows whatever a later block subtracts, so unnormalized data on a layer fed straight from a model input can overflow where the standard layout’s shorter sums would not.

The report’s fp16_overflow lists each such layer whose input is a model input read through value-preserving operators only (RESHAPE, SQUEEZE, EXPAND_DIMS, TRANSPOSE, SLICE, STRIDED_SLICE, and PAD with a zero fill value).

Each entry names the operator, its entry, the model input, the operator types on the path, and threshold, the input magnitude below which no sign pattern can make any lane block reach 65504, allowing for float16 rounding.

• After n round-to-nearest fused multiply-adds a block partial is at most (1 + g) times the exact sum of magnitudes, with g = n·u / (1 − n·u) and u = 2^-11 (n is the block’s taps, plus one for the bias in block 0).
• The threshold is (65504 / (1 + g) − |bias|) / sum|w| over the first block and 65504 / (1 + g) / sum|w| over each later one, minimized over output channels and blocks, with the channel, block, weight_sum and bias attaining it.
• At larger magnitudes the worst sign pattern overflows within about 2% above the threshold; typical data is far below it.
• The threshold describes the lane-block artifact only: below it no lane block overflows, but the layer’s float16 output can still overflow when its exact value exceeds the float16 range (a reduction longer than 32 taps is narrowed from float32 to float16 at the output), as on any float16 kernel.
• A layer on another entry, with standard weights, or fed through any other operator, a PAD with another fill value or a constant is not listed, whatever its exposure. The list is a fact about the weights; it changes no generated code and no conversion decision.