Memory placement

Decide where the two big objects of a run live: the tensor arena (activations and scratch) and the model weights (the flatbuffer, the AOT constant blobs or the PTE program). Placement can affect cycle counts and power. This page covers the tiers, the automatic policy, the explicit controls, the tuning loop and the engine differences.

Prerequisites

A model that profiles with default placement, so you have a Memory table and an allocated_arena to start from (First profile).

Which tier

Tier	What it is	Relative latency	Boot-time copy	arena_location	weights_location
TCM (DTCM)	Tightly coupled memory inside the Cortex-M55 core; AP3 and AP4 parts expose one combined TCM. hpx places data in DTCM only; ITCM is for code.	single-cycle	yes, from flash at boot	tcm	tcm
SRAM	Shared on-chip SRAM outside the core	a few cycles	yes	sram	sram
MRAM	On-chip non-volatile memory; program and read-only data	slowest on chip	no; read in place	not allowed	mram
PSRAM	External pseudo-static RAM over the external bus	external bus	no; uploaded or written at run time	psram	psram

Capacities differ per SoC and are registered in src/helia_profiler/platform/soc.py (MemoryLayout: dtcm_kb, sram_kb, mram_kb, psram_kb, with some boards overriding psram_kb). The run report prints the capacity next to the usage for every region, which is the number to trust for your board.

The memory plan

Memory planningConsumers on the left are placed into the regions on the right by the planner or by the two placement keys.

• Consumers
  • Arenaactivations, scratch
  • Weightsflatbuffer, blobs, PTE
  • Firmwarestack, heap, BSS slack
• Plannerfastest fit, arena first
• Regions
  • DTCM
  • SRAM
  • MRAM
  • PSRAMopt-in only

Text description of the diagram

• Consumers
  • Arena: activations, scratch
  • Weights: flatbuffer, blobs, PTE
  • Firmware: stack, heap, BSS slack
• Planner: fastest fit, arena first
• Regions
  • DTCM
  • SRAM
  • MRAM
  • PSRAM: opt-in only

Steps

1. Read what automatic placement chose

Run with no placement keys and read the terminal report. It prints a measured table per region (used, reserved, free, from the linked ELF against the SoC’s verified windows); the planning table replaces it only when the measured view is unavailable, for instance on a custom SoC:

Memory (measured) (gnu link):          # captured on atomiq110_fpga_turbo; capacities are that SoC's
  DTCM     16.3 KB   479.7 KB  of 496.0 KB   3%   Reserved 479.7 KB
  MRAM     45.0 KB   3.9 MB    of 3.9 MB     1%

summary.json carries both: memory_regions (measured) and memory_plan (the pre-build decision with regions[], each with region, capacity, used and named consumers). memory.allocated_arena is what the firmware actually used of the arena you requested.

python -c "import json; d=json.load(open('results/summary.json')); print(d['memory']['allocated_arena'], d['memory_plan']['regions'])"

2. Understand the policy

With both keys omitted the interpreter engines use a greedy fastest-fit policy with the arena placed before the weights, walking down the tiers:

1. Both arena and weights fit in TCM: both go to TCM.
2. Otherwise arena in TCM, weights in SRAM (or MRAM if SRAM is full).
3. Otherwise arena in SRAM, weights in MRAM.
4. Otherwise the placement validator reports an overflow before any build.

The arena wins ties because every inference touches it; weights stream once per layer and gain less from a single-cycle tier. A slack budget is reserved in TCM and SRAM for stack, heap and BSS so the firmware still links. Automatic placement never chooses PSRAM.

3. Pin placement

model:
  path: model.tflite
  arena_size: 65536
  arena_location: tcm
  weights_location: mram

hpx profile model.tflite --arena-location tcm --weights-location mram

arena_location accepts tcm, sram or psram; weights_location accepts tcm, sram, mram or psram. Pin both to one region to place both objects together. Pin placement whenever a comparison must not drift: a small model-size change can otherwise cross a tier boundary between two runs and the delta is the tier, not the change you made.

4. Compare placements fairly

Hold every other axis constant and flip only the placement keys:

hpx profile model.tflite --arena-location tcm --weights-location tcm  --output-dir results/place_tcm_tcm
hpx profile model.tflite --arena-location tcm --weights-location mram --output-dir results/place_tcm_mram
hpx compare results/place_tcm_tcm results/place_tcm_mram

Report both the absolute measurements and the change, with the model, clock, compiler and placement recorded. Neither counts nor ratios are guaranteed to transfer to a different configuration. Memory placement alone does not establish which regions were power-gated; use a configured power capture to measure the effect.

5. Tighten the arena

arena_size is a request. The interpreter default is 256 KiB; ExecuTorch can use the sidecar’s planned_arena_size. AOT plans its own arenas unless external-arena mode is enabled. For an interpreter, use the reported allocated_arena as a starting point, retain headroom for alignment and model/runtime changes, then run again to check allocation and region capacity.

6. PSRAM

PSRAM is opt-in with weights_location: psram (or --weights-location psram). How the data gets there depends on the engine:

• tflm and helia-rt: the firmware initialises PSRAM and signals the host, which uploads the model over J-Link. This path requires transport: rtt.
• helia-aot: the firmware writes its own flash-resident constant blobs into PSRAM at boot, with no host upload and no transport requirement. The path exists only in external-arena mode, so PSRAM placement also requires engine.config.aot_args.memory.allocate_arenas: false; otherwise preflight refuses the run, because the generated firmware would contain no PSRAM code while the plan claims tensors live there.

Select the PSRAM interface clock under the target; hpx validates the choice and the resolved neuralSPOT-X board module rejects clocks the part does not offer. The default is 48000000 (PsramConfig.clock_hz):

target:
  board: apollo510b_evb
  psram:
    clock_hz: 125000000

hpx reports the initialised capacity, clock, XIP state, timing status and RXDQS delay in summary.json, run_metadata.json and the terminal memory panel.

7. What an overflow looks like

A placement that does not fit fails before any firmware is built:

Error: Memory plan does not fit:        # an Apollo5-family part; the capacity is that SoC's registry value
  DTCM: 540672 B used > 524288 B capacity (over by 16384 B)
  Hint: DTCM is over capacity.  Try one of:
    * shrink the tensor arena (--arena-size);
    * pick a less-aggressive arena or weights placement;
    * move weights to PSRAM (--weights-location psram) if the
      board has PSRAM;
    * reduce model size (quantise / prune); or
    * pick a larger-memory board.

Requesting a region the board does not have (tcm on a board without DTCM, psram with no PSRAM fitted) fails the same way.

Placement models

	heliaRT and TFLM	heliaAOT	ExecuTorch
Placement unit	one arena, one weights blob	individual tensors (or tensor kinds)	five fixed runtime buffers
Who owns the memory	the interpreter	the generated code; every tensor is a named C object	the caller; the runtime receives five opaque blocks
Configuration	arena_location, weights_location	the same two keys as coarse control, plus engine.config.aot_args.memory.tensors rules over constant, persistent and scratch	arena_location (tcm or sram) for all five, plus planned_arena_location, method_arena_location, temporary_arena_location, io_location
Activations	packed in the arena	each tensor placeable	packed into the planned arena at export time; invisible to the firmware

heliaAOT per-tensor placement

The AOT compiler emits per-tensor section attributes (PUT_IN_DTCM, PUT_IN_SRAM and so on). The two placement keys give coarse control: arena placement covers scratch and persistent tensors, weights placement covers constants. engine.config.aot_args.memory.tensors rules over constant, persistent and scratch override the coarse keys.

ExecuTorch buffers

Only the planned arena is touched by every kernel on every inference, so it is the one buffer that rewards DTCM; the method arena (cold after load), the temporary arena (kernel scratch) and the I/O buffers (one copy per inference) belong in SRAM. Rough vocabulary: heliaAOT scratch is roughly the planned plus temporary arenas, persistent roughly the method arena, constant the PTE program placed by weights_location. Per-tensor placement inside the planned arena is not available: the export-time planner bakes each activation’s offset into the PTE, so the firmware’s finest unit is that arena. ExecuTorch’s planner supports several planned buffers, so a future export option could split activations across two arenas; tracked in nsx-executorch#3.

Expected output

A run whose report shows every region under capacity, an allocated_arena that matches the request with a little headroom, and run_metadata.json recording the resolved placement so a comparison can prove both runs used the same one.

What next

Engines for the engine whose placement model you are using; the Python API for the placement types when you script runs.