# Memory placement

Decide where the two big objects of a run live: the tensor arena
(activations and scratch) and the model weights (the flatbuffer, the AOT
constant blobs or the PTE program). Placement can affect cycle counts and power. This page covers the tiers, the automatic
policy, the explicit controls, the tuning loop and the engine differences.

## Prerequisites

A model that profiles with default placement, so you have a Memory table
and an `allocated_arena` to start from ([First profile](https://ambiqai.github.io/helia-profiler/getting-started/first-profile/)).

## Which tier

| Tier | What it is | Relative latency | Boot-time copy | `arena_location` | `weights_location` |
|---|---|---|---|---|---|
| TCM (DTCM) | Tightly coupled memory inside the Cortex-M55 core; AP3 and AP4 parts expose one combined TCM. hpx places data in DTCM only; ITCM is for code. | single-cycle | yes, from flash at boot | `tcm` | `tcm` |
| SRAM | Shared on-chip SRAM outside the core | a few cycles | yes | `sram` | `sram` |
| MRAM | On-chip non-volatile memory; program and read-only data | slowest on chip | no; read in place | not allowed | `mram` |
| PSRAM | External pseudo-static RAM over the external bus | external bus | no; uploaded or written at run time | `psram` | `psram` |

Capacities differ per SoC and are registered in
`src/helia_profiler/platform/soc.py` (`MemoryLayout`: `dtcm_kb`, `sram_kb`,
`mram_kb`, `psram_kb`, with some boards overriding `psram_kb`). The run
report prints the capacity next to the usage for every region, which is
the number to trust for your board.

## The memory plan

Memory planning: Consumers on the left are placed into the regions on the right by the planner or by the two placement keys.

- Consumers
  - Arena: activations, scratch
  - Weights: flatbuffer, blobs, PTE
  - Firmware: stack, heap, BSS slack
- Planner: fastest fit, arena first
- Regions
  - DTCM
  - SRAM
  - MRAM
  - PSRAM: opt-in only
Text description of the diagram

- Consumers
- Arena: activations, scratch
- Weights: flatbuffer, blobs, PTE
- Firmware: stack, heap, BSS slack
- Planner: fastest fit, arena first
- Regions
- DTCM
- SRAM
- MRAM
- PSRAM: opt-in only

## Steps

### 1. Read what automatic placement chose

Run with no placement keys and read the terminal report. It prints a
measured table per region (used, reserved, free, from the linked ELF
against the SoC's verified windows); the planning table replaces it only
when the measured view is unavailable, for instance on a custom SoC:

```text
Memory (measured) (gnu link):          # captured on atomiq110_fpga_turbo; capacities are that SoC's
  DTCM     16.3 KB   479.7 KB  of 496.0 KB   3%   Reserved 479.7 KB
  MRAM     45.0 KB   3.9 MB    of 3.9 MB     1%
```

`summary.json` carries both: `memory_regions` (measured) and `memory_plan`
(the pre-build decision with `regions[]`, each with `region`, `capacity`,
`used` and named `consumers`). `memory.allocated_arena` is what the
firmware actually used of the arena you requested.

```bash
python -c "import json; d=json.load(open('results/summary.json')); print(d['memory']['allocated_arena'], d['memory_plan']['regions'])"
```

### 2. Understand the policy

With both keys omitted the interpreter engines use a greedy fastest-fit
policy with the arena placed before the weights, walking down the tiers:

1. Both arena and weights fit in TCM: both go to TCM.
2. Otherwise arena in TCM, weights in SRAM (or MRAM if SRAM is full).
3. Otherwise arena in SRAM, weights in MRAM.
4. Otherwise the placement validator reports an overflow before any build.

The arena wins ties because every inference touches it; weights stream
once per layer and gain less from a single-cycle tier. A slack budget is
reserved in TCM and SRAM for stack, heap and BSS so the firmware still
links. Automatic placement never chooses PSRAM.

### 3. Pin placement

```yaml
model:
  path: model.tflite
  arena_size: 65536
  arena_location: tcm
  weights_location: mram
```

```bash
hpx profile model.tflite --arena-location tcm --weights-location mram
```

`arena_location` accepts `tcm`, `sram` or `psram`; `weights_location`
accepts `tcm`, `sram`, `mram` or `psram`. Pin both to one region to place
both objects together. Pin placement whenever a comparison must not drift:
a small model-size change can otherwise cross a tier boundary between two
runs and the delta is the tier, not the change you made.

### 4. Compare placements fairly

Hold every other axis constant and flip only the placement keys:

```bash
hpx profile model.tflite --arena-location tcm --weights-location tcm  --output-dir results/place_tcm_tcm
hpx profile model.tflite --arena-location tcm --weights-location mram --output-dir results/place_tcm_mram
hpx compare results/place_tcm_tcm results/place_tcm_mram
```

Report both the absolute measurements and the change, with the model,
clock, compiler and placement recorded. Neither counts nor ratios are
guaranteed to transfer to a different configuration. Memory placement alone
does not establish which regions were power-gated; use a configured power
capture to measure the effect.

### 5. Tighten the arena

`arena_size` is a request. The interpreter default is 256 KiB; ExecuTorch
can use the sidecar's `planned_arena_size`. AOT plans its own arenas unless
external-arena mode is enabled. For an interpreter, use the reported
`allocated_arena` as a starting point, retain headroom for alignment and
model/runtime changes, then run again to check allocation and region capacity.

### 6. PSRAM

PSRAM is opt-in with `weights_location: psram` (or `--weights-location
psram`). How the data gets there depends on the engine:

- `tflm` and `helia-rt`: the firmware initialises PSRAM and signals the
  host, which uploads the model over J-Link. This path requires
  `transport: rtt`.
- `helia-aot`: the firmware writes its own flash-resident constant blobs
  into PSRAM at boot, with no host upload and no transport requirement.
  The path exists only in external-arena mode, so PSRAM placement also
  requires `engine.config.aot_args.memory.allocate_arenas: false`;
  otherwise preflight refuses the run, because the generated firmware would
  contain no PSRAM code while the plan claims tensors live there.

Select the PSRAM interface clock under the target; hpx validates the
choice and the resolved neuralSPOT-X board module rejects clocks the part
does not offer. The default is `48000000` (`PsramConfig.clock_hz`):

```yaml
target:
  board: apollo510b_evb
  psram:
    clock_hz: 125000000
```

hpx reports the initialised capacity, clock, XIP state, timing status and
RXDQS delay in `summary.json`, `run_metadata.json` and the terminal memory
panel.

### 7. What an overflow looks like

A placement that does not fit fails before any firmware is built:

```text
Error: Memory plan does not fit:        # an Apollo5-family part; the capacity is that SoC's registry value
  DTCM: 540672 B used > 524288 B capacity (over by 16384 B)
  Hint: DTCM is over capacity.  Try one of:
    * shrink the tensor arena (--arena-size);
    * pick a less-aggressive arena or weights placement;
    * move weights to PSRAM (--weights-location psram) if the
      board has PSRAM;
    * reduce model size (quantise / prune); or
    * pick a larger-memory board.
```

Requesting a region the board does not have (`tcm` on a board without
DTCM, `psram` with no PSRAM fitted) fails the same way.

## Placement models

| | heliaRT and TFLM | heliaAOT | ExecuTorch |
|---|---|---|---|
| Placement unit | one arena, one weights blob | individual tensors (or tensor kinds) | five fixed runtime buffers |
| Who owns the memory | the interpreter | the generated code; every tensor is a named C object | the caller; the runtime receives five opaque blocks |
| Configuration | `arena_location`, `weights_location` | the same two keys as coarse control, plus `engine.config.aot_args.memory.tensors` rules over `constant`, `persistent` and `scratch` | `arena_location` (`tcm` or `sram`) for all five, plus `planned_arena_location`, `method_arena_location`, `temporary_arena_location`, `io_location` |
| Activations | packed in the arena | each tensor placeable | packed into the planned arena at export time; invisible to the firmware |

### heliaAOT per-tensor placement

The AOT compiler emits per-tensor section attributes (`PUT_IN_DTCM`,
`PUT_IN_SRAM` and so on). The two placement keys give coarse control:
arena placement covers scratch and persistent tensors, weights placement
covers constants. `engine.config.aot_args.memory.tensors` rules over
`constant`, `persistent` and `scratch` override the coarse keys.

### ExecuTorch buffers

Only the planned arena is touched by every kernel on every inference, so
it is the one buffer that rewards DTCM; the method arena (cold after
load), the temporary arena (kernel scratch) and the I/O buffers (one copy
per inference) belong in SRAM. Rough vocabulary: heliaAOT `scratch` is
roughly the planned plus temporary arenas, `persistent` roughly the method
arena, `constant` the PTE program placed by `weights_location`. Per-tensor
placement inside the planned arena is not available: the export-time
planner bakes each activation's offset into the PTE, so the firmware's
finest unit is that arena. ExecuTorch's planner supports several planned
buffers, so a future export option could split activations across two
arenas; tracked in
[nsx-executorch#3](https://github.com/AmbiqAI/nsx-executorch/issues/3).

## Expected output

A run whose report shows every region under capacity, an `allocated_arena`
that matches the request with a little headroom, and `run_metadata.json`
recording the resolved placement so a comparison can prove both runs used
the same one.

## What next

[Engines](https://ambiqai.github.io/helia-profiler/guide/engines/) for the engine whose placement model you are
using; the [Python API](https://ambiqai.github.io/helia-profiler/reference/api/) for the placement types when
you script runs.