Tensor Attributes

This page explains how to steer where tensors live in memory in the generated C inference module. Tensor attributes are provided via CLI directly or using YAML and applied during code generation.

Tensor IDs match the original model (stringified). See "How do I determine entity IDs?" in the docs for tips.

---

What you can control?

Right now tensor attributes focus on memory placement:

• memory: Choose a valid target memory for a tensor (e.g., DTCM, SRAM, PSRAM, MRAM).

Typical uses:

• Pin big constant weights to MRAM (or external flash/NVM) to save on SRAM/TCM.
• Keep scratch working sets in DTCM for speed.
• Place persistent model state in SRAM or PSRAM depending on size/speed needs.
• Some operators (e.g. FULLY_CONNECTED) are highly memory bound and benefit from having tensors in faster memory (SRAM/TCM).

---

Memory types

MemoryType	Typical use	Notes
ITCM	Instructions (code)	Not commonly used for tensor data.
DTCM	Hot scratch/activation buffers	Fast, tightly-coupled; limited capacity.
SRAM	General-purpose data	Good default for persistent data if it fits.
PSRAM	Large, slower, off-chip working sets	Volatile; great for big tensors that don’t fit on-chip.
MRAM	Non-volatile constants (e.g., weights)	Ideal for immutable tensors as it's read-only.

The backend/platform decides how a MemoryType maps to linker sections and where data actually ends up.

---

Rule structure & precedence

Tensor attributes are expressed with a generic “attribute ruleset”:

tensors:
    - type: "*"                         # Tensor kind: SCRATCH | PERSISTENT | CONSTANT | "*" for all
    id: null                          # Tensor id or list of ids, or null for all
    attributes:
        memory: DTCM                    # MemoryType

    - type: PERSISTENT
    attributes:
        memory: SRAM

    - id: ["conv2d_3_weight", "conv2d_3_bias"]
    attributes:
        memory: MRAM

    - type: SCRATCH
    id: "large_scratch_tensor"
    attributes:
        memory: PSRAM

Matching & precedence (from lowest to highest):

1. (type="*", id=None) — catch-all
2. (type=KIND, id=None)
3. (type="*", id=ID or [IDs])
4. (type=KIND, id=ID or [IDs]) — most specific

KIND must match the tensor’s kind in the model (e.g., SCRATCH, PERSISTENT, CONSTANT). Use "*" to match all.

---

Examples

1) Global defaults, with targeted overrides

```yaml
tensors:
    # Default: everything to DTCM
    - attributes: { memory: DTCM }

    # Persistent tensors (variables) to SRAM
    - type: PERSISTENT
    attributes: { memory: SRAM }

    # Specific large weights to MRAM
    - id: ["fc_5_weight", "fc_5_bias"]
    attributes: { memory: MRAM }
```

2) Push all scratch to PSRAM, except two hot buffers

```yaml
tensors:
   - type: SCRATCH
   attributes: { memory: PSRAM }

   - type: SCRATCH
   id: ["hot_buf_0", "hot_buf_1"]
   attributes: { memory: DTCM }
```

3) Only move constants to MRAM, leave others on defaults

```yaml
tensors:
   - type: CONSTANT
     attributes: { memory: MRAM }
```

---

How attributes are applied

• During codegen, we compute the effective attributes per tensor by:

  1. collecting all matching rules,
  2. sorting by specificity and config order,
  3. merging in order (later wins at the same specificity).

• The resulting memory selection informs linker placement / section mapping.

---

Finding tensor IDs

• IDs generally match the original model (stringified). Use Netron or our Model Explorer to view them.
• The generated offline documentation includes tables of tensors and a model diagram.

---

Best practices

• Start with a global default (e.g., scratch → DTCM, constants → MRAM), then override problematic tensors.
• Use PSRAM (if available) for large temporaries that don’t fit on-chip; measure the impact.
• Keep hot activations in DTCM if they’re re-used within a layer block.
• Prefer MRAM for immutable weights to minimize SRAM/TCM pressure.

---

Troubleshooting

• "My rule didn’t apply." → Check the tensor’s kind and ID spelled exactly as in the model; remember precedence rules.
• "Build succeeded but memory usage looks unchanged." → Confirm your platform maps the selected MemoryType to a real section, and that the tensor is large enough to notice the shift.
• "Runtime stalls after moving tensors." → Ensure SCRATCH and PERSISTENT tensors are not placed in read-only memory like MRAM.