Skip to content

Operator Attributes

Operator attributes let you steer how each operator is implemented and where its code/scratch lives in memory. They are provided via CLI directly or using YAML and applied during code generation.

Operator IDs match the original model (stringified). See "How do I determine entity IDs?" in the docs for tips.

What you can control (base attributes)

For now we expose two base attributes that apply to all operator types:

  • code_placement — where the operator’s wrapper/dispatch code is linked. Typical values: ITCM, MRAM (platform maps these to sections).
  • scratch_placement — where the operator’s temporary working memory (op-local scratch) is allocated from. Typical values: DTCM, SRAM, PSRAM.

Note: Operator attributes do not move global tensors (weights/activations). Use Tensor Attributes for that. scratch_placement only affects ephemeral scratch tensors the operator itself requests.

Quick reference

MemoryType Good for Notes
ITCM Hot code paths Fastest instruction access; limited capacity.
DTCM Hot scratch Fastest data access; limited capacity.
SRAM General code/scratch Balanced default.
PSRAM Large scratch Off-chip/volatile; good for big temporaries.
MRAM Non-volatile code/data great for size, not always the fastest.

Rule structure & precedence

Operator attributes use the same AttributeRuleset pattern as tensors:

operators:
    - type: "*"                    # Operator type, e.g. CONV_2D, ADD, or "*" for all
    id: null                     # Operator id or list of ids, or null for all
    attributes:
        code_placement: MRAM
        scratch_placement: SRAM

    - type: CONV_2D
    attributes:
        code_placement: ITCM       # Put CONV_2D wrappers in ITCM

    - id: ["7", "15"]              # Specific operator instances (by ID)
    attributes:
        scratch_placement: DTCM    # Give these ops faster scratch

Matching & precedence (from lowest to highest):

  1. (type="*", id=None) — catch-all
  2. (type=OP_TYPE, id=None)
  3. (type="*", id=ID or [IDs])
  4. (type=OP_TYPE, id=ID or [IDs]) — most specific

If multiple rules have the same specificity, the later rule wins.

Examples

1) Global defaults with type-level overrides

operators:
    # Default for everything
    - attributes:
        code_placement: MRAM
        scratch_placement: SRAM

    # All depthwise convs run from ITCM (hot)
    - type: DEPTHWISE_CONV_2D
    attributes:
        code_placement: ITCM

2) Target a single hot operator by ID

operators:
    - id: "23"
    attributes:
        code_placement: ITCM
        scratch_placement: DTCM

3) Mix of broad + specific

operators:
    # Broad tuning
    - type: CONV_2D
    attributes: { code_placement: ITCM }

    # Exception for op 9 (maybe too big for ITCM)
    - type: CONV_2D
    id: "9"
    attributes: { code_placement: MRAM }

How operator attributes are applied

  • We gather all rules that match an operator’s (type, id).
  • We sort by specificity (see precedence above) and config order.
  • We merge attribute dictionaries in that order (later wins on ties).
  • The resolved code_placement and scratch_placement feed into:
    • Linker placement for operator code and
    • The allocator used for op-local scratch.

If a memory type isn’t supported on your target, you’ll get a warning or build error depending on configuration.

Interplay with Tensor Attributes

  • Operator attributes affect op wrappers/dispatch and op-local scratch.
  • Tensor attributes affect model tensors (constants, persistents, and (optionally) scratch tensors).
  • Use both attributes together:
    • Place global constants (weights) with Tensor Attributes (e.g., MRAM).
    • Keep hot op wrappers in ITCM with Operator Attributes.
    • Keep op scratch in DTCM when it’s performance-critical.

Operator-specific attributes (coming soon)

Some operators expose extra tuning knobs (e.g., algorithm selection, tiling hints, kernel allow/deny lists). This section will document those per-operator attributes once stabilized.

  • CONV_2D attributes — planned
  • DEPTHWISE_CONV_2D attributes — planned
  • FULLY_CONNECTED attributes — planned
  • POOLING/ACTIVATION attributes — planned

Until then, any extra keys you include in operator rules will be carried through (we allow extras) and consumed once the corresponding operator gains a schema.

Best practices

  • Start with a global default (e.g., code_placement: MRAM, scratch_placement: SRAM) and selectively promote hot paths to ITCM/DTCM.
  • Measure: ITCM/DTCM are scarce—reserve them for bottleneck ops.
  • If moving operators to more performant memory, you might also need to move I/O tensors to faster memory as well.

Troubleshooting

  • "My op still runs from SRAM." → Double-check code_placement, the operator type and id, and the platform’s linker mapping.
  • "I run out of ITCM/DTCM." → Relax rules, move less critical ops back to SRAM/MRAM/PSRAM.
  • "Placing code in MRAM slowed things down." → Try ITCM for the few hottest wrappers; keep most in MRAM/SRAM.