ExecuTorch NS Tier-1 Kernels#

ExecuTorch PTEs exported for Cortex-M can be lowered against two CMSIS-NN providers. The arm provider uses upstream arm-cmsis-nn and leaves ops outside the stock cortex_m:: set as portable ATen fallbacks — float kernels wrapped in a dequantize/requantize island. The ns provider adds Ambiq's out-of-tree cortex_m_ns:: Tier-1 operators (sub, hardswish, mean, relu/relu6/hardtanh/clamp, leaky_relu) backed by ns-cmsis-nn, so those ops stay int8 end to end.

This page is the measured evidence for that difference. See Inference Engines for how to turn the NS kernels on.

Models#

Each pair is two deterministic random-weight int8 exports of the same PyTorch module, produced by tools/export/executorch_tier1/export_tier1.py through nsx_cortex_m.export(kernel_provider=...):

tier1 — channels_last conv trunk (16×32×32 activations): conv → hardswish → conv → leaky_relu → sub → relu.

• arm PTE: aten::leaky_relu.out, aten::sub.out, and aten::clamp.out (the relu) run as float portable kernels inside a dq…q island; hardswish uses the stock cortex_m::minimum + cortex_m::quantized_mul decomposition.
• ns PTE: cortex_m_ns::quantized_hardswish/leaky_relu/sub/relu, zero portable ops.

tier1mean — contiguous-layout micro model: mean(dim=(2,3), keepdim).

• arm PTE: float portable aten::mean.out; ns PTE: cortex_m_ns::quantized_mean.
• mean is kept out of the conv trunk deliberately — see Caveats.

Configs live in configs/executorch/tier1_{arm,ns}.yaml and configs/executorch/tier1mean_{arm,ns}.yaml; the ns runs set engine.config.ns_ops: true. The tier1_{arm,ns}_atfe.yaml variants build the same PTEs with ATfE (requires ATFE_ROOT), so a gcc-vs-ATfE toolchain comparison is one pair of runs plus hpx compare results/tier1_arm results/tier1_arm_atfe.

Verifying the right kernels ran#

Three independent checks, because "the build succeeded" does not prove which kernels linked:

1. Link-time symbols — tools/verify_executorch_kernels.py runs arm-none-eabi-nm over the built firmware. arm builds show the portable registration table and named torch::executor::native:: kernels, and no arm_convolve_weight_sum or cortex_m_ns symbols. ns builds show the ns-cmsis-nn v7.29.2 weight-sum ABI (arm_convolve_weight_sum, arm_convolve_s8_get_weights_sum_size) plus the cortex_m_ns::native::quantized_*_out kernels.
2. Load-time fail-fast — a PTE containing cortex_m_ns:: ops fails Method::load() with "operator missing" on any build without NSX_EXECUTORCH_ENABLE_NS_OPS=ON. Every ns run that loads at all is itself proof of registration. The same holds for the selective portable list: the arm runs only load because NSX_EXECUTORCH_PORTABLE_SELECT_OPS_LIST registered exactly the PTE's ATen fallbacks.
3. Per-instruction PMU signature — the captured instruction streams match the operator counts recorded in examples/models/tier1/tier1_manifest.json exactly (12 arm / 8 ns instructions for tier1; 5 / 3 for tier1mean).

Results — tier1 (conv trunk)#

Convolutions are effectively identical across providers (<0.5%), which confirms the delta comes from the Tier-1 ops rather than the conv path.

Results — tier1mean (mean micro model)#

The ns kernel is MVE-vectorized (arm_mean_reduce_spatial_mve_s8) and works on int8 in place. The portable path dequantizes 16K values to float, reduces scalar-by-scalar, and requantizes.

Reproducing#

$NSX_EXECUTORCH is the qualified nsx-executorch checkout.

# 1. Export both PTEs (deterministic, seed 20260819)
PYTHONPATH="$NSX_EXECUTORCH/external/executorch/src:$NSX_EXECUTORCH/aot" \
  python tools/export/executorch_tier1/export_tier1.py \
    --output-dir examples/models/tier1

# 2. Profile each side
hpx profile --config configs/executorch/tier1_arm.yaml
hpx profile --config configs/executorch/tier1_ns.yaml

# 3. Verify the linked kernels match the declared provider
python tools/verify_executorch_kernels.py --provider arm --expect-portable <arm ELF>
python tools/verify_executorch_kernels.py --provider ns  --ns-ops         <ns ELF>

# 4. Per-instruction cross-PTE comparison
python tools/export/executorch_tier1/compare_tier1.py \
  --manifest examples/models/tier1/tier1_manifest.json \
  --model tier1 --baseline results/tier1_arm --candidate results/tier1_ns

Add --jlink-serial <serial> to step 2 if more than one probe is connected.

Caveats#

• mean and channels_last. The ns qualifier rejects channels_last input for mean, and the portable aten::mean.out kernel fails Method::execute() with InvalidArgument (error 18) on channels_last data. A channels_last graph that keeps mean portable is therefore unrunnable on both providers. Keep mean on contiguous tensors, or lower it to cortex_m_ns::quantized_mean.
• Cross-PTE comparison. hpx compare requires the same model SHA, so it will not compare an arm run against an ns run. Use tools/export/executorch_tier1/compare_tier1.py, which matches operators through the export manifest instead.
• Portable-op reporting. nsx-executorch's ExportResult.portable_fallback_ops under-reports — it misses the aten::relu.out / aten::leaky_relu.out edge spellings. Derive the portable op list from the serialized PTE operator list instead.