Compare toolchains

Goal

See how much of a cycle count is the compiler’s doing. You will learn how to vary only target.toolchain, what hpx compare marks as different, and how to line up per-layer rows to find where a toolchain gains.

Hardware and wiring

An Apollo510 EVB on its J-Link USB port; no power instrument.

Software prerequisites

Two or three toolchains installed and on PATH: the GNU Arm toolchain (gcc), the Arm Toolchain for Embedded (atfe) and, if licensed, Arm Compiler 6 (armclang). Toolchains has the install notes and the aliases; hpx doctor reports which ones it can see.

Config file

No new file: pass --toolchain on the command line over one of the checked-in configs, and give each run its own output directory. The armclang variants exist as files for the two engines: examples/quickstart/hpx_rt_armclang.yml and examples/quickstart/hpx_aot_armclang.yml.

Run

Run from the repository root, passing the model path explicitly:

hpx profile examples/quickstart/kws_model.tflite --config examples/quickstart/hpx_rt.yml --toolchain gcc  --output-dir results/rt_gcc
hpx profile examples/quickstart/kws_model.tflite --config examples/quickstart/hpx_rt.yml --toolchain atfe --output-dir results/rt_atfe
hpx compare results/rt_gcc results/rt_atfe --output-dir results/rt_gcc-vs-atfe

Add --toolchain armclang for a third run when Arm Compiler 6 is installed. Keep --engine, the board, the counters and the iteration count identical across runs.

Expected output

The real hpx compare output for the two heliaRT bundles under Validated on (paths shortened; the rest of the Config table and the Run table are elided, marked with …):

  Total cycles     2,060,239 cyc → 1,947,189 cyc  -113,050 cyc (-5.5%)
  Avg inference    21,717 us → 20,526 us  -1,191 us (-5.5%)

╭─ Warnings ──────────────────────────────────────────────╮
│    - Comparison dimension differs: toolchain.           │
│    - Comparison dimension differs: compiler_version.    │
│    - Comparison dimension differs: architecture_flags.  │
╰─────────────────────────────────────────────────────────╯

 Field              Baseline                 Candidate                  Status
 Engine             helia-rt                 helia-rt                    same
 …
 Toolchain          arm-none-eabi-gcc        atfe                        diff
 Link family        gnu                      gnu                         same
 Compiler version   arm-none-eabi-...15.2…   clang version 22.1.0        diff
 …

Clean-window cycles per inference from the four bundles:

Engine	GNU Arm 15.2.1	ATfE (clang 22.1.0)	Change
heliaRT	2,051,232	1,938,240	-5.5%
heliaAOT	2,044,512	1,931,904	-5.5%

How to read it

• The toolchain dimension is informative, not blocking: hpx compare shows every delta and tells you the compiler changed. Per-layer deltas stay because the operation sequence is identical.
• Both engines gain 5.5% from ATfE in these captured runs. The shared kernels are a useful place to investigate, but these totals alone do not isolate the source of the improvement.
• To see where the gain lands, sort layer_diff.csv in the compare output directory, or line the two per-layer files up:

cycles() { awk -F, 'NR==1{for(i=1;i<=NF;i++)if($i=="cycles")c=i}{print $1","$2","$c}' "$1"; }
paste -d, <(cycles results/rt_gcc/profile_results.csv) <(cycles results/rt_atfe/profile_results.csv | cut -d, -f3) | head

• The column position of cycles depends on how many counters the run captured, which is why the helper above finds it by name.
• link_family stays gnu for ATfE, so the per-region memory rows remain comparable; armclang links with armlink, and the comparison omits the per-region used and free rows for that pair while keeping the section sizes (Output files).

Variations

• armclang: run the two armclang configs above, or --toolchain armclang; the link_family note applies.
• Vary the engine as well and you have the six-run grid; compare within a toolchain to isolate the engine, within an engine to isolate the toolchain (Compare engines).
• --cpu-clock hp on Apollo510 changes microseconds and, through memory timing, cycles; keep the tier fixed across a toolchain comparison (Boards and targets).

Validated on

Board	apollo510_evb, lp clock tier
Run	nightly hardware validation run 35078330526, 2026-09-16, hpx 0.1.6 at 0714893
Cases	apollo510_evb-kws-rt-ns-arm-none-eabi-gcc-rtt-auto, apollo510_evb-kws-rt-ns-atfe-rtt-auto, apollo510_evb-kws-aot-ns-arm-none-eabi-gcc-rtt-auto, apollo510_evb-kws-aot-ns-atfe-rtt-auto, all valid
nsx modules	as recorded in each bundle’s nsx.lock (baseline hpx-neuralspotx-0.8.1-2026-09, nsx-helia-rt helia-rt-v1.20.0, nsx-cmsis-nn aaeb145)
Toolchains	arm-none-eabi-gcc 15.2.1; ATfE clang 22.1.0
armclang	not yet validated: no armclang case runs in the nightly suite
Differences from the configs above	the nightly cases use arena_size: 32768, pmu_counters: {cpu: default}, iterations: 3, warmup: 1
Bundles	examples/results/hardware-validation-2026-09-16/