# Data capture

Explain the path from a firmware `printf` to a typed result, and the reset
and readiness steps that run before it, so a capture failure can be placed
at the lifecycle, the transport, the stream or the parser.

## Diagram

The capture subsystem: What runs inside CapturePmuStage and CapturePowerStage. The lifecycle and transport blocks are shared by both phases; the parser is profile-only and the power block is skipped unless power.enabled is true.

- Target lifecycle
  - ensure powered
  - reset strategy
  - readiness poll
- Transport: one per run
  - rtt: default
  - usb_cdc
  - uart
  - swo
- Wire stream
  - sentinels
  - metadata lines
  - heartbeats
  - CSV rows
- Parser
  - extract metadata
  - group presets
  - aggregate iterations
  - merge presets
- Power capture: when enabled
  - driver: joulescope, ina228
  - sync controller
  - gated window
  - terminal record
- Results
  - PmuResult
  - PowerResult
Text description of the diagram

- Target lifecycle: ensure powered, reset strategy, readiness poll
- Transport (one per run): rtt (default), usb_cdc, uart, swo
- Wire stream: sentinels, metadata lines, heartbeats, CSV rows
- Parser: extract metadata, group presets, aggregate iterations, merge presets
- Power capture (when enabled): driver (joulescope, ina228), sync controller, gated window, terminal record
- Results: `PmuResult`, `PowerResult`

## The blocks

### Target lifecycle

Before either capture phase, `prepare_target_for_phase()` in
`target/lifecycle.py` applies a `TargetLifecyclePlan`: it may power-cycle
the board through the power driver, then applies a `ResetStrategy` through
the `ResetController` the probe stage installed, and returns the actions
it took so the run metadata records them. `EnsureBoardPoweredStage` ran
earlier so the rail is up before the probe is touched. After a reset the
transports poll for readiness with a deadline (`open_jlink_with_retry()` in
`target/probe/jlink.py` for the J-Link attach) instead of sleeping blindly.

### Transport backends

`transport/__init__.py` holds a registry keyed by the `Transport` enum;
`resolve_transport()` returns the backend and `register_transport()` lets
tests swap one in. Every backend implements the `CaptureTransport` protocol
in `transport/base.py`: `prepare(ctx, args)`, then `collect(ctx)`
returning the captured lines. The shared line-collection
loop is `transport/protocol.py`: it reads until the end sentinel or a
timeout, and any `HPX_HEARTBEAT` line resets the inactivity deadline
(`HEARTBEAT_TIMEOUT_S`), with the `clean_window_begin` heartbeat extending
it to cover the announced window. Which transport to choose, and what each
needs on the bench, is on [Capture transports](https://ambiqai.github.io/helia-profiler/guide/transports/).

### The wire stream

The firmware prints one line-oriented text stream, declared once in
`src/helia_profiler/wire/` for both the templates and the host. The
elements a reader meets:

| Line form | Meaning |
|---|---|
| `--- HPX_START ---` | start of profiling output; everything before it is ignored |
| `HPX_<KEY>=<value>` | metadata (`HPX_VERSION`, `HPX_ENGINE`, `HPX_MODEL_SIZE`, `HPX_ARENA_SIZE`, `HPX_ALLOCATED_ARENA`, `HPX_NUM_PRESETS`, `HPX_PRESETS`, the clean-window fields) |
| `HPX_HEARTBEAT ...` | progress marker; resets the inactivity timeout, never parsed as data |
| `--- HPX_PRESET <name> ---` | start of one counter pass |
| `--- HPX_ITER <n> ---` | start of iteration n; the next line is the CSV header |
| `"Layer","Op",<counters>,"overflow"` | header naming this pass's counter columns |
| `<idx>,<op>,<c1>,...,<overflow>` | one layer row |
| `--- HPX_END ---` | all profiling complete |

The full vocabulary, including the fault and warning catalogues and the
power binary's terminal record, is the [wire protocol](https://ambiqai.github.io/helia-profiler/reference/wire-protocol/)
reference, marked internal because it is a contract between hpx's own
firmware and host, not a user surface.

### The parser

`parse_firmware_output(lines, aggregation)` in `capture/parser.py` runs
four steps: metadata lines become a `FirmwareMeta`; lines are grouped by
preset marker; each iteration's CSV rows are parsed and, per layer,
reduced across iterations by `profiling.aggregation` (`median` by default,
`mean` or `trimmed`) after structurally invalid samples (uint32 wrap,
all-zero rows in a pass containing `ARM_PMU_CPU_CYCLES`) are rejected; and the per-preset `PresetResult`s are
merged into one `PmuResult` with every counter column on each
`LayerResult`, plus per-compute-unit groups keyed by the `<group>_<index>`
pass naming. Layers are matched by explicit id with the same operator name
required across iterations and presets; a missing, duplicate or
conflicting identity raises `CaptureError` rather than publishing a
position-shifted aggregate. A missing end sentinel, missing or repeated announced pass, or missing iteration raises `CaptureError`. Truncated captures fail the run instead of publishing partial aggregates.

Multiple presets exist because a SoC exposes a fixed number of PMU
counters; `platform/counters.py` plans the passes from the requested
groups and the SoC's capability record, and the firmware runs them in
sequence in one boot.

### Power capture

When `power.enabled` is true, `capture_power()` in `capture/__init__.py`
resolves the `PowerDriver` (`power/joulescope/` or `power/ina228_driver.py`),
builds the sync controller that owns the READY, GO and GATE handshake
(`power/sync.py`, `power/joulescope/sync.py`),
runs the lifecycle plan, and calls the driver's `capture_gated()` for the
GPIO-gated window or `capture()` for a free-form one. The result is a
`PowerResult` with the accepted window, the energy integration and the
gating diagnostics; `CollectPowerTerminalStage` then reads the one status
record the power firmware prints after the gate closes
(`capture/power_terminal.py`). The measurement scopes and their health
signals are on [Power measurement](https://ambiqai.github.io/helia-profiler/guide/power/) and
[Verifying a capture](https://ambiqai.github.io/helia-profiler/guide/power-verify/).

### Failure handling

| Scenario | Behaviour |
|---|---|
| no output at all | the overall timeout expires and `CaptureError` names the transport |
| firmware stalls mid-run | no line for `HEARTBEAT_TIMEOUT_S` raises `CaptureError` with the last line seen |
| silent clean window | the `clean_window_begin` heartbeat carries the estimated duration and extends the deadline |
| firmware reports a fault | the `HPX_ERROR` line maps to a `CaptureError` carrying the catalogue's hint |
| identity mismatch across iterations | `CaptureError`; nothing partial is published |

## Where the code lives

| Block | Path under `src/helia_profiler/` |
|---|---|
| target lifecycle | `target/lifecycle.py`, `target/probe/`, `stages/ensure_powered.py` |
| transports | `transport/{rtt,usb_cdc,uart,swo}.py`, `transport/base.py`, `transport/protocol.py`, `transport/timing.py` |
| wire stream | `wire/` |
| parser | `capture/parser.py`, `results/models.py` |
| power capture | `capture/__init__.py` (`capture_power`), `power/`, `capture/power_terminal.py` |
| stages | `stages/capture_pmu.py`, `stages/capture_power.py`, `stages/collect_power_terminal.py` |

## What next

[Firmware generation](https://ambiqai.github.io/helia-profiler/guide/concepts/firmware/) for the side of the stream the
templates print; [Pipeline](https://ambiqai.github.io/helia-profiler/guide/concepts/pipeline/) for where the capture stages sit;
[PMU counters](https://ambiqai.github.io/helia-profiler/guide/pmu-counters/) for what the columns mean.

Capture backends respect `target.heartbeat.overall_timeout_s` and `target.heartbeat.host_timeout_s`. SWO retries share one overall deadline. Only complete lines refresh progress deadlines. Passes without a cycle counter retain all-zero rows.