# ETHOS_U_STREAM_SPLIT contract

Owner of the rewrite and this contract: helia-npu (`helia_npu.fencing`).
Consumers: helx's local FVP runner (kernel in
`helia_estimator/local_fvp_runtime/src/stream_split.cpp`), heliaRT (in-tree
registration factory when fenced models become a supported format; app-side
`AddCustom` until then), heliaAOT 1.3.0 and later. All three engines support HELX fenced per-segment profiling.
Contract version: **1** (custom-op version 1). Any change below bumps it.

The custom-op identifier replaces the earlier `STREAM_SPLIT` name. Regenerate
fenced models from their original unfenced input before using this runner;
the old identifier is not registered as an alias. Version 1 describes the
unchanged identity/copy semantics of the new identifier. Python constant names
and `/stream_split` tensor-name suffixes remain unchanged.

## Purpose

Vela ends the current Ethos-U command stream at a custom op it cannot fuse.
`ETHOS_U_STREAM_SPLIT` is an identity op inserted between two original operators so
that the graph is compiled as N NPU command streams with a CPU handoff at
each fence. It exists to make a boundary, not to compute.

## Wire format (TFLite flatbuffer)

| Item | Value |
|---|---|
| `OperatorCode.customCode` | `"ETHOS_U_STREAM_SPLIT"` (exact, case-sensitive; consumers may canonicalize case on lookup) |
| `OperatorCode.builtinCode` / `deprecatedBuiltinCode` | `CUSTOM` (32) |
| `OperatorCode.version` | 1 |
| inputs | N ≥ 1: all tensors produced at/before `after_op` and consumed later or returned by the graph |
| outputs | N outputs, paired by index with the inputs; first output is the sidecar join key; additional outputs have cut-specific names |
| `builtinOptionsType` | NONE (0); `builtinOptions` absent |
| `customOptions` | absent (no flexbuffer) |
| output tensor | same `shape`, `type`, and a deep copy of `quantization` (scale, zero point, per-axis fields) as the input; `isVariable = false`; new empty buffer |
| rewiring | every later operator that consumed the intercepted tensor now consumes the new tensor; if the intercepted tensor was a graph output, the graph output is rewired to the new tensor |
| placement | inserted immediately after operator `after_op`; fences are applied in descending order so original indices stay valid during the rewrite |
| existing operator code | a `ETHOS_U_STREAM_SPLIT` entry already in the model is reused only if its effective builtin code is `CUSTOM` (the larger of `builtin_code` and `deprecated_builtin_code`, as TFLite's `GetBuiltinCode` reads the v3 and v3a schemas, so `32/32`, `32/0` and `0/32` qualify, while a non-CUSTOM `builtin_code` with a CUSTOM deprecated field does not, since Vela reads `builtin_code` first) and its version is 1; a reused entry is passed through as written, not rewritten to `32/32`; any other same-name entry is rejected before output is written, never reused or duplicated |
| subgraph | 0 only; a model with more than one subgraph is rejected by `_primary_subgraph` before anything is written |

Supported dense storage types: **int8, uint8, int16, int32, int64, float16,
float32 and bool**. Pairs may use different types and shapes; each output must
match its corresponding input. Byte counts come from tensor metadata, not a
fixed element size. Packed INT4, sparse, string/resource and variable tensors
are outside this contract. FP16 copying requires no floating-point arithmetic.

The rewrite carries residual paths and all live outputs, not just the selected
operator's first output. Constants and graph inputs are already available and
are not copied. This establishes data dependencies across the cut; it does not
impose a global order on unrelated branches with no crossing data dependencies.
This is the initial contract for the renamed operator, including multi-tensor support.

## Boundary semantics

- **Identity of contents per pair**: output bytes equal input bytes; dtype, shape,
  quantization unchanged. `byte length = flat size × element size` and must
  match on both sides; a kernel must fail `Prepare` on any count, type, shape
  or byte-length mismatch rather than truncate.
- **Copy, not alias, by default**: the output is a distinct buffer with a
  distinct lifetime. Exact alias (`out == in`) is permitted and is a no-op.
  Outputs overlapping another pair's inputs or outputs, and partial overlap
  within a pair, are rejected before copying so no destination clobbers data
  still needed by another pair.
- **Execution boundary**: the op must remain a scheduling point between the
  NPU region that produces the input and the NPU region that consumes the
  output. A compiler must not eliminate it as a reshape or no-op even when
  aliasing would be legal.
- **What it does not promise**: no cache maintenance, no coherency, no
  resumable or cooperative scheduling. Those belong to the runtime's Ethos-U
  driver handoff and are qualified on the target, not by this op.

## Identity carried with a fenced model

`RewriteResult` and the `<stem>.fenced_boundaries.json` sidecar record:
`source_model`, `source_sha256` (of the model the fences were cut into),
`subgraph`, `num_segments`, `boundary_after_ops` (original operator indices,
authoritative), and per segment `op_range_start`, `op_range_end`,
`tensor_pairs` (actual input/output names in the final fenced graph),
`end_tensor_name` (the first intercepted tensor in the original source model) and `fence_tensor_name`
(`<name>/stream_split`). `plan_fences` attaches the same `output_tensor_name`,
`opcode`, `segment_index` to each `Fence` and records `model_sha256` when
given the model. A consumer joins across recompilation by
`(source_sha256, after_op)` first and tensor names second; compiled and
runtime operator indices are different coordinates and are not carried here.

A plan is bound to the model it was cut for. `fence_model` (via
`check_plan_binding`) rejects, before writing, a `FencePlan` whose
`model_sha256` (when the planner recorded one) or operator count does not
match the model it is applied to or whose `subgraph` is not 0, and any `Fence` carrying identity
whose `output_tensor_name` or `opcode` is not what sits at `after_op` in that
model, or whose `after_op` follows the last operator (`FenceBindingError`). A
plan cut without the model has no hash and is bound by count and names only.
The unbound path is positions alone: `plan.after_ops`, a list of ints, or a
`FenceConfig` (`fence_config_for` is the deliberate reduction of a plan to
positions) given to `rewrite_model_with_fences`; that is the caller asserting
the coordinates apply.

## Cost planning caveat

`plan_fences` budgets on Vela's estimate attributed to original operators.
`FenceCoverage` reports the denominator (sum of `op_cycles` over all Vela
rows), the matched part, and the unmatched rows. A cycle-based policy raises
when coverage is below `min_coverage` (default 100%) or when there are no rows
at all; lowering the threshold records uncertainty, it does not create a
latency bound. An operator larger than the budget is isolated, not split.