runner

Run exported LiteRT model bytes with ai-edge-litert.

LiteRTRunnerclassRun a single-input, single-output .tflite model one sample at a time.
LiteRTStreamRunnerclassRun a streaming .tflite model one step at a time, carrying its state as raw tensors.

LiteRTRunner(content: bytes, *, reference_kernels: bool = False, num_threads: int = 1) -> None

Run a single-input, single-output .tflite model one sample at a time.

The input tensor is resized to each sample’s shape along the model’s dynamic dimensions only; fixed dimensions must match. Native float16 graphs run when the runtime has float16 kernels for every operator; otherwise the runtime’s error is raised unchanged.

Parameters

Parameters of LiteRTRunner
Name	Type	Default	Description
`content`	`bytes`	Required	Model flatbuffer bytes.
`reference_kernels`	`bool`	False	Use LiteRT's reference kernels instead of its optimized kernels.
`num_threads`	`int`	1	Interpreter threads.

interpreter = litert.Interpreter(model_content=content, num_threads=num_threads, experimental_op_resolver_type=resolver)

run(x: npt.NDArray) -> npt.NDArray

Invoke on samples along axis 0 already in the model’s input dtype; return raw outputs.

encode(x: npt.NDArray) -> npt.NDArray

Convert real-valued samples to the input dtype, rounding and saturating integer inputs.

decode(y: npt.NDArray) -> npt.NDArray

Convert raw outputs to float32 values.

predict(x: npt.NDArray) -> npt.NDArray

Encode real-valued samples, run them and decode the outputs.

LiteRTStreamRunner(content: bytes, *, reference_kernels: bool = False, num_threads: int = 1) -> None

Run a streaming .tflite model one step at a time, carrying its state as raw tensors.

Inputs and outputs are addressed by signature name. State inputs state_in_k start at zero, and at each later step take the raw state_out_k of the previous step unchanged, so an integer state keeps its value only if the pair has one scale and zero point (export_model ties them).

Parameters

Parameters of LiteRTStreamRunner
Name	Type	Default	Description
`content`	`bytes`	Required	Model flatbuffer bytes with one signature.
`reference_kernels`	`bool`	False	Use LiteRT's reference kernels instead of its optimized kernels.
`num_threads`	`int`	1	Interpreter threads.

Raises

Errors raised by LiteRTStreamRunner
Type	Description
`ValueError`	If the model does not have exactly one signature, or its state inputs and outputs do not pair up.

interpreter = litert.Interpreter(model_content=content, num_threads=num_threads, experimental_op_resolver_type=resolver)

inputs: dict[str, dict] = dict(runner.get_input_details())

outputs: dict[str, dict] = dict(runner.get_output_details())

pairs: tuple[int, ...] = tuple(ins)

signals: tuple[str, ...] = tuple()

initial_state() -> dict[str, npt.NDArray]

Each state input at zero: the zero point for an integer state.

step(inputs: Mapping[str, npt.NDArray]) -> dict[str, npt.NDArray]

Invoke once on raw tensors for every input name; return every output by name.

run(
    signals: Mapping[str, npt.NDArray],
    resets: Collection[int] = (),
) -> tuple[dict[str, npt.NDArray], dict[str, npt.NDArray]]

Stream raw signal inputs, carrying the state.

Parameters

Parameters of run
Name	Type	Default	Description
`signals`	`Mapping[str, npt.NDArray]`	Required	Every input that is not a state, by name, as raw tensors stacked along a leading step axis (``[steps, *input shape]``).
`resets`	`Collection[int]`	()	Steps at which the state inputs return to ``initial_state``.

Returns

Returns of run
Value	Type	Description
`tuple`	`tuple[dict[str, npt.NDArray], dict[str, npt.NDArray]]`	Every input as fed and every output as produced, by name, stacked along the step axis.

encode(name: str, x: npt.NDArray) -> npt.NDArray

Convert real values to input name’s dtype, rounding and saturating integer inputs.

decode(name: str, y: npt.NDArray) -> npt.NDArray

Convert raw values of output name to float32.