arm_nnsupportfunctions.h

Machine-readable model

• USE_FAST_DW_CONV_S16_FUNCTIONmacro
• LEFT_SHIFTmacro
• RIGHT_SHIFTmacro
• MASK_IF_ZEROmacro
• MASK_IF_NON_ZEROmacro
• SELECT_USING_MASKmacro
• ARM_NN_MAXmacro
• ARM_NN_MINmacro
• ARM_NN_CLAMPmacro
• arm_nn_min_f16hfunctionMinimum of two scalar f16 values.
• arm_nn_max_f16hfunctionMaximum of two scalar f16 values.
• arm_nn_propagate_nan_f16hfunctionReturns x when x is NaN, otherwise y.
• arm_nn_clamp_f16hfunctionDrop-in equivalent of ARMNNCLAMP(x, h, l) for scalar Float16 operands.
• arm_nn_clamp_propagate_nan_f16hfunctionScalar f16 clamp with TFLite NaN semantics: NaN passes through unchanged.
• arm_nn_abs_f16hfunctionAbsolute value of a scalar f16 value.
• ARM_NN_ROUND_UPmacro
• REDUCE_MULTIPLIERmacro
• CH_IN_BLOCK_MVEmacro
• S4_CH_IN_BLOCK_MVEmacro
• MAX_COL_COUNTmacro
• REVERSE_TCOL_EFFICIENT_THRESHOLDmacro
• CONVERT_DW_CONV_WITH_ONE_INPUT_CH_AND_OUTPUT_CH_ABOVE_THRESHOLDmacro
• OPTIONAL_RESTRICT_KEYWORDmacro
• arm_nn_size_mulfunctionFold one dimension into a running buffer-size product, reporting overflow as -1.
• arm_nn_size_addfunctionAdd to a running buffer-size product, reporting overflow as -1.
• PACK_S8x4_32x1macrodefinition to pack four 8 bit values.
• PACK_Q15x2_32x1macrodefinition to pack two 16 bit values.
• GetNearestNeighborfunctionMap an output index to the nearest input index for resize.
• arm_nn_is_convolve_1x1functionCheck if convolution parameters correspond to a 1x1 convolution.
• arm_nn_is_convolve_1x1_fastfunctionCheck if a 1x1 convolution qualifies for the fast (unit stride) path.
• arm_nn_is_convolve_1_x_nfunctionCheck if convolution parameters correspond to a 1xN convolution.
• arm_nn_convolve_1_x_n_padding_supportedfunctionCheck that armconvolve1xns4() handles the horizontal padding of a 1xN convolution.
• arm_nn_convolve_1_x_n_s8_padding_supportedfunctionCheck that armconvolve1xns8() accepts the padding and output shape of a 1xN convolution.
• arm_nn_convolve_1_x_n_padded_columnsfunctionCount the output columns of a 1xN convolution whose window reads padding.
• arm_nn_dw_conv_opt_dilation_supportedfunctionCheck if the dilation, stride and padding of a depthwise layer allow the armdepthwiseconvs8opt() or armdepthwiseconvfasts16() route.
• arm_q7_to_q15_with_offsetfunctionConverts the elements from a s8 vector to a s16 vector with an added offset.
• arm_depthwise_conv_s8_opt_get_buffer_size_mvefunctionGet the required buffer size for optimized s8 depthwise convolution function with constraint that inchannel equals outchannel.
• arm_depthwise_conv_s8_opt_get_buffer_size_dspfunctionGet the required buffer size for optimized s8 depthwise convolution function with constraint that inchannel equals outchannel.
• arm_nn_depthwise_conv_s8_corefunctionDepthwise conv on an im2col buffer where the input channel equals output channel.
• arm_nn_mat_mult_s8functionGeneral Matrix-multiplication function with per-channel requantization.
• arm_nn_mat_mult_kernel_s16functionMatrix-multiplication function for convolution with per-channel requantization for 16 bits convolution.
• arm_nn_mat_mul_core_1x_s8functionGeneral Vector by Matrix multiplication with requantization and storage of result.
• arm_nn_mat_mul_core_1x_s4functionGeneral Vector by Matrix multiplication with requantization, storage of result and int4 weights packed into an int8 buffer.
• arm_nn_mat_mul_core_4x_s8functionMatrix-multiplication with requantization & activation function for four rows and one column.
• arm_nn_mat_mult_nt_t_s4functionGeneral Matrix-multiplication function with per-channel requantization.
• arm_nn_mat_mult_nt_interleaved_t_even_s4functionGeneral Matrix-multiplication function with per-channel requantization.
• arm_nn_mat_mult_nt_t_s8functionGeneral Matrix-multiplication function with per-channel requantization.
• arm_nn_mat_mult_nt_t_1x1_out_s8functionGeneral Matrix-multiplication function with per-channel requantization.
• arm_nn_mat_mult_nt_t_s16functionGeneral Matrix-multiplication function with per-channel requantization and int16 input (LHS) and output.
• arm_nn_mat_mult_nt_t_s8_s32functionGeneral Matrix-multiplication function with int8 input and int32 output.
• arm_nn_vec_mat_mult_t_s4functions4 Vector by Matrix (transposed) multiplication
• arm_nn_vec_mat_mult_t_s8functions8 Vector by Matrix (transposed) multiplication
• arm_nn_vec_mat_mult_t_per_ch_s8functions8 Vector by Matrix (transposed) multiplication using per channel quantization for output
• arm_nn_vec_mat_mult_t_s16functions16 Vector by s8 Matrix (transposed) multiplication
• arm_nn_vec_mat_mult_t_per_ch_s16functions16 vector(lhs) by s8 matrix (transposed) multiplication and per channel quant output
• arm_nn_vec_mat_mult_t_s16_s16functions16 Vector by s16 Matrix (transposed) multiplication
• arm_nn_vec_mat_mult_t_svdf_s8functions8 Vector by Matrix (transposed) multiplication with s16 output
• arm_nn_depthwise_conv_nt_t_padded_s8functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
• arm_nn_depthwise_conv_nt_t_s8functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
• arm_nn_depthwise_conv_s8_planar_candidatefunctionNecessary conditions of the planar rule that are cheap to test inline: at most 32 channels and stride 1.
• arm_nn_is_convolve_s8_small_cinfunctionThe gate of armconvolves8smallcin(): upscaledims NULL, input depth 1 to 3 with filter depth equal to it, dilation 1, a kernel of at least 1x1 with kernel width…
• arm_nn_is_convolve_s8_3x3_c16_s1functionThe gate of armconvolves83x3c16s1(): upscaledims NULL, input and filter depth 16, a 3x3 kernel, and stride and dilation 1.
• arm_nn_convolve_s8_groups_invalidfunctionThe group check of armconvolves8(), for its direct entries: with groups = CIN / filter C, CIN or COUT is not a multiple of groups.
• arm_nn_depthwise_conv_s8_planar_bytesfunctionPlane size in bytes that armnndepthwiseconvs8planar() needs for a layer, or -1 when the layer is not one it takes.
• arm_nn_depthwise_conv_s8_planarfunctions8 depthwise convolution with channel multiplier 1 and stride 1, vectorized across the output pixels of one channel plane instead of across channels.
• arm_nn_depthwise_conv_nt_t_s4functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
• arm_nn_depthwise_conv_nt_t_s16functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
• arm_nn_transpose_conv_row_s8_s32functionRow of s8 scalars multiplicated with a s8 matrix ad accumulated into a s32 rolling scratch buffer.
• arm_nn_read_q15x2_iafunctionRead 2 s16 elements and post increment pointer.
• arm_nn_read_s8x4_iafunctionRead 4 s8 from s8 pointer and post increment pointer.
• arm_nn_read_s8x2_iafunctionRead 2 s8 from s8 pointer and post increment pointer.
• arm_nn_read_s16x2functionRead 2 int16 values from int16 pointer.
• arm_nn_read_s8x4functionRead 4 s8 values.
• arm_nn_read_s8x2functionRead 2 s8 values.
• arm_nn_write_s8x4_iafunctionWrite four s8 to s8 pointer and increment pointer afterwards.
• arm_memset_s8functionmemset optimized for MVE
• arm_memset_s16functionmemset optimized for MVE for 16-bit data.
• arm_nn_mat_mult_kernel_s4_s16functionMatrix-multiplication function for convolution with per-channel requantization and 4 bit weights.
• arm_nn_mat_mult_kernel_s8_s16functionMatrix-multiplication function for convolution with per-channel requantization.
• arm_nn_mat_mult_kernel_row_offset_s8_s16functionMatrix-multiplication function for convolution with per-channel requantization, supporting an address offset between rows.
• arm_nn_softmax_common_s8functionCommon softmax function for s8 input and s8 or s16 output.
• NN_ROUNDmacromacro for adding rounding offset
• MUL_SATmacro
• MUL_SAT_MVEmacro
• MUL_POW2macro
• DIV_POW2macro
• DIV_POW2_MVEmacro
• EXP_ON_NEGmacro
• ONE_OVER1macro
• arm_nn_doubling_high_multfunctionSaturating doubling high multiply.
• arm_nn_doubling_high_mult_no_satfunctionDoubling high multiply without saturation.
• arm_nn_divide_by_power_of_twofunctionRounding divide by power of two.
• arm_nn_nonneg_divide_by_pot_s32functionRounding divide by power of two for non-negative values.
• arm_nn_requantizefunctionRequantize a given value.
• arm_nn_requantize_s64functionRequantize a given 64 bit value.
• arm_nn_sat_lshift_s16functionSaturating left shift for int16t.
• arm_nn_sqrdmulh_s16functionSaturating Rounding Doubling High Mul (s16).
• arm_nn_sqdmulh_s16functionSaturating Non-rounded Doubling High Mul (s16).
• arm_nn_divide_by_power_of_two_s16functionRounding divide by power of two (s16), midpoint away from zero.
• arm_memcpy_s8functionmemcpy optimized for MVE
• arm_memcpy_s16functionmemcpy optimized for MVE
• arm_memcpy_s32functionmemcpy optimized for MVE
• arm_memcpy_q15functionmemcpy wrapper for int16
• arm_nn_exp_on_negative_valuesfunctionFixed-point exp() of a non-positive value.
• SELECT_IF_NON_ZEROmacro
• arm_nn_mult_by_power_of_twofunctionSaturating multiply by a power of two.
• arm_nn_one_over_one_plus_x_for_x_in_0_1functionFixed-point 1 / (1 + x) for x in [0, 1), computed with Newton-Raphson iterations.
• arm_nn_write_q15x2_iafunctionWrite 2 s16 elements and post increment pointer.
• arm_nn_write_s8x2_iafunctionWrite 2 s8 elements and post increment pointer.
• arm_cmsis_nn_dim_atfunctionGet dimension value at specific index.
• arm_cmsis_nn_shape_productfunctionCalculate the product of all dimensions in a shape array.
• arm_nn_lstm_step_s8functionUpdate LSTM function for an iteration step using s8 input and output, and s16 internally.
• arm_nn_lstm_step_s16functionUpdate LSTM function for an iteration step using s16 input and output, and s16 internally.
• arm_nn_lstm_calculate_gate_s8_s16functionUpdates a LSTM gate for an iteration step of LSTM function, int8x816 version.
• arm_nn_lstm_calculate_gate_s16functionUpdates a LSTM gate for an iteration step of LSTM function, int16x816 version.
• arm_nn_vec_mat_mul_result_acc_s8_s16functionThe result of the multiplication is accumulated to the passed result buffer.
• arm_nn_vec_mat_mul_result_acc_s16functionThe result of the multiplication is accumulated to the passed result buffer.
• arm_elementwise_mul_s16_s8functions16 elementwise multiplication with s8 output
• arm_elementwise_mul_s16_batch_offsetfunctions16 elementwise multiplication with s16 output
• arm_elementwise_mul_acc_s16functions16 elementwise multiplication.
• arm_check_broadcast_requiredfunctionCheck if a broadcast is required between 2 cmsisnndims.
• arm_reduce_get_middle_block_from_arraysfunctionReports whether the reduced axes of a 4-D tensor form one contiguous block followed by kept axes, as in a NHWC mean over H and W, and gives the flattened sizes.
• ARM_NN_SQRT_S16_TABLEFREE_SHIFTmacro
• ARM_NN_SQRT_S16_TABLEFREE_MAGICmacro
• ARM_NN_SQRT_S16_TABLEFREE_K0macro
• ARM_NN_SQRT_S16_TABLEFREE_K1macro
• arm_nn_sqrt_s16_tablefree_elementfunctionOne element of armsqrts16tablefree(): the float32 chain the MVE path evaluates per lane, so the two agree bit for bit on any IEEE-754 float32 implementation wi…

macro

USE_FAST_DW_CONV_S16_FUNCTION

C

Include/arm_nnsupportfunctions.h:45

#define USE_FAST_DW_CONV_S16_FUNCTION(dw_conv_params, filter_dims, input_dims, output_dims) (dw_conv_params->ch_mult == 1 && \ arm_nn_dw_conv_opt_dilation_supported(dw_conv_params, input_dims, filter_dims, output_dims) && \ filter_dims->w * filter_dims->h < 512)

Parameters

Parameters of USE_FAST_DW_CONV_S16_FUNCTION

Name	Description
dw_conv_params
filter_dims
input_dims
output_dims

macro

LEFT_SHIFT

C

Include/arm_nnsupportfunctions.h:50

#define LEFT_SHIFT(_shift) (_shift > 0 ? _shift : 0)

Parameters

Parameters of LEFT_SHIFT

Name	Description
_shift

macro

RIGHT_SHIFT

C

Include/arm_nnsupportfunctions.h:51

#define RIGHT_SHIFT(_shift) (_shift > 0 ? 0 : -_shift)

Parameters

Parameters of RIGHT_SHIFT

Name	Description
_shift

macro

MASK_IF_ZERO

C

Include/arm_nnsupportfunctions.h:52

#define MASK_IF_ZERO(x) (x) == 0 ? ~0 : 0

Parameters

Parameters of MASK_IF_ZERO

Name	Description
x

macro

MASK_IF_NON_ZERO

C

Include/arm_nnsupportfunctions.h:53

#define MASK_IF_NON_ZERO(x) (x) != 0 ? ~0 : 0

Parameters

Parameters of MASK_IF_NON_ZERO

Name	Description
x

macro

SELECT_USING_MASK

C

Include/arm_nnsupportfunctions.h:54

#define SELECT_USING_MASK(mask, a, b) ((mask) & (a)) ^ (~(mask) & (b))

Parameters

Parameters of SELECT_USING_MASK

Name	Description
mask
a
b

macro

ARM_NN_MAX

C

Include/arm_nnsupportfunctions.h:57

#define ARM_NN_MAX(A, B) ((A) > (B) ? (A) : (B))

Parameters

Parameters of ARM_NN_MAX

Name	Description
A
B

macro

ARM_NN_MIN

C

Include/arm_nnsupportfunctions.h:58

#define ARM_NN_MIN(A, B) ((A) < (B) ? (A) : (B))

Parameters

Parameters of ARM_NN_MIN

Name	Description
A
B

macro

ARM_NN_CLAMP

C

Include/arm_nnsupportfunctions.h:59

#define ARM_NN_CLAMP(x, h, l) ARM_NN_MAX(ARM_NN_MIN((x), (h)), (l))

Parameters

Parameters of ARM_NN_CLAMP

Name	Description
x
h
l

function

arm_nn_min_f16h

C

Minimum of two scalar f16 values.

Include/arm_nnsupportfunctions.h:98

static _Float16 arm_nn_min_f16h(_Float16 a, _Float16 b)

Minimum of two scalar f16 values.

With ARM_NN_F16_CMOV_WORKAROUND this is IEEE 754 minNum via VMINNM.F16: a NaN operand is suppressed and the non-NaN operand wins. The scalar fallback is ARM_NN_MIN, an ordered compare, so its NaN handling depends on operand order: a NaN b is returned, a NaN a is not. Do not rely on NaN suppression on non-MVE builds.

Parameters

Parameters of arm_nn_min_f16h

Name	Type	Direction	Description
a	_Float16	in	First operand
b	_Float16	in	Second operand

Returns

Returns of arm_nn_min_f16h

Description
The smaller of `a` and `b`

function

arm_nn_max_f16h

C

Maximum of two scalar f16 values.

Include/arm_nnsupportfunctions.h:121

static _Float16 arm_nn_max_f16h(_Float16 a, _Float16 b)

Maximum of two scalar f16 values.

With ARM_NN_F16_CMOV_WORKAROUND this is IEEE 754 maxNum via VMAXNM.F16: a NaN operand is suppressed and the non-NaN operand wins. The scalar fallback is ARM_NN_MAX, an ordered compare, so its NaN handling depends on operand order: a NaN b is returned, a NaN a is not. Do not rely on NaN suppression on non-MVE builds.

Parameters

Parameters of arm_nn_max_f16h

Name	Type	Direction	Description
a	_Float16	in	First operand
b	_Float16	in	Second operand

Returns

Returns of arm_nn_max_f16h

Description
The larger of `a` and `b`

function

arm_nn_propagate_nan_f16h

C

Returns x when x is NaN, otherwise y.

Include/arm_nnsupportfunctions.h:167

static _Float16 arm_nn_propagate_nan_f16h(_Float16 x, _Float16 y)

Returns x when x is NaN, otherwise y.

Both the NaN test and the select are performed on the bit patterns: the test is (bits & 0x7FFF) > 0x7C00 (all-ones exponent, non-zero mantissa), which is integer arithmetic that -ffinite-math-only (implied by the shipped -Ofast) has no license to fold, unlike the former floating-point self-compare x != x (#333 / #334); the bit-pattern select neither expands to an HFmode conditional move (PR target/118460) nor quiets/retags the NaN payload. This helper backs the f16 elementwise clamp and, via arm_nn_clamp_scalar_f16 / arm_nn_clamp_propagate_nan_f16h, the other f16 scalar clamp users arm_svdf_f16, arm_max_pool_f16 / arm_avg_pool_f16, the packed f16 matmul (arm_nn_mat_mult_nt_n_packed_f16), the scalar f16 RELU/RELU6/LEAKY_RELU activation legs, and arm_nn_vector_clamp_f16’s scalar leg (conv/depthwise/transpose-conv f16, the 3x3 depthwise, and arm_nn_maxpool1d_f16) so wherever that scalar clamp runs, a NaN passes through it at every optimization level on the gated toolchains. That is a guarantee about the clamp, not the whole kernel: which builds run the scalar clamp, and whether a NaN survives the rest of the kernel to reach it, is per kernel several of these callers clamp with vmaxnmq/vminnmq on MVE builds (a NaN resolves to a bound there), and arm_max_pool_f16’s max reduction drops a NaN before the clamp. The kernels with a NaN

Note

(svdf, max/avg pool, packed matmul, activation) state their exact scope there; the arm_nn_vector_clamp_f16 family is covered by a test assertion in the transpose-conv f16 suite rather than per-kernel notes. The cortex-m55 MVE RELU/RELU6 f16 legs reach the same guarantee by the vector form of this idiom rather than by calling this helper: they restore NaN lanes with arm_nn_max_propagate_nan_mve_f16 / arm_nn_clamp_propagate_nan_mve_f16 (#382). The same idiom (bit-classified select) appears in arm_prelu_f16, which does not call this helper.

Parameters

Parameters of arm_nn_propagate_nan_f16h

Name	Type	Direction	Description
x	_Float16	in	Value whose NaN-ness selects the result. Returned unchanged when it is NaN.
y	_Float16	in	Value returned when `x` is not NaN

Returns

Returns of arm_nn_propagate_nan_f16h

Description
`x` if `x` is NaN, otherwise `y`

function

arm_nn_clamp_f16h

C

Drop-in equivalent of ARMNNCLAMP(x, h, l) for scalar Float16 operands.

Include/arm_nnsupportfunctions.h:193

static _Float16 arm_nn_clamp_f16h(_Float16 x, _Float16 h, _Float16 l)

Drop-in equivalent of ARM_NN_CLAMP(x, h, l) for scalar _Float16 operands.

Includes the macro’s NaN behaviour: ARM_NN_MIN(NaN, h) is h, so a NaN input resolves to the high bound, exactly as the macro does. Use arm_nn_clamp_propagate_nan_f16h() where TFLite NaN propagation is required.

Parameters

Parameters of arm_nn_clamp_f16h

Name	Type	Direction	Description
x	_Float16	in	Value to clamp
h	_Float16	in	Upper bound
l	_Float16	in	Lower bound

Returns

Returns of arm_nn_clamp_f16h

Description
`x` clamped to [`l`, `h`]

function

arm_nn_clamp_propagate_nan_f16h

C

Scalar f16 clamp with TFLite NaN semantics: NaN passes through unchanged.

Include/arm_nnsupportfunctions.h:212

static _Float16 arm_nn_clamp_propagate_nan_f16h(_Float16 x, _Float16 l, _Float16 h)

Scalar f16 clamp with TFLite NaN semantics: NaN passes through unchanged.

Mirrors the MVE idiom in arm_nn_clamp_propagate_nan_mve_f16() (lower bound first, then upper bound, then restore NaN lanes). The NaN restore in arm_nn_propagate_nan_f16h() tests the integer bit pattern, so it holds at every optimization level including the shipped -Ofast; see #333 / #334. Bounds are assumed ordered (l <= h); inverted bounds are unspecified.

Parameters

Parameters of arm_nn_clamp_propagate_nan_f16h

Name	Type	Direction	Description
x	_Float16	in	Value to clamp
l	_Float16	in	Lower bound
h	_Float16	in	Upper bound

Returns

Returns of arm_nn_clamp_propagate_nan_f16h

Description
`x` clamped to [`l`, `h`], or `x` itself when it is NaN

function

arm_nn_abs_f16h

C

Absolute value of a scalar f16 value.

Include/arm_nnsupportfunctions.h:224

static _Float16 arm_nn_abs_f16h(_Float16 x)

Absolute value of a scalar f16 value.

Parameters

Parameters of arm_nn_abs_f16h

Name	Type	Direction	Description
x	_Float16	in	Input value

Returns

Returns of arm_nn_abs_f16h

Description
|`x`|

macro

ARM_NN_ROUND_UP

C

Include/arm_nnsupportfunctions.h:236

#define ARM_NN_ROUND_UP(x, multiple) ((((x) + (multiple) - 1) / (multiple)) * (multiple))

Parameters

Parameters of ARM_NN_ROUND_UP

Name	Description
x
multiple

macro

REDUCE_MULTIPLIER

C

Include/arm_nnsupportfunctions.h:237

#define REDUCE_MULTIPLIER(_mult) ((_mult < 0x7FFF0000) ? ((_mult + (1 << 15)) >> 16) : 0x7FFF)

Parameters

Parameters of REDUCE_MULTIPLIER

Name	Description
_mult

macro

CH_IN_BLOCK_MVE

C

Include/arm_nnsupportfunctions.h:245

#define CH_IN_BLOCK_MVE (124)

macro

S4_CH_IN_BLOCK_MVE

C

Include/arm_nnsupportfunctions.h:250

#define S4_CH_IN_BLOCK_MVE (124)

macro

MAX_COL_COUNT

C

Include/arm_nnsupportfunctions.h:254

#define MAX_COL_COUNT (512)

macro

REVERSE_TCOL_EFFICIENT_THRESHOLD

C

Include/arm_nnsupportfunctions.h:258

#define REVERSE_TCOL_EFFICIENT_THRESHOLD (16)

macro

CONVERT_DW_CONV_WITH_ONE_INPUT_CH_AND_OUTPUT_CH_ABOVE_THRESHOLD

C

Include/arm_nnsupportfunctions.h:266

#define CONVERT_DW_CONV_WITH_ONE_INPUT_CH_AND_OUTPUT_CH_ABOVE_THRESHOLD (1)

macro

OPTIONAL_RESTRICT_KEYWORD

C

Include/arm_nnsupportfunctions.h:272

#define OPTIONAL_RESTRICT_KEYWORD

function

arm_nn_size_mul

C

Fold one dimension into a running buffer-size product, reporting overflow as -1.

Include/arm_nnsupportfunctions.h:307

static int64_t arm_nn_size_mul(const int64_t acc, const int64_t factor)

Fold one dimension into a running buffer-size product, reporting overflow as -1.

Buffer-size queries return an int32_t byte count, so the product of the dimensions they multiply has to be rejected as soon as it cannot fit. Folding one factor at a time keeps the accumulator bounded: an accumulator already known to be <= INT32_MAX times a factor <= INT32_MAX cannot exceed about 2^62, so the int64_t accumulator itself never wraps. Chaining raw (int64_t) casts across three or more int32_t dims does not have that property - 65536 * 65536 * 65536 * 65536 is exactly 2^64 and folds back to 0, which would sail through a trailing “> INT32_MAX” test.

Note

This is the -1 sentinel family, used by the s8/s16 integer buffer-size queries, by the eight SVDF staging queries (arm_svdf_{s8,state_s16_s8,f32,f16}_{input,output}ctx_get_buffer_size) and by the s8/s16 LSTM temp-buffer queries and the GRU temp queries (arm_lstm_unidirectional{s8,s16}_temp{1,2}get_buffer_size, arm_gru_unidirectional{f32,f16}_temp1_get_buffer_size). The four f32/f16 LSTM temp queries have no dimensions to fold (the buffers are unused) and answer -1 only for NULL params, 0 otherwise. It is not interchangeable with the arm_nn_checked_size_mul() / arm_nn_size_to_i32_or_zero() helpers in Source/NNSupportFunctions (shared header for the float sizers), which most f32 and f16 buffer-size queries use and which report an out-of-range size as 0. Mixing the two silently flips a sizer’s out-of-range contract from “must never be used to size a buffer” to “you may pass { NULL, 0 }”, so pick the one the surrounding family already uses.

Note

The split is per sizer, not per datatype. The four SVDF f32/f16 staging queries deliberately use this -1 family rather than the 0 one their neighbours use, because their kernels read ctx->size and a size of 0 opts out of the scratch-size check - so a 0-on-overflow answer fed back as { alloc(0), 0 } would disable the very check meant to catch it. Do not infer a sizer’s sentinel from its datatype suffix.

Parameters

Parameters of arm_nn_size_mul

Name	Type	Direction	Description
acc	const int64_t	in	Running product, or -1 if an earlier fold already overflowed.
factor	const int64_t	in	Next factor to fold in.

Returns

Returns of arm_nn_size_mul

Description
acc * factor, or -1 if acc is already -1, factor is negative or out of int32_t range, or the product exceeds INT32_MAX.

function

arm_nn_size_add

C

Add to a running buffer-size product, reporting overflow as -1.

Include/arm_nnsupportfunctions.h:332

static int64_t arm_nn_size_add(const int64_t acc, const int64_t addend)

Add to a running buffer-size product, reporting overflow as -1.

Companion to arm_nn_size_mul() for the sizers that append a fixed slack term.

Note

Same sentinel caveat as arm_nn_size_mul() - see its note for the full split, including why the four SVDF f32/f16 staging queries use this -1 family rather than the 0-returning arm_nn_checked_size_mul() / arm_nn_size_to_i32_or_zero() family that most other float sizers use.

Parameters

Parameters of arm_nn_size_add

Name	Type	Direction	Description
acc	const int64_t	in	Running product, or -1 if an earlier step already overflowed.
addend	const int64_t	in	Value to add. Must be non-negative.

Returns

Returns of arm_nn_size_add

Description
acc + addend, or -1 if acc is already -1 or the sum exceeds INT32_MAX.

macro

PACK_S8x4_32x1

C

definition to pack four 8 bit values.

Include/arm_nnsupportfunctions.h:358

#define PACK_S8x4_32x1(v0, v1, v2, v3) ((int32_t)((((uint32_t)(v0)) & 0xFFu) | ((((uint32_t)(v1)) & 0xFFu) << 8) | ((((uint32_t)(v2)) & 0xFFu) << 16) | \ ((((uint32_t)(v3)) & 0xFFu) << 24)))

definition to pack four 8 bit values.

Byte lanes are masked and shifted in uint32_t so a negative value never feeds a signed left shift (UB); masking before the shift keeps the same bits the old shift-then-mask form kept. Bit-identical for every input. Deliberate divergence from upstream ARM-software/CMSIS-NN, which still carries the signed-shift form do not paste the upstream text back on a sync (issue #357).

Parameters

Parameters of PACK_S8x4_32x1

Name	Description
v0
v1
v2
v3

macro

PACK_Q15x2_32x1

C

definition to pack two 16 bit values.

Include/arm_nnsupportfunctions.h:369

#define PACK_Q15x2_32x1(v0, v1) ((int32_t)((((uint32_t)(v0)) & 0xFFFFu) | (((uint32_t)(v1)) << 16)))

definition to pack two 16 bit values.

Same treatment: the high half is shifted in uint32_t, not int32_t, so a negative v1 is defined; the low half keeps its mask. Bit-identical for every input. Same deliberate upstream divergence as PACK_S8x4_32x1 above.

Parameters

Parameters of PACK_Q15x2_32x1

Name	Description
v0
v1

function

GetNearestNeighbor

C

Map an output index to the nearest input index for resize.

Include/arm_nnsupportfunctions.h:391

static int32_t GetNearestNeighbor(
    const int input_value,
    const int32_t input_size,
    const float scale,
    const float offset,
    const bool align_corners,
    const bool half_pixel_centers
)

Map an output index to the nearest input index for resize.

This helper follows the TensorFlow Lite nearest-neighbor resize mapping rules.

Parameters

Parameters of GetNearestNeighbor

Name	Type	Direction	Description
input_value	const int	in	Output index (x or y).
input_size	const int32_t	in	Input size along the same axis.
scale	const float	in	Precomputed scaling factor for the axis.
offset	const float	in	Precomputed offset for the axis.
align_corners	const bool	in	If true, use align-corners scaling.
half_pixel_centers	const bool	in	If true, use half-pixel center offset.

Returns

Returns of GetNearestNeighbor

Description
Nearest input index for the given output index.

function

arm_nn_is_convolve_1x1

C

Check if convolution parameters correspond to a 1x1 convolution.

Include/arm_nnsupportfunctions.h:416

static bool arm_nn_is_convolve_1x1(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims
)

Check if convolution parameters correspond to a 1x1 convolution.

Parameters

Parameters of arm_nn_is_convolve_1x1

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters
input_dims	const cmsis_nn_dims *	in	Input dimensions
filter_dims	const cmsis_nn_dims *	in	Filter dimensions

Returns

Returns of arm_nn_is_convolve_1x1

Description
true if parameters describe a 1x1 convolution, false otherwise.

function

arm_nn_is_convolve_1x1_fast

C

Check if a 1x1 convolution qualifies for the fast (unit stride) path.

Include/arm_nnsupportfunctions.h:432

static bool arm_nn_is_convolve_1x1_fast(const cmsis_nn_conv_params *conv_params)

Check if a 1x1 convolution qualifies for the fast (unit stride) path.

Note

Does not validate that the kernel is 1x1. Call arm_nn_is_convolve_1x1() first.

Parameters

Parameters of arm_nn_is_convolve_1x1_fast

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters

Returns

Returns of arm_nn_is_convolve_1x1_fast

Description
true if stride is 1x1, false otherwise.

function

arm_nn_is_convolve_1_x_n

C

Check if convolution parameters correspond to a 1xN convolution.

Include/arm_nnsupportfunctions.h:444

static bool arm_nn_is_convolve_1_x_n(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims
)

Check if convolution parameters correspond to a 1xN convolution.

Parameters

Parameters of arm_nn_is_convolve_1_x_n

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters
input_dims	const cmsis_nn_dims *	in	Input dimensions
filter_dims	const cmsis_nn_dims *	in	Filter dimensions

Returns

Returns of arm_nn_is_convolve_1_x_n

Description
true if parameters describe a 1xN convolution, false otherwise.

function

arm_nn_convolve_1_x_n_padding_supported

C

Check that armconvolve1xns4() handles the horizontal padding of a 1xN convolution.

Include/arm_nnsupportfunctions.h:473

static bool arm_nn_convolve_1_x_n_padding_supported(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Check that arm_convolve_1_x_n_s4() handles the horizontal padding of a 1xN convolution.

The kernel places pad.w columns on the left and pad.w + (total_pad % 2) on the right, where total_pad = (output W - 1) * stride.w + filter W - input W, and needs the output columns that read padding to fit in output W. Its padded-column code also assumes that each such column reads at least one input column and that the filter is no wider than the input; otherwise it forms input and filter addresses outside the tensors. On MVE builds another pad placement, or too many padded columns, returns ARM_CMSIS_NN_FAILURE. A VALID layer whose stride leaves trailing input unused (negative total_pad) is therefore rejected, and the wrapper routes it to another convolution. The kernel also computes a single output row, so vertical padding or an output height other than 1 is rejected. A non-positive stride.w is left to the kernel’s argument checks.

Parameters

Parameters of arm_nn_convolve_1_x_n_padding_supported

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters
input_dims	const cmsis_nn_dims *	in	Input dimensions
filter_dims	const cmsis_nn_dims *	in	Filter dimensions
output_dims	const cmsis_nn_dims *	in	Output dimensions

Returns

Returns of arm_nn_convolve_1_x_n_padding_supported

Description
true when `arm_convolve_1_x_n_s4()` handles the padding, false otherwise.

function

arm_nn_convolve_1_x_n_s8_padding_supported

C

Check that armconvolve1xns8() accepts the padding and output shape of a 1xN convolution.

Include/arm_nnsupportfunctions.h:517

static bool arm_nn_convolve_1_x_n_s8_padding_supported(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Check that arm_convolve_1_x_n_s8() accepts the padding and output shape of a 1xN convolution.

The kernel computes a single output row for any pad.w >= 0 and any output width, including an odd total padding, a filter wider than the input and a VALID layer whose stride leaves trailing input unused. It rejects vertical padding, an output height other than 1, a negative pad.w and an empty filter; the wrapper routes those layers to another convolution. A non-positive stride.w is left to the kernel’s argument checks.

Parameters

Parameters of arm_nn_convolve_1_x_n_s8_padding_supported

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters
filter_dims	const cmsis_nn_dims *	in	Filter dimensions
output_dims	const cmsis_nn_dims *	in	Output dimensions

Returns

Returns of arm_nn_convolve_1_x_n_s8_padding_supported

Description
true when `arm_convolve_1_x_n_s8()` computes the layer, false otherwise.

function

arm_nn_convolve_1_x_n_padded_columns

C

Count the output columns of a 1xN convolution whose window reads padding.

Include/arm_nnsupportfunctions.h:539

static void arm_nn_convolve_1_x_n_padded_columns(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims,
    int64_t *left_num,
    int64_t *right_num
)

Count the output columns of a 1xN convolution whose window reads padding.

Output column j reads input columns j * stride.w - pad.w to j * stride.w - pad.w + filter W - 1. The leading columns whose window starts before the input are left-padded; of the others, the trailing columns whose window ends past the input are right-padded. A window can do both only when it is left-padded.

Parameters

Parameters of arm_nn_convolve_1_x_n_padded_columns

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters. stride.w >= 1 and pad.w >= 0.
input_dims	const cmsis_nn_dims *	in	Input dimensions. w >= 0.
filter_dims	const cmsis_nn_dims *	in	Filter dimensions. w >= 1.
output_dims	const cmsis_nn_dims *	in	Output dimensions. w >= 0.
left_num	int64_t *	out	Number of left-padded output columns, output W at most.
right_num	int64_t *	out	Number of right-padded output columns, output W - left_num at most.

function

arm_nn_dw_conv_opt_dilation_supported

C

Check if the dilation, stride and padding of a depthwise layer allow the armdepthwiseconvs8opt() or armdepthwiseconvfasts16() route.

Include/arm_nnsupportfunctions.h:572

static bool arm_nn_dw_conv_opt_dilation_supported(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Check if the dilation, stride and padding of a depthwise layer allow the arm_depthwise_conv_s8_opt() or arm_depthwise_conv_fast_s16() route.

Note

Does not check ch_mult, the batch count or the kernel size: arm_depthwise_conv_wrapper_s8(), arm_depthwise_conv_wrapper_s16() and their buffer-size functions apply their own conditions on those, and all of them take this predicate so that routing and sizing agree.

Parameters

Parameters of arm_nn_dw_conv_opt_dilation_supported

Name	Type	Direction	Description
dw_conv_params	const cmsis_nn_dw_conv_params *	in	Depthwise convolution parameters
input_dims	const cmsis_nn_dims *	in	Input dimensions
filter_dims	const cmsis_nn_dims *	in	Filter dimensions
output_dims	const cmsis_nn_dims *	in	Output dimensions

Returns

Returns of arm_nn_dw_conv_opt_dilation_supported

Description
true for an undilated layer (dilation 1 in both dimensions), or for a 1D layer dilated along the width only: filter, input and output height 1, stride 1 in both dimensions, no vertical padding, dilation.h == 1 and dilation.w >= 1. false otherwise.

function

arm_q7_to_q15_with_offset

C

Converts the elements from a s8 vector to a s16 vector with an added offset.

Include/arm_nnsupportfunctions.h:649

void arm_q7_to_q15_with_offset(const int8_t *src, int16_t *dst, int32_t block_size, int16_t offset)

Converts the elements from a s8 vector to a s16 vector with an added offset.

Output elements are ordered. The equation used for the conversion process is:

dst[n] = (int16_t) src[n] + offset; 0 <= n < block_size.

Parameters

Parameters of arm_q7_to_q15_with_offset

Name	Type	Direction	Description
src	const int8_t *	in	pointer to the s8 input vector
dst	int16_t *	out	pointer to the s16 output vector
block_size	int32_t	in	length of the input vector
offset	int16_t	in	s16 offset to be added to each input vector element.

function

arm_depthwise_conv_s8_opt_get_buffer_size_mve

C

Get the required buffer size for optimized s8 depthwise convolution function with constraint that inchannel equals outchannel.

Include/arm_nnsupportfunctions.h:695

int32_t arm_depthwise_conv_s8_opt_get_buffer_size_mve(const cmsis_nn_dims *input_dims, const cmsis_nn_dims *filter_dims)

Get the required buffer size for optimized s8 depthwise convolution function with constraint that in_channel equals out_channel. This is for processors with MVE extension.

The dimensions are checked here, so a negative dimension returns -1 on every build target. The byte count is range-checked inside the selected leg instead, because the Helium and DSP legs use different formulas and the plain-C build needs no buffer at all.

Note

Intended for compilation on Host. If compiling for an Arm target, use arm_depthwise_conv_s8_opt_get_buffer_size(). Note also this is a support function, so not recommended to call directly even on Host.

Note

This leg sizes its buffer from a fixed channel block rather than from input_dims->c, but it applies the same dimension check as arm_depthwise_conv_s8_opt_get_buffer_size() anyway, returning -1 for a negative input_dims->c or filter dimension, or for a byte count that would not fit in an int32_t. Without that check this entry point - and every s4 depthwise sizer, which route here - answered a negative channel count with a plausible positive size (issue #318).

Parameters

Parameters of arm_depthwise_conv_s8_opt_get_buffer_size_mve

Name	Type	Direction	Description
input_dims	const cmsis_nn_dims *	in	Input (activation) tensor dimensions. Format: [1, H, W, C_IN] Batch argument N is not used.
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]

Returns

Returns of arm_depthwise_conv_s8_opt_get_buffer_size_mve

Description
The function returns required buffer size in bytes, or -1 if any dimension it reads is negative or the required size would not fit in an int32_t

function

arm_depthwise_conv_s8_opt_get_buffer_size_dsp

C

Get the required buffer size for optimized s8 depthwise convolution function with constraint that inchannel equals outchannel.

Include/arm_nnsupportfunctions.h:710

int32_t arm_depthwise_conv_s8_opt_get_buffer_size_dsp(const cmsis_nn_dims *input_dims, const cmsis_nn_dims *filter_dims)

Get the required buffer size for optimized s8 depthwise convolution function with constraint that in_channel equals out_channel. This is for processors with DSP extension.

The dimensions are checked here, so a negative dimension returns -1 on every build target. The byte count is range-checked inside the selected leg instead, because the Helium and DSP legs use different formulas and the plain-C build needs no buffer at all.

Note

Intended for compilation on Host. If compiling for an Arm target, use arm_depthwise_conv_s8_opt_get_buffer_size(). Note also this is a support function, so not recommended to call directly even on Host.

Parameters

Parameters of arm_depthwise_conv_s8_opt_get_buffer_size_dsp

Name	Type	Direction	Description
input_dims	const cmsis_nn_dims *	in	Input (activation) tensor dimensions. Format: [1, H, W, C_IN] Batch argument N is not used.
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]

Returns

Returns of arm_depthwise_conv_s8_opt_get_buffer_size_dsp

Description
The function returns required buffer size in bytes, or -1 if any dimension it reads is negative or the required size would not fit in an int32_t

function

arm_nn_depthwise_conv_s8_core

C

Depthwise conv on an im2col buffer where the input channel equals output channel.

Include/arm_nnsupportfunctions.h:732

int8_t * arm_nn_depthwise_conv_s8_core(
    const int8_t *row,
    const int16_t *col,
    const uint16_t num_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t kernel_size,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise conv on an im2col buffer where the input channel equals output channel.

Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_depthwise_conv_s8_core

Name	Type	Direction	Description
row	const int8_t *	in	pointer to row
col	const int16_t *	in	pointer to im2col buffer, always consists of 2 columns.
num_ch	const uint16_t	in	number of channels
out_shift	const int32_t *	in	pointer to per output channel requantization shift parameter.
out_mult	const int32_t *	in	pointer to per output channel requantization multiplier parameter.
out_offset	const int32_t	in	output tensor offset.
activation_min	const int32_t	in	minimum value to clamp the output to. Range : int8
activation_max	const int32_t	in	maximum value to clamp the output to. Range : int8
kernel_size	const uint16_t	in	number of elements in one column.
output_bias	const int32_t *const	in	per output channel bias. Range : int32
out	int8_t *	out	pointer to output

Returns

Returns of arm_nn_depthwise_conv_s8_core

Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

function

arm_nn_mat_mult_s8

C

General Matrix-multiplication function with per-channel requantization.

Include/arm_nnsupportfunctions.h:766

int8_t * arm_nn_mat_mult_s8(
    const int8_t *input_row,
    const int8_t *input_col,
    const uint16_t output_ch,
    const uint16_t col_batches,
    const int32_t *output_shift,
    const int32_t *output_mult,
    const int32_t out_offset,
    const int32_t col_offset,
    const int32_t row_offset,
    const int16_t out_activation_min,
    const int16_t out_activation_max,
    const uint16_t row_len,
    const int32_t *const bias,
    int8_t *out
)

General Matrix-multiplication function with per-channel requantization.

Supported framework: TensorFlow Lite

Parameters

Parameters of arm_nn_mat_mult_s8

Name	Type	Direction	Description
input_row	const int8_t *	in	pointer to row operand
input_col	const int8_t *	in	pointer to col operand
output_ch	const uint16_t	in	number of rows of input_row
col_batches	const uint16_t	in	number of column batches. Range: 1 to 4
output_shift	const int32_t *	in	pointer to per output channel requantization shift parameter.
output_mult	const int32_t *	in	pointer to per output channel requantization multiplier parameter.
out_offset	const int32_t	in	output tensor offset.
col_offset	const int32_t	in	input tensor(col) offset.
row_offset	const int32_t	in	kernel offset(row). Not used.
out_activation_min	const int16_t	in	minimum value to clamp the output to. Range : int8
out_activation_max	const int16_t	in	maximum value to clamp the output to. Range : int8
row_len	const uint16_t	in	number of elements in each row
bias	const int32_t *const	in	per output channel bias. Range : int32
out	int8_t *	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_s8

Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

function

arm_nn_mat_mult_kernel_s16

C

Matrix-multiplication function for convolution with per-channel requantization for 16 bits convolution.

Include/arm_nnsupportfunctions.h:804

int16_t * arm_nn_mat_mult_kernel_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const int32_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t num_col_a,
    const cmsis_nn_bias_data *const bias_data,
    int16_t *out_0,
    const int32_t row_address_offset
)

Matrix-multiplication function for convolution with per-channel requantization for 16 bits convolution.

This function does the matrix multiplication of weight matrix for all output channels with 2 columns from im2col and produces two elements/output_channel. The outputs are clamped in the range provided by activation min and max. Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_mat_mult_kernel_s16

Name	Type	Direction	Description
input_a	const int8_t *	in	pointer to operand A
input_b	const int16_t *	in	pointer to operand B, always consists of 2 vectors.
output_ch	const int32_t	in	number of rows of A
out_shift	const int32_t *	in	pointer to per output channel requantization shift parameter.
out_mult	const int32_t *	in	pointer to per output channel requantization multiplier parameter.
activation_min	const int32_t	in	minimum value to clamp the output to. Range : int16
activation_max	const int32_t	in	maximum value to clamp the output to. Range : int16
num_col_a	const int32_t	in	number of columns of A
bias_data	const cmsis_nn_bias_data *const	in	pointer to struct with bias vector. The length of this vector is equal to the number of output columns (or RHS input rows). The vector can be int32 or int64 indicated by a flag in the struct.
out_0	int16_t *	in, out	pointer to output
row_address_offset	const int32_t	in	Address offset between rows in output.

Returns

Returns of arm_nn_mat_mult_kernel_s16

Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

function

arm_nn_mat_mul_core_1x_s8

C

General Vector by Matrix multiplication with requantization and storage of result.

Include/arm_nnsupportfunctions.h:843

arm_cmsis_nn_status arm_nn_mat_mul_core_1x_s8(
    int32_t row_elements,
    const int32_t skipped_row_elements,
    const int8_t *row_base_ref,
    const int8_t *col_base_ref,
    const int32_t out_ch,
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const int32_t *bias,
    int8_t *output
)

General Vector by Matrix multiplication with requantization and storage of result.

Pseudo-code *output = 0 sum_col = 0 for (j = 0; j < out_ch; j++) for (i = 0; i < row_elements; i++) *output += row_base_ref[i] * col_base_ref[i] sum_col += col_base_ref[i] scale sum_col using quant_params and bias store result in ‘output’

Parameters

Parameters of arm_nn_mat_mul_core_1x_s8

Name	Type	Direction	Description
row_elements	int32_t	in	number of row elements
skipped_row_elements	const int32_t	in	number of row elements skipped due to padding. row_elements + skipped_row_elements = (kernel_x * kernel_y) * input_ch
row_base_ref	const int8_t *	in	pointer to row operand
col_base_ref	const int8_t *	in	pointer to col operand
out_ch	const int32_t	in	Number of output channels
conv_params	const cmsis_nn_conv_params *	in	Pointer to convolution parameters like offsets and activation values
quant_params	const cmsis_nn_per_channel_quant_params *	in	Pointer to per-channel quantization parameters
bias	const int32_t *	in	Pointer to optional per-channel bias
output	int8_t *	out	Pointer to output where int8 results are stored.

Returns

Returns of arm_nn_mat_mul_core_1x_s8

Description
The function performs matrix(row_base_ref) multiplication with vector(col_base_ref) and scaled result is stored in memory.

function

arm_nn_mat_mul_core_1x_s4

C

General Vector by Matrix multiplication with requantization, storage of result and int4 weights packed into an int8 buffer.

Include/arm_nnsupportfunctions.h:881

arm_cmsis_nn_status arm_nn_mat_mul_core_1x_s4(
    int32_t row_elements,
    const int32_t skipped_row_elements,
    const int8_t *row_base_ref,
    const int8_t *col_base_ref,
    const int32_t out_ch,
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const int32_t *bias,
    int8_t *output
)

General Vector by Matrix multiplication with requantization, storage of result and int4 weights packed into an int8 buffer.

Pseudo-code as int8 example. Int4 filter data will be unpacked. *output = 0 sum_col = 0 for (j = 0; j < out_ch; j++) for (i = 0; i < row_elements; i++) *output += row_base_ref[i] * col_base_ref[i] sum_col += col_base_ref[i] scale sum_col using quant_params and bias store result in ‘output’

Parameters

Parameters of arm_nn_mat_mul_core_1x_s4

Name	Type	Direction	Description
row_elements	int32_t	in	number of row elements
skipped_row_elements	const int32_t	in	number of row elements skipped due to padding. row_elements + skipped_row_elements = (kernel_x * kernel_y) * input_ch
row_base_ref	const int8_t *	in	pointer to row operand
col_base_ref	const int8_t *	in	pointer to col operand as packed int4
out_ch	const int32_t	in	Number of output channels
conv_params	const cmsis_nn_conv_params *	in	Pointer to convolution parameters like offsets and activation values
quant_params	const cmsis_nn_per_channel_quant_params *	in	Pointer to per-channel quantization parameters
bias	const int32_t *	in	Pointer to optional per-channel bias
output	int8_t *	out	Pointer to output where int8 results are stored.

Returns

Returns of arm_nn_mat_mul_core_1x_s4

Description
The function performs matrix(row_base_ref) multiplication with vector(col_base_ref) and scaled result is stored in memory.

function

arm_nn_mat_mul_core_4x_s8

C

Matrix-multiplication with requantization & activation function for four rows and one column.

Include/arm_nnsupportfunctions.h:908

int8_t * arm_nn_mat_mul_core_4x_s8(
    const int32_t row_elements,
    const int32_t offset,
    const int8_t *row_base,
    const int8_t *col_base,
    const int32_t out_ch,
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const int32_t *bias,
    int8_t *output
)

Matrix-multiplication with requantization & activation function for four rows and one column.

Compliant to TFLM int8 specification. MVE implementation only

Parameters

Parameters of arm_nn_mat_mul_core_4x_s8

Name	Type	Direction	Description
row_elements	const int32_t	in	number of row elements
offset	const int32_t	in	offset between rows. Can be the same as row_elements. For e.g, in a 1x1 conv scenario with stride as 1.
row_base	const int8_t *	in	pointer to row operand
col_base	const int8_t *	in	pointer to col operand
out_ch	const int32_t	in	Number of output channels
conv_params	const cmsis_nn_conv_params *	in	Pointer to convolution parameters like offsets and activation values
quant_params	const cmsis_nn_per_channel_quant_params *	in	Pointer to per-channel quantization parameters
bias	const int32_t *	in	Pointer to per-channel bias
output	int8_t *	out	Pointer to output where int8 results are stored.

Returns

Returns of arm_nn_mat_mul_core_4x_s8

Description
The function returns the updated output pointer or NULL if implementation is not available.

function

arm_nn_mat_mult_nt_t_s4

C

General Matrix-multiplication function with per-channel requantization.

Include/arm_nnsupportfunctions.h:950

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s4(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. This function assumes:

• LHS input matrix NOT transposed (nt)
• RHS input matrix transposed (t)
• RHS is int8 packed with 2x int4
• LHS is int8

Note

This operation also performs the broadcast bias addition before the requantization

Parameters

Parameters of arm_nn_mat_mult_nt_t_s4

Name	Type	Direction	Description
lhs	const int8_t *	in	Pointer to the LHS input matrix
rhs	const int8_t *	in	Pointer to the RHS input matrix
bias	const int32_t *	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
dst	int8_t *	out	Pointer to the output matrix with "m" rows and "n" columns
dst_multipliers	const int32_t *	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
dst_shifts	const int32_t *	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
lhs_rows	const int32_t	in	Number of LHS input rows
rhs_rows	const int32_t	in	Number of RHS input rows
rhs_cols	const int32_t	in	Number of LHS/RHS input columns
lhs_offset	const int32_t	in	Offset to be applied to the LHS input value
dst_offset	const int32_t	in	Offset to be applied the output result
activation_min	const int32_t	in	Minimum value to clamp down the output. Range : int8
activation_max	const int32_t	in	Maximum value to clamp up the output. Range : int8
lhs_cols_offset	const int32_t	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_t_s4

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_mat_mult_nt_interleaved_t_even_s4

C

General Matrix-multiplication function with per-channel requantization.

Include/arm_nnsupportfunctions.h:999

arm_cmsis_nn_status arm_nn_mat_mult_nt_interleaved_t_even_s4(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. This function assumes:

• LHS input matrix NOT transposed (nt)
• RHS input matrix transposed (t)
• RHS is int8 packed with 2x int4
• LHS is int8
• LHS/RHS input columns must be even numbered
• LHS must be interleaved. Compare to arm_nn_mat_mult_nt_t_s4 where LHS is not interleaved.

Note

This operation also performs the broadcast bias addition before the requantization

Parameters

Parameters of arm_nn_mat_mult_nt_interleaved_t_even_s4

Name	Type	Direction	Description
lhs	const int8_t *	in	Pointer to the LHS input matrix
rhs	const int8_t *	in	Pointer to the RHS input matrix
bias	const int32_t *	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
dst	int8_t *	out	Pointer to the output matrix with "m" rows and "n" columns
dst_multipliers	const int32_t *	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
dst_shifts	const int32_t *	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
lhs_rows	const int32_t	in	Number of LHS input rows
rhs_rows	const int32_t	in	Number of RHS input rows
rhs_cols	const int32_t	in	Number of LHS/RHS input columns. Note this must be even.
lhs_offset	const int32_t	in	Offset to be applied to the LHS input value
dst_offset	const int32_t	in	Offset to be applied the output result
activation_min	const int32_t	in	Minimum value to clamp down the output. Range : int8
activation_max	const int32_t	in	Maximum value to clamp up the output. Range : int8
lhs_cols_offset	const int32_t	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_interleaved_t_even_s4

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_mat_mult_nt_t_s8

C

General Matrix-multiplication function with per-channel requantization.

Include/arm_nnsupportfunctions.h:1046

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s8(
    const int32_t *weight_sum_buf,
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t row_address_offset,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. This function assumes:

• LHS input matrix NOT transposed (nt)
• RHS input matrix transposed (t)

Note

This operation also performs the broadcast bias addition before the requantization

Parameters

Parameters of arm_nn_mat_mult_nt_t_s8

Name	Type	Direction	Description
weight_sum_buf	const int32_t *	in	Pointer to the weight sum multiplied by lhs_offset and summed bias buffer
lhs	const int8_t *	in	Pointer to the LHS input matrix
rhs	const int8_t *	in	Pointer to the RHS input matrix
bias	const int32_t *	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
dst	int8_t *	out	Pointer to the output matrix with "m" rows and "n" columns
dst_multipliers	const int32_t *	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
dst_shifts	const int32_t *	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
lhs_rows	const int32_t	in	Number of LHS input rows
rhs_rows	const int32_t	in	Number of RHS input rows
rhs_cols	const int32_t	in	Number of LHS/RHS input columns
lhs_offset	const int32_t	in	Offset to be applied to the LHS input value
dst_offset	const int32_t	in	Offset to be applied the output result
activation_min	const int32_t	in	Minimum value to clamp down the output. Range : int8
activation_max	const int32_t	in	Maximum value to clamp up the output. Range : int8
row_address_offset	const int32_t	in	Address offset between rows in output. NOTE: Only used for MVEI extension.
lhs_cols_offset	const int32_t	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_t_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_mat_mult_nt_t_1x1_out_s8

C

General Matrix-multiplication function with per-channel requantization.

Include/arm_nnsupportfunctions.h:1097

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_1x1_out_s8(
    const int32_t *weight_sum_buf,
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t row_address_offset,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. Output is calculated with multiple channels in parallel, rather than multiple output indices in a single channel This function assumes:

• LHS input matrix NOT transposed (nt)
• RHS input matrix transposed (t)

Note

This operation also performs the broadcast bias addition before the requantization

Parameters

Parameters of arm_nn_mat_mult_nt_t_1x1_out_s8

Name	Type	Direction	Description
weight_sum_buf	const int32_t *	in	Pointer to the weight sum multiplied by lhs_offset and summed bias buffer
lhs	const int8_t *	in	Pointer to the LHS input matrix
rhs	const int8_t *	in	Pointer to the RHS input matrix
bias	const int32_t *	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
dst	int8_t *	out	Pointer to the output matrix with "m" rows and "n" columns
dst_multipliers	const int32_t *	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
dst_shifts	const int32_t *	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
lhs_rows	const int32_t	in	Number of LHS input rows
rhs_rows	const int32_t	in	Number of RHS input rows
rhs_cols	const int32_t	in	Number of LHS/RHS input columns
lhs_offset	const int32_t	in	Offset to be applied to the LHS input value
dst_offset	const int32_t	in	Offset to be applied the output result
activation_min	const int32_t	in	Minimum value to clamp down the output. Range : int8
activation_max	const int32_t	in	Maximum value to clamp up the output. Range : int8
row_address_offset	const int32_t	in	Address offset between rows in output. NOTE: Only used for MVEI extension.
lhs_cols_offset	const int32_t	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_t_1x1_out_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_mat_mult_nt_t_s16

C

General Matrix-multiplication function with per-channel requantization and int16 input (LHS) and output.

Include/arm_nnsupportfunctions.h:1155

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const cmsis_nn_bias_data *bias_data,
    int16_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t row_address_offset
)

General Matrix-multiplication function with per-channel requantization and int16 input (LHS) and output. This function assumes:

• LHS input matrix NOT transposed (nt)
• RHS input matrix transposed (t)

Note

This operation also performs the broadcast bias addition before the requantization

MVE implementation only.

Parameters

Parameters of arm_nn_mat_mult_nt_t_s16

Name	Type	Direction	Description
lhs	const int16_t *	in	Pointer to the LHS input matrix
rhs	const int8_t *	in	Pointer to the RHS input matrix
bias_data	const cmsis_nn_bias_data *	in	Pointer to struct with bias vector. The length of this vector is equal to the number of output columns (or RHS input rows). The vector can be int32 or int64 indicated by a flag in the struct.
dst	int16_t *	out	Pointer to the output matrix with "m" rows and "n" columns
dst_multipliers	const int32_t *	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
dst_shifts	const int32_t *	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
lhs_rows	const int32_t	in	Number of LHS input rows
rhs_rows	const int32_t	in	Number of RHS input rows
rhs_cols	const int32_t	in	Number of LHS/RHS input columns
activation_min	const int32_t	in	Minimum value to clamp down the output. Range : int16
activation_max	const int32_t	in	Maximum value to clamp up the output. Range : int16
row_address_offset	const int32_t	in	Address offset between rows in output. NOTE: Only used for MVEI extension.

Returns

Returns of arm_nn_mat_mult_nt_t_s16

Description
The function returns `ARM_CMSIS_NN_SUCCESS` or `ARM_CMSIS_NN_NO_IMPL_ERROR` if not for MVE |---row_address_offset---| |____rhs_rows__________________| | | | | --- | --- | | | | | | | | | | lhs_rows | | | | --- | --- | | _______________ | ______________ |

function

arm_nn_mat_mult_nt_t_s8_s32

C

General Matrix-multiplication function with int8 input and int32 output.

Include/arm_nnsupportfunctions.h:1189

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s8_s32(
    const int8_t *lhs,
    const int8_t *rhs,
    int32_t *dst,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_idx_offset
)

General Matrix-multiplication function with int8 input and int32 output. This function assumes:

• LHS input matrix NOT transposed (nt)
• RHS input matrix transposed (t)

Note

Dst/output buffer must be zeroed out before calling this function.

Parameters

Parameters of arm_nn_mat_mult_nt_t_s8_s32

Name	Type	Direction	Description
lhs	const int8_t *	in	Pointer to the LHS input matrix
rhs	const int8_t *	in	Pointer to the RHS input matrix
dst	int32_t *	in, out	Pointer to the output matrix with "m" rows and "n" columns. Accumulated into, so it must be zeroed by the caller before the call
lhs_rows	const int32_t	in	Number of LHS input rows
rhs_rows	const int32_t	in	Number of LHS input columns/RHS input rows
rhs_cols	const int32_t	in	Number of RHS input columns
lhs_offset	const int32_t	in	Offset to be applied to the LHS input value
dst_idx_offset	const int32_t	in	Offset between subsequent output results

Returns

Returns of arm_nn_mat_mult_nt_t_s8_s32

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_s4

C

s4 Vector by Matrix (transposed) multiplication

Include/arm_nnsupportfunctions.h:1218

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s4(
    const int8_t *lhs,
    const int8_t *packed_rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s4 Vector by Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s4

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side vector
packed_rhs	const int8_t *	in	Input right-hand side matrix (transposed)
bias	const int32_t *	in	Input bias
dst	int8_t *	out	Output vector
lhs_offset	const int32_t	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
dst_offset	const int32_t	in	Offset to be added to the output values. Range: -127 to 128
dst_multiplier	const int32_t	in	Output multiplier
dst_shift	const int32_t	in	Output shift
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8

Returns

Returns of arm_nn_vec_mat_mult_t_s4

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_s8

C

s8 Vector by Matrix (transposed) multiplication

Include/arm_nnsupportfunctions.h:1256

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *kernel_sum,
    const int32_t *bias,
    int8_t *dst,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t address_offset,
    const int32_t rhs_offset
)

s8 Vector by Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s8

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side vector
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
kernel_sum	const int32_t *	in	Kernel sums of the kernels (rhs). See arm_vector_sum_s8 for more info.
bias	const int32_t *	in	Input bias
dst	int8_t *	out	Output vector
lhs_offset	const int32_t	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
dst_offset	const int32_t	in	Offset to be added to the output values. Range: -127 to 128
dst_multiplier	const int32_t	in	Output multiplier
dst_shift	const int32_t	in	Output shift
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8
address_offset	const int32_t	in	Memory position offset for dst. First output is stored at 'dst', the second at 'dst + address_offset' and so on. Default value is typically 1.
rhs_offset	const int32_t	in	Offset to be added to the input values of the right-hand side vector. Range: -127 to 128

Returns

Returns of arm_nn_vec_mat_mult_t_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_per_ch_s8

C

s8 Vector by Matrix (transposed) multiplication using per channel quantization for output

Include/arm_nnsupportfunctions.h:1297

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_per_ch_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *kernel_sum,
    const int32_t *bias,
    int8_t *dst,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t *dst_multiplier,
    const int32_t *dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t address_offset,
    const int32_t rhs_offset
)

s8 Vector by Matrix (transposed) multiplication using per channel quantization for output

Parameters

Parameters of arm_nn_vec_mat_mult_t_per_ch_s8

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side vector
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
kernel_sum	const int32_t *	in	Kernel sums of the kernels (rhs). See arm_vector_sum_s8 for more info.
bias	const int32_t *	in	Input bias
dst	int8_t *	out	Output vector
lhs_offset	const int32_t	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
dst_offset	const int32_t	in	Offset to be added to the output values. Range: -127 to 128
dst_multiplier	const int32_t *	in	Output multipliers
dst_shift	const int32_t *	in	Output shifts
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8
address_offset	const int32_t	in	Memory position offset for dst. First output is stored at 'dst', the second at 'dst + address_offset' and so on. Default value is typically 1.
rhs_offset	const int32_t	in	Offset to be added to the input values of the right-hand side vector. Range: -127 to 128

Returns

Returns of arm_nn_vec_mat_mult_t_per_ch_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_s16

C

s16 Vector by s8 Matrix (transposed) multiplication

Include/arm_nnsupportfunctions.h:1330

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const int64_t *bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s16 Vector by s8 Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s16

Name	Type	Direction	Description
lhs	const int16_t *	in	Input left-hand side vector
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
bias	const int64_t *	in	Input bias
dst	int16_t *	out	Output vector
dst_multiplier	const int32_t	in	Output multiplier
dst_shift	const int32_t	in	Output shift
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int16
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_s16

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_per_ch_s16

C

s16 vector(lhs) by s8 matrix (transposed) multiplication and per channel quant output

Include/arm_nnsupportfunctions.h:1358

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_per_ch_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const int64_t *bias,
    int16_t *dst,
    const int32_t *dst_multiplier,
    const int32_t *dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s16 vector(lhs) by s8 matrix (transposed) multiplication and per channel quant output

Parameters

Parameters of arm_nn_vec_mat_mult_t_per_ch_s16

Name	Type	Direction	Description
lhs	const int16_t *	in	Input left-hand side vector
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
bias	const int64_t *	in	Input bias
dst	int16_t *	out	Output vector
dst_multiplier	const int32_t *	in	Per channel output multiplier. Length of vector is equal to rhs_rows
dst_shift	const int32_t *	in	Per channel output shift. Length of vector is equal to rhs_rows
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int16
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_per_ch_s16

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_s16_s16

C

s16 Vector by s16 Matrix (transposed) multiplication

Include/arm_nnsupportfunctions.h:1386

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s16_s16(
    const int16_t *lhs,
    const int16_t *rhs,
    const int64_t *bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s16 Vector by s16 Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s16_s16

Name	Type	Direction	Description
lhs	const int16_t *	in	Input left-hand side vector
rhs	const int16_t *	in	Input right-hand side matrix (transposed)
bias	const int64_t *	in	Input bias
dst	int16_t *	out	Output vector
dst_multiplier	const int32_t	in	Output multiplier
dst_shift	const int32_t	in	Output shift
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int16
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_s16_s16

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mult_t_svdf_s8

C

s8 Vector by Matrix (transposed) multiplication with s16 output

Include/arm_nnsupportfunctions.h:1417

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_svdf_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    int16_t *dst,
    const int32_t lhs_offset,
    const int32_t scatter_offset,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s8 Vector by Matrix (transposed) multiplication with s16 output

Parameters

Parameters of arm_nn_vec_mat_mult_t_svdf_s8

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side vector
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
dst	int16_t *	out	Output vector
lhs_offset	const int32_t	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
scatter_offset	const int32_t	in	Address offset for dst. First output is stored at 'dst', the second at 'dst + scatter_offset' and so on.
dst_multiplier	const int32_t	in	Output multiplier
dst_shift	const int32_t	in	Output shift
rhs_cols	const int32_t	in	Number of columns in the right-hand side input matrix
rhs_rows	const int32_t	in	Number of rows in the right-hand side input matrix
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int16
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_svdf_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_depthwise_conv_nt_t_padded_s8

C

Depthwise convolution of transposed rhs matrix with 4 lhs matrices.

Include/arm_nnsupportfunctions.h:1453

arm_cmsis_nn_status arm_nn_depthwise_conv_nt_t_padded_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t lhs_offset,
    const int32_t active_ch,
    const int32_t total_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in padded cases where the padding is -lhs_offset(Range: int8). Dimensions are the same for lhs and rhs.

Note

Tail channel loads and stores are predicated, so channel-indexed arrays are not accessed beyond active_ch.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_padded_s8

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side matrix
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
lhs_offset	const int32_t	in	LHS matrix offset(input offset). Range: -127 to 128
active_ch	const int32_t	in	Subset of total_ch processed
total_ch	const int32_t	in	Number of channels in LHS/RHS
out_shift	const int32_t *	in	Per channel output shift. Length of vector is equal to number of channels
out_mult	const int32_t *	in	Per channel output multiplier. Length of vector is equal to number of channels
out_offset	const int32_t	in	Offset to be added to the output values. Range: -127 to 128
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8
row_x_col	const uint16_t	in	(row_dimension * col_dimension) of LHS/RHS matrix
output_bias	const int32_t *const	in	Per channel output bias. Length of vector is equal to number of channels
out	int8_t *	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_padded_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS` if an implementation is available or `ARM_CMSIS_NN_NO_IMPL_ERROR` otherwise

function

arm_nn_depthwise_conv_nt_t_s8

C

Depthwise convolution of transposed rhs matrix with 4 lhs matrices.

Include/arm_nnsupportfunctions.h:1492

arm_cmsis_nn_status arm_nn_depthwise_conv_nt_t_s8(
    const int32_t *weight_sum_buf,
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t lhs_offset,
    const int32_t active_ch,
    const int32_t total_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in non-padded cases. Dimensions are the same for lhs and rhs.

Note

Tail channel loads and stores are predicated, so channel-indexed arrays are not accessed beyond active_ch.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_s8

Name	Type	Direction	Description
weight_sum_buf	const int32_t *	in	Pointer to the weight sum multiplied by lhs_offset and summed bias buffer
lhs	const int8_t *	in	Input left-hand side matrix
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
lhs_offset	const int32_t	in	LHS matrix offset(input offset). Range: -127 to 128
active_ch	const int32_t	in	Subset of total_ch processed
total_ch	const int32_t	in	Number of channels in LHS/RHS
out_shift	const int32_t *	in	Per channel output shift. Length of vector is equal to number of channels.
out_mult	const int32_t *	in	Per channel output multiplier. Length of vector is equal to number of channels.
out_offset	const int32_t	in	Offset to be added to the output values. Range: -127 to 128
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8
row_x_col	const uint16_t	in	(row_dimension * col_dimension) of LHS/RHS matrix
output_bias	const int32_t *const	in	Per channel output bias. Length of vector is equal to number of channels.
out	int8_t *	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_s8

Description
The function returns `ARM_CMSIS_NN_SUCCESS` if an implementation is available or `ARM_CMSIS_NN_NO_IMPL_ERROR` otherwise

function

arm_nn_depthwise_conv_s8_planar_candidate

C

Necessary conditions of the planar rule that are cheap to test inline: at most 32 channels and stride 1.

Include/arm_nnsupportfunctions.h:1517

static int32_t arm_nn_depthwise_conv_s8_planar_candidate(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims
)

Necessary conditions of the planar rule that are cheap to test inline: at most 32 channels and stride 1. A caller can skip arm_nn_depthwise_conv_s8_planar() for layers that fail them without changing which layers it takes.

Parameters

Parameters of arm_nn_depthwise_conv_s8_planar_candidate

Name	Type	Direction	Description
dw_conv_params	const cmsis_nn_dw_conv_params *	in	Depthwise convolution parameters
input_dims	const cmsis_nn_dims *	in	Input tensor dimensions. Format: [1, H, W, C_IN]

Returns

Returns of arm_nn_depthwise_conv_s8_planar_candidate

Description
1 when the layer may take the planar path, 0 when it cannot.

function

arm_nn_is_convolve_s8_small_cin

C

The gate of armconvolves8smallcin(): upscaledims NULL, input depth 1 to 3 with filter depth equal to it, dilation 1, a kernel of at least 1x1 with kernel width…

Include/arm_nnsupportfunctions.h:1536

static int32_t arm_nn_is_convolve_s8_small_cin(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims,
    const cmsis_nn_dims *upscale_dims
)

The gate of arm_convolve_s8_small_cin(): upscale_dims NULL, input depth 1 to 3 with filter depth equal to it, dilation 1, a kernel of at least 1x1 with kernel width x depth at most 16 and at most 48 values, and a positive multiple of 4 output channels. Plain C; it evaluates the same on every build.

Parameters

Parameters of arm_nn_is_convolve_s8_small_cin

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters
input_dims	const cmsis_nn_dims *	in	Input tensor dimensions. Format: [N, H, W, C_IN]
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
output_dims	const cmsis_nn_dims *	in	Output tensor dimensions. Format: [N, H, W, C_OUT]
upscale_dims	const cmsis_nn_dims *	in	Upscale tensor dimensions, or NULL

Returns

Returns of arm_nn_is_convolve_s8_small_cin

Description
1 when the layer is in the gate, 0 otherwise.

function

arm_nn_is_convolve_s8_3x3_c16_s1

C

The gate of armconvolves83x3c16s1(): upscaledims NULL, input and filter depth 16, a 3x3 kernel, and stride and dilation 1.

Include/arm_nnsupportfunctions.h:1562

static int32_t arm_nn_is_convolve_s8_3x3_c16_s1(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *upscale_dims
)

The gate of arm_convolve_s8_3x3_c16_s1(): upscale_dims NULL, input and filter depth 16, a 3x3 kernel, and stride and dilation 1. Plain C; it evaluates the same on every build.

Parameters

Parameters of arm_nn_is_convolve_s8_3x3_c16_s1

Name	Type	Direction	Description
conv_params	const cmsis_nn_conv_params *	in	Convolution parameters
input_dims	const cmsis_nn_dims *	in	Input tensor dimensions. Format: [N, H, W, C_IN]
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
upscale_dims	const cmsis_nn_dims *	in	Upscale tensor dimensions, or NULL

Returns

Returns of arm_nn_is_convolve_s8_3x3_c16_s1

Description
1 when the layer is in the gate, 0 otherwise.

function

arm_nn_convolve_s8_groups_invalid

C

The group check of armconvolves8(), for its direct entries: with groups = CIN / filter C, CIN or COUT is not a multiple of groups.

Include/arm_nnsupportfunctions.h:1582

static int32_t arm_nn_convolve_s8_groups_invalid(
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

The group check of arm_convolve_s8(), for its direct entries: with groups = C_IN / filter C, C_IN or C_OUT is not a multiple of groups. A filter C of zero or above C_IN gives no group count and is not reported.

Parameters

Parameters of arm_nn_convolve_s8_groups_invalid

Name	Type	Direction	Description
input_dims	const cmsis_nn_dims *	in	Input tensor dimensions. Format: [N, H, W, C_IN]
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
output_dims	const cmsis_nn_dims *	in	Output tensor dimensions. Format: [N, H, W, C_OUT]

Returns

Returns of arm_nn_convolve_s8_groups_invalid

Description
1 when `arm_convolve_s8()` reports the group count as an argument error, 0 otherwise.

function

arm_nn_depthwise_conv_s8_planar_bytes

C

Plane size in bytes that armnndepthwiseconvs8planar() needs for a layer, or -1 when the layer is not one it takes.

Include/arm_nnsupportfunctions.h:1601

int32_t arm_nn_depthwise_conv_s8_planar_bytes(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Plane size in bytes that arm_nn_depthwise_conv_s8_planar() needs for a layer, or -1 when the layer is not one it takes. The rule is plain C and evaluates the same on every build.

Parameters

Parameters of arm_nn_depthwise_conv_s8_planar_bytes

Name	Type	Direction	Description
dw_conv_params	const cmsis_nn_dw_conv_params *	in	Depthwise convolution parameters
input_dims	const cmsis_nn_dims *	in	Input tensor dimensions. Format: [1, H, W, C_IN]
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]
output_dims	const cmsis_nn_dims *	in	Output tensor dimensions. Format: [1, H, W, C_OUT]

Returns

Returns of arm_nn_depthwise_conv_s8_planar_bytes

Description
The plane size in bytes, or -1.

function

arm_nn_depthwise_conv_s8_planar

C

s8 depthwise convolution with channel multiplier 1 and stride 1, vectorized across the output pixels of one channel plane instead of across channels.

Include/arm_nnsupportfunctions.h:1626

arm_cmsis_nn_status arm_nn_depthwise_conv_s8_planar(
    const cmsis_nn_context *ctx,
    const cmsis_nn_context *weight_sum_ctx,
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const cmsis_nn_dims *input_dims,
    const int8_t *input,
    const cmsis_nn_dims *filter_dims,
    const int8_t *kernel,
    const cmsis_nn_dims *output_dims,
    int8_t *output
)

s8 depthwise convolution with channel multiplier 1 and stride 1, vectorized across the output pixels of one channel plane instead of across channels. It serves the few-channel and 1xk layers of arm_depthwise_conv_s8_opt(), with the same scratch buffer and weight sums.

Parameters

Parameters of arm_nn_depthwise_conv_s8_planar

Name	Type	Direction	Description
ctx	const cmsis_nn_context *	in, out	Scratch buffer of `arm_depthwise_conv_s8_opt_get_buffer_size()` bytes
weight_sum_ctx	const cmsis_nn_context *	in	Per-channel weight sums from `arm_depthwise_convolve_weight_sum()`, bias included
dw_conv_params	const cmsis_nn_dw_conv_params *	in	Depthwise convolution parameters
quant_params	const cmsis_nn_per_channel_quant_params *	in	Per-channel quantization parameters
input_dims	const cmsis_nn_dims *	in	Input tensor dimensions. Format: [1, H, W, C_IN]
input	const int8_t *	in	Input data pointer
filter_dims	const cmsis_nn_dims *	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]
kernel	const int8_t *	in	Filter data pointer
output_dims	const cmsis_nn_dims *	in	Output tensor dimensions. Format: [1, H, W, C_OUT]
output	int8_t *	out	Output data pointer

Returns

Returns of arm_nn_depthwise_conv_s8_planar

Description
`ARM_CMSIS_NN_SUCCESS` when the layer was computed, or `ARM_CMSIS_NN_NO_IMPL_ERROR` when it is not one this path takes or its plane does not fit in ctx->size (then nothing is written), or MVE is not available.

function

arm_nn_depthwise_conv_nt_t_s4

C

Depthwise convolution of transposed rhs matrix with 4 lhs matrices.

Include/arm_nnsupportfunctions.h:1663

arm_cmsis_nn_status arm_nn_depthwise_conv_nt_t_s4(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t lhs_offset,
    const int32_t active_ch,
    const int32_t total_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in non-padded cases. rhs consists of packed int4 data. Dimensions are the same for lhs and rhs.

Note

Tail channel loads and stores are predicated, so channel-indexed arrays are not accessed beyond active_ch.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_s4

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side matrix
rhs	const int8_t *	in	Input right-hand side matrix (transposed). Consists of int4 data packed in an int8 buffer.
lhs_offset	const int32_t	in	LHS matrix offset(input offset). Range: -127 to 128
active_ch	const int32_t	in	Subset of total_ch processed
total_ch	const int32_t	in	Number of channels in LHS/RHS
out_shift	const int32_t *	in	Per channel output shift. Length of vector is equal to number of channels.
out_mult	const int32_t *	in	Per channel output multiplier. Length of vector is equal to number of channels.
out_offset	const int32_t	in	Offset to be added to the output values. Range: -127 to 128
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8
row_x_col	const uint16_t	in	(row_dimension * col_dimension) of LHS/RHS matrix
output_bias	const int32_t *const	in	Per channel output bias. Length of vector is equal to number of channels.
out	int8_t *	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_s4

Description
The function returns one of the two - Updated output pointer if an implementation is available - NULL if no implementation is available.

function

arm_nn_depthwise_conv_nt_t_s16

C

Depthwise convolution of transposed rhs matrix with 4 lhs matrices.

Include/arm_nnsupportfunctions.h:1699

int16_t * arm_nn_depthwise_conv_nt_t_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const uint16_t num_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int64_t *const output_bias,
    int16_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in non-padded cases. Dimensions are the same for lhs and rhs.

Note

Tail channel loads and stores are predicated, so channel-indexed arrays are not accessed beyond num_ch.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_s16

Name	Type	Direction	Description
lhs	const int16_t *	in	Input left-hand side matrix
rhs	const int8_t *	in	Input right-hand side matrix (transposed)
num_ch	const uint16_t	in	Number of channels in LHS/RHS
out_shift	const int32_t *	in	Per channel output shift. Length of vector is equal to number of channels.
out_mult	const int32_t *	in	Per channel output multiplier. Length of vector is equal to number of channels.
activation_min	const int32_t	in	Minimum value to clamp the output to. Range: int8
activation_max	const int32_t	in	Maximum value to clamp the output to. Range: int8
row_x_col	const uint16_t	in	(row_dimension * col_dimension) of LHS/RHS matrix
output_bias	const int64_t *const	in	Per channel output bias. Length of vector is equal to number of channels.
out	int16_t *	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_s16

Description
The function returns one of the two - Updated output pointer if an implementation is available - NULL if no implementation is available.

function

arm_nn_transpose_conv_row_s8_s32

C

Row of s8 scalars multiplicated with a s8 matrix ad accumulated into a s32 rolling scratch buffer.

Include/arm_nnsupportfunctions.h:1735

arm_cmsis_nn_status arm_nn_transpose_conv_row_s8_s32(
    const int8_t *lhs,
    const int8_t *rhs,
    int32_t *output_start,
    const int32_t output_index,
    const int32_t output_max,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t input_channels,
    const int32_t output_channels,
    const int32_t lhs_offset,
    const int32_t row_offset,
    const int32_t input_x,
    const int32_t stride_x,
    const int32_t skip_row_top,
    const int32_t skip_row_bottom
)

Row of s8 scalars multiplicated with a s8 matrix ad accumulated into a s32 rolling scratch buffer. Helpfunction for transposed convolution.

Note

Rolling buffer refers to how the function wraps around the scratch buffer, e.g. it starts writing at [output_start + output_index], writes to [output_start + output_max] and then continues at [output_start] again.

Parameters

Parameters of arm_nn_transpose_conv_row_s8_s32

Name	Type	Direction	Description
lhs	const int8_t *	in	Input left-hand side scalars
rhs	const int8_t *	in	Input right-hand side matrix
output_start	int32_t *	out	Output buffer start
output_index	const int32_t	in	Output buffer current index
output_max	const int32_t	in	Output buffer size
rhs_rows	const int32_t	in	Number of rows in rhs matrix
rhs_cols	const int32_t	in	Number of columns in rhs matrix
input_channels	const int32_t	in	Number of input channels
output_channels	const int32_t	in	Number of output channels
lhs_offset	const int32_t	in	Offset added to lhs before multiplication
row_offset	const int32_t	in	Address offset between each row of data output
input_x	const int32_t	in	Length of lhs scalar row.
stride_x	const int32_t	in	Address offset between each scalar-matrix multiplication result.
skip_row_top	const int32_t	in	Skip rows on top of the filter, used for padding.
skip_row_bottom	const int32_t	in	Skip rows in the bottom of the filter, used for padding.

Returns

Returns of arm_nn_transpose_conv_row_s8_s32

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_nn_read_q15x2_ia

C

Read 2 s16 elements and post increment pointer.

Include/arm_nnsupportfunctions.h:1756

static int32_t arm_nn_read_q15x2_ia(const int16_t **in_q15)

Read 2 s16 elements and post increment pointer.

Parameters

Parameters of arm_nn_read_q15x2_ia

Name	Type	Direction	Description
in_q15	const int16_t **	in, out	Pointer to pointer that holds address of input. Advanced past the elements read.

Returns

Returns of arm_nn_read_q15x2_ia

Description
q31 value

function

arm_nn_read_s8x4_ia

C

Read 4 s8 from s8 pointer and post increment pointer.

Include/arm_nnsupportfunctions.h:1771

static int32_t arm_nn_read_s8x4_ia(const int8_t **in_s8)

Read 4 s8 from s8 pointer and post increment pointer.

Parameters

Parameters of arm_nn_read_s8x4_ia

Name	Type	Direction	Description
in_s8	const int8_t **	in, out	Pointer to pointer that holds address of input. Advanced past the elements read.

Returns

Returns of arm_nn_read_s8x4_ia

Description
q31 value

function

arm_nn_read_s8x2_ia

C

Read 2 s8 from s8 pointer and post increment pointer.

Include/arm_nnsupportfunctions.h:1785

static int32_t arm_nn_read_s8x2_ia(const int8_t **in_s8)

Read 2 s8 from s8 pointer and post increment pointer.

Parameters

Parameters of arm_nn_read_s8x2_ia

Name	Type	Direction	Description
in_s8	const int8_t **	in, out	Pointer to pointer that holds address of input. Advanced past the elements read.

Returns

Returns of arm_nn_read_s8x2_ia

Description
q31 value

function

arm_nn_read_s16x2

C

Read 2 int16 values from int16 pointer.

Include/arm_nnsupportfunctions.h:1799

static int32_t arm_nn_read_s16x2(const int16_t *in)

Read 2 int16 values from int16 pointer.

Parameters

Parameters of arm_nn_read_s16x2

Name	Type	Direction	Description
in	const int16_t *	in	pointer to address of input.

Returns

Returns of arm_nn_read_s16x2

Description
s32 value

function

arm_nn_read_s8x4

C

Read 4 s8 values.

Include/arm_nnsupportfunctions.h:1812

static int32_t arm_nn_read_s8x4(const int8_t *in_s8)

Read 4 s8 values.

Parameters

Parameters of arm_nn_read_s8x4

Name	Type	Direction	Description
in_s8	const int8_t *	in	pointer to address of input.

Returns

Returns of arm_nn_read_s8x4

Description
s32 value

function

arm_nn_read_s8x2

C

Read 2 s8 values.

Include/arm_nnsupportfunctions.h:1824

static int32_t arm_nn_read_s8x2(const int8_t *in_s8)

Read 2 s8 values.

Parameters

Parameters of arm_nn_read_s8x2

Name	Type	Direction	Description
in_s8	const int8_t *	in	pointer to address of input.

Returns

Returns of arm_nn_read_s8x2

Description
s32 value

function

arm_nn_write_s8x4_ia

C

Write four s8 to s8 pointer and increment pointer afterwards.

Include/arm_nnsupportfunctions.h:1837

static void arm_nn_write_s8x4_ia(int8_t **in, int32_t value)

Write four s8 to s8 pointer and increment pointer afterwards.

Parameters

Parameters of arm_nn_write_s8x4_ia

Name	Type	Direction	Description
in	int8_t **	in, out	Double pointer to destination. Advanced past the bytes written.
value	int32_t	in	Four bytes to copy

function

arm_memset_s8

C

memset optimized for MVE

Include/arm_nnsupportfunctions.h:1850

static void arm_memset_s8(int8_t *dst, const int8_t val, uint32_t block_size)

memset optimized for MVE

Parameters

Parameters of arm_memset_s8

Name	Type	Direction	Description
dst	int8_t *	in, out	Destination pointer
val	const int8_t	in	Value to set
block_size	uint32_t	in	Number of bytes to copy.

function

arm_memset_s16

C

memset optimized for MVE for 16-bit data.

Include/arm_nnsupportfunctions.h:1873

static void arm_memset_s16(int16_t *dst, const int16_t val, uint32_t block_size)

memset optimized for MVE for 16-bit data.

Parameters

Parameters of arm_memset_s16

Name	Type	Direction	Description
dst	int16_t *	in, out	Destination pointer.
val	const int16_t	in	16-bit value to set.
block_size	uint32_t	in	Number of int16_t values to set.

function

arm_nn_mat_mult_kernel_s4_s16

C

Matrix-multiplication function for convolution with per-channel requantization and 4 bit weights.

Include/arm_nnsupportfunctions.h:2094

int8_t * arm_nn_mat_mult_kernel_s4_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const uint16_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t num_col_a,
    const int32_t *const output_bias,
    int8_t *out_0
)

Matrix-multiplication function for convolution with per-channel requantization and 4 bit weights.

This function does the matrix multiplication of weight matrix for all output channels with 2 columns from im2col and produces two elements/output_channel. The outputs are clamped in the range provided by activation min and max. Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_mat_mult_kernel_s4_s16

Name	Type	Direction	Description
input_a	const int8_t *	in	pointer to operand A, int8 packed with 2x int4.
input_b	const int16_t *	in	pointer to operand B, always consists of 2 vectors.
output_ch	const uint16_t	in	number of rows of A
out_shift	const int32_t *	in	pointer to per output channel requantization shift parameter.
out_mult	const int32_t *	in	pointer to per output channel requantization multiplier parameter.
out_offset	const int32_t	in	output tensor offset.
activation_min	const int32_t	in	minimum value to clamp the output to. Range : int8
activation_max	const int32_t	in	maximum value to clamp the output to. Range : int8
num_col_a	const int32_t	in	number of columns of A
output_bias	const int32_t *const	in	per output channel bias. Range : int32
out_0	int8_t *	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_kernel_s4_s16

Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

function

arm_nn_mat_mult_kernel_s8_s16

C

Matrix-multiplication function for convolution with per-channel requantization.

Include/arm_nnsupportfunctions.h:2128

int8_t * arm_nn_mat_mult_kernel_s8_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const uint16_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int16_t activation_min,
    const int16_t activation_max,
    const int32_t num_col_a,
    const int32_t aligned_num_col_a,
    const int32_t *const output_bias,
    int8_t *out_0
)

Matrix-multiplication function for convolution with per-channel requantization.

This function does the matrix multiplication of weight matrix for all output channels with 2 columns from im2col and produces two elements/output_channel. The outputs are clamped in the range provided by activation min and max. Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_mat_mult_kernel_s8_s16

Name	Type	Direction	Description
input_a	const int8_t *	in	pointer to operand A
input_b	const int16_t *	in	pointer to operand B, always consists of 2 vectors.
output_ch	const uint16_t	in	number of rows of A
out_shift	const int32_t *	in	pointer to per output channel requantization shift parameter.
out_mult	const int32_t *	in	pointer to per output channel requantization multiplier parameter.
out_offset	const int32_t	in	output tensor offset.
activation_min	const int16_t	in	minimum value to clamp the output to. Range : int8
activation_max	const int16_t	in	maximum value to clamp the output to. Range : int8
num_col_a	const int32_t	in	number of columns of A
aligned_num_col_a	const int32_t	in	number of columns of A aligned by 4
output_bias	const int32_t *const	in	per output channel bias. Range : int32
out_0	int8_t *	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_kernel_s8_s16

Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

function

arm_nn_mat_mult_kernel_row_offset_s8_s16

C

Matrix-multiplication function for convolution with per-channel requantization, supporting an address offset between rows.

Include/arm_nnsupportfunctions.h:2168

int8_t * arm_nn_mat_mult_kernel_row_offset_s8_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const uint16_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int16_t activation_min,
    const int16_t activation_max,
    const int32_t num_col_a,
    const int32_t aligned_num_col_a,
    const int32_t *const output_bias,
    const int32_t row_address_offset,
    int8_t *out_0
)

Matrix-multiplication function for convolution with per-channel requantization, supporting an address offset between rows.

This function does the matrix multiplication of weight matrix for all output channels with 2 columns from im2col and produces two elements/output_channel. The outputs are clamped in the range provided by activation min and max.

This function is slighly less performant than arm_nn_mat_mult_kernel_s8_s16, but allows support for grouped convolution. Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_mat_mult_kernel_row_offset_s8_s16

Name	Type	Direction	Description
input_a	const int8_t *	in	pointer to operand A
input_b	const int16_t *	in	pointer to operand B, always consists of 2 vectors.
output_ch	const uint16_t	in	number of rows of A
out_shift	const int32_t *	in	pointer to per output channel requantization shift parameter.
out_mult	const int32_t *	in	pointer to per output channel requantization multiplier parameter.
out_offset	const int32_t	in	output tensor offset.
activation_min	const int16_t	in	minimum value to clamp the output to. Range : int8
activation_max	const int16_t	in	maximum value to clamp the output to. Range : int8
num_col_a	const int32_t	in	number of columns of A
aligned_num_col_a	const int32_t	in	number of columns of A aligned by 4
output_bias	const int32_t *const	in	per output channel bias. Range : int32
row_address_offset	const int32_t	in	address offset between rows in the output
out_0	int8_t *	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_kernel_row_offset_s8_s16

Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

function

arm_nn_softmax_common_s8

C

Common softmax function for s8 input and s8 or s16 output.

Include/arm_nnsupportfunctions.h:2197

void arm_nn_softmax_common_s8(
    const int8_t *input,
    const int32_t num_rows,
    const int32_t row_size,
    const int32_t mult,
    const int32_t shift,
    const int32_t diff_min,
    const bool int16_output,
    void *output
)

Common softmax function for s8 input and s8 or s16 output.

Note

Supported framework: TensorFlow Lite micro (bit-accurate)

Parameters

Parameters of arm_nn_softmax_common_s8

Name	Type	Direction	Description
input	const int8_t *	in	Pointer to the input tensor
num_rows	const int32_t	in	Number of rows in the input tensor
row_size	const int32_t	in	Number of elements in each input row
mult	const int32_t	in	Input quantization multiplier
shift	const int32_t	in	Input quantization shift within the range [0, 31]
diff_min	const int32_t	in	Minimum difference with max in row. Used to check if the quantized exponential operation can be performed
int16_output	const bool	in	Indicating s8 output if 0 else s16 output
output	void *	out	Pointer to the output tensor

macro

NN_ROUND

C

macro for adding rounding offset

Include/arm_nnsupportfunctions.h:2210

#define NN_ROUND(out_shift) ((0x1 << out_shift) >> 1)

macro for adding rounding offset

Parameters

Parameters of NN_ROUND

Name	Description
out_shift

macro

MUL_SAT

C

Include/arm_nnsupportfunctions.h:2216

#define MUL_SAT(a, b) arm_nn_doubling_high_mult((a), (b))

Parameters

Parameters of MUL_SAT

Name	Description
a
b

macro

MUL_SAT_MVE

C

Include/arm_nnsupportfunctions.h:2217

#define MUL_SAT_MVE(a, b) arm_doubling_high_mult_mve_32x4((a), (b))

Parameters

Parameters of MUL_SAT_MVE

Name	Description
a
b

macro

MUL_POW2

C

Include/arm_nnsupportfunctions.h:2218

#define MUL_POW2(a, b) arm_nn_mult_by_power_of_two((a), (b))

Parameters

Parameters of MUL_POW2

Name	Description
a
b

macro

DIV_POW2

C

Include/arm_nnsupportfunctions.h:2220

#define DIV_POW2(a, b) arm_nn_divide_by_power_of_two((a), (b))

Parameters

Parameters of DIV_POW2

Name	Description
a
b

macro

DIV_POW2_MVE

C

Include/arm_nnsupportfunctions.h:2221

#define DIV_POW2_MVE(a, b) arm_divide_by_power_of_two_mve((a), (b))

Parameters

Parameters of DIV_POW2_MVE

Name	Description
a
b

macro

EXP_ON_NEG

C

Include/arm_nnsupportfunctions.h:2223

#define EXP_ON_NEG(x) arm_nn_exp_on_negative_values((x))

Parameters

Parameters of EXP_ON_NEG

Name	Description
x

macro

ONE_OVER1

C

Include/arm_nnsupportfunctions.h:2224

#define ONE_OVER1(x) arm_nn_one_over_one_plus_x_for_x_in_0_1((x))

Parameters

Parameters of ONE_OVER1

Name	Description
x

function

arm_nn_doubling_high_mult

C

Saturating doubling high multiply.

Include/arm_nnsupportfunctions.h:2234

static int32_t arm_nn_doubling_high_mult(const int32_t m1, const int32_t m2)

Saturating doubling high multiply. Result matches NEON instruction VQRDMULH.

Parameters

Parameters of arm_nn_doubling_high_mult

Name	Type	Direction	Description
m1	const int32_t	in	Multiplicand. Range: {NN_Q31_MIN, NN_Q31_MAX}
m2	const int32_t	in	Multiplier. Range: {NN_Q31_MIN, NN_Q31_MAX}

Returns

Returns of arm_nn_doubling_high_mult

Description
Result of multiplication.

function

arm_nn_doubling_high_mult_no_sat

C

Doubling high multiply without saturation.

Include/arm_nnsupportfunctions.h:2272

static int32_t arm_nn_doubling_high_mult_no_sat(int32_t m1, int32_t m2)

Doubling high multiply without saturation. This is intended for requantization where the scale is a positive integer.

Note

The result of this matches that of neon instruction VQRDMULH for m1 in range {NN_Q31_MIN, NN_Q31_MAX} and m2 in range {NN_Q31_MIN + 1, NN_Q31_MAX}. Saturation occurs when m1 equals m2 equals NN_Q31_MIN and that is not handled by this function.

Parameters

Parameters of arm_nn_doubling_high_mult_no_sat

Name	Type	Direction	Description
m1	int32_t	in	Multiplicand. Range: {NN_Q31_MIN, NN_Q31_MAX}
m2	int32_t	in	Multiplier Range: {NN_Q31_MIN, NN_Q31_MAX}

Returns

Returns of arm_nn_doubling_high_mult_no_sat

Description
Result of multiplication.

function

arm_nn_divide_by_power_of_two

C

Rounding divide by power of two.

Include/arm_nnsupportfunctions.h:2323

static int32_t arm_nn_divide_by_power_of_two(const int32_t dividend, const int32_t exponent)

Rounding divide by power of two.

Parameters

Parameters of arm_nn_divide_by_power_of_two

Name	Type	Direction	Description
dividend	const int32_t	in	- Dividend
exponent	const int32_t	in	- Divisor = power(2, exponent) Range: [0, 31]

Returns

Returns of arm_nn_divide_by_power_of_two

Description
Rounded result of division. Midpoint is rounded away from zero.

function

arm_nn_nonneg_divide_by_pot_s32

C

Rounding divide by power of two for non-negative values.

Include/arm_nnsupportfunctions.h:2378

static int32_t arm_nn_nonneg_divide_by_pot_s32(int32_t dividend, int32_t exponent)

Rounding divide by power of two for non-negative values.

Parameters

Parameters of arm_nn_nonneg_divide_by_pot_s32

Name	Type	Direction	Description
dividend	int32_t	in	- Dividend (assumed to be non-negative)
exponent	int32_t	in	- Divisor = power(2, exponent) Range: [0, 31]

Returns

Returns of arm_nn_nonneg_divide_by_pot_s32

Description
Rounded result of division. Midpoint is rounded away from zero.

function

arm_nn_requantize

C

Requantize a given value.

Include/arm_nnsupportfunctions.h:2416

static int32_t arm_nn_requantize(const int32_t val, const int32_t multiplier, const int32_t shift)

Requantize a given value.

Essentially returns (val * multiplier)/(2 ^ shift) with different rounding depending if CMSIS_NN_USE_SINGLE_ROUNDING is defined or not.

Parameters

Parameters of arm_nn_requantize

Name	Type	Direction	Description
val	const int32_t	in	Value to be requantized
multiplier	const int32_t	in	Multiplier. Range {NN_Q31_MIN + 1, Q32_MAX}
shift	const int32_t	in	Shift. Range: {-31, 30} Default branch: If shift is positive left shift 'val * multiplier' with shift If shift is negative right shift 'val * multiplier' with abs(shift) Single round branch: Input for total_shift in divide by '2 ^ total_shift'

Returns

Returns of arm_nn_requantize

Description
Default branch: Returns (val * multiplier) with rounding divided by (2 ^ shift) with rounding Single round branch: Returns (val * multiplier)/(2 ^ (31 - shift)) with rounding

function

arm_nn_requantize_s64

C

Requantize a given 64 bit value.

Include/arm_nnsupportfunctions.h:2453

static int32_t arm_nn_requantize_s64(const int64_t val, const int32_t reduced_multiplier, const int32_t shift)

Requantize a given 64 bit value.

Parameters

Parameters of arm_nn_requantize_s64

Name	Type	Direction	Description
val	const int64_t	in	Value to be requantized in the range {-(1<<47)} to {(1<<47) - 1}
reduced_multiplier	const int32_t	in	Reduced multiplier in the range {NN_Q31_MIN + 1, Q32_MAX} to {Q16_MIN + 1, Q16_MAX}
shift	const int32_t	in	Left or right shift for 'val * multiplier' in the range {-31} to {7}

Returns

Returns of arm_nn_requantize_s64

Description
Returns (val * multiplier)/(2 ^ shift)

function

arm_nn_sat_lshift_s16

C

Saturating left shift for int16t.

Include/arm_nnsupportfunctions.h:2471

static int16_t arm_nn_sat_lshift_s16(int16_t x, int shift)

Saturating left shift for int16_t.

Parameters

Parameters of arm_nn_sat_lshift_s16

Name	Type	Direction	Description
x	int16_t	in	value to be shifted
shift	int	in	Nonpositive values return x; positive values multiply by 2^shift with s16 saturation.

Returns

Returns of arm_nn_sat_lshift_s16

Description
shifted value

function

arm_nn_sqrdmulh_s16

C

Saturating Rounding Doubling High Mul (s16).

Include/arm_nnsupportfunctions.h:2490

static int16_t arm_nn_sqrdmulh_s16(int16_t a, int16_t b)

Saturating Rounding Doubling High Mul (s16).

Matches NEON SQRDMULH s16

Parameters

Parameters of arm_nn_sqrdmulh_s16

Name	Type	Direction	Description
a	int16_t	in	Multiplicand
b	int16_t	in	Multiplier

Returns

Returns of arm_nn_sqrdmulh_s16

Description
Result of multiplication.

function

arm_nn_sqdmulh_s16

C

Saturating Non-rounded Doubling High Mul (s16).

Include/arm_nnsupportfunctions.h:2510

static int16_t arm_nn_sqdmulh_s16(int16_t a, int16_t b)

Saturating Non-rounded Doubling High Mul (s16).

Matches NEON SQDMULH s16

Parameters

Parameters of arm_nn_sqdmulh_s16

Name	Type	Direction	Description
a	int16_t	in	Multiplicand
b	int16_t	in	Multiplier

Returns

Returns of arm_nn_sqdmulh_s16

Description
Result of multiplication.

function

arm_nn_divide_by_power_of_two_s16

C

Rounding divide by power of two (s16), midpoint away from zero.

Include/arm_nnsupportfunctions.h:2530

static int16_t arm_nn_divide_by_power_of_two_s16(int16_t x, int exponent)

Rounding divide by power of two (s16), midpoint away from zero.

Mirrors arm_nn_divide_by_power_of_two() semantics for s16.

Parameters

Parameters of arm_nn_divide_by_power_of_two_s16

Name	Type	Direction	Description
x	int16_t	in	Dividend
exponent	int	in	Divisor = power(2, exponent) Range: [0, 15]

Returns

Returns of arm_nn_divide_by_power_of_two_s16

Description
Rounded result of division. Midpoint is rounded away from zero.

function

arm_memcpy_s8

C

memcpy optimized for MVE

Include/arm_nnsupportfunctions.h:2545

static void arm_memcpy_s8(int8_t *dst, const int8_t *src, uint32_t block_size)

memcpy optimized for MVE

Parameters

Parameters of arm_memcpy_s8

Name	Type	Direction	Description
dst	int8_t *	in, out	Destination pointer
src	const int8_t *	in	Source pointer.
block_size	uint32_t	in	Number of bytes to copy.

function

arm_memcpy_s16

C

memcpy optimized for MVE

Include/arm_nnsupportfunctions.h:2569

static void arm_memcpy_s16(int16_t *dst, const int16_t *src, uint32_t block_size)

memcpy optimized for MVE

Parameters

Parameters of arm_memcpy_s16

Name	Type	Direction	Description
dst	int16_t *	in, out	Destination pointer
src	const int16_t *	in	Source pointer.
block_size	uint32_t	in	Number of values to copy.

function

arm_memcpy_s32

C

memcpy optimized for MVE

Include/arm_nnsupportfunctions.h:2581

static void arm_memcpy_s32(int32_t *dst, const int32_t *src, uint32_t block_size)

memcpy optimized for MVE

Parameters

Parameters of arm_memcpy_s32

Name	Type	Direction	Description
dst	int32_t *	in, out	Destination pointer
src	const int32_t *	in	Source pointer.
block_size	uint32_t	in	Number of values to copy.

function

arm_memcpy_q15

C

memcpy wrapper for int16

Include/arm_nnsupportfunctions.h:2593

static void arm_memcpy_q15(int16_t *dst, const int16_t *src, uint32_t block_size)

memcpy wrapper for int16

Parameters

Parameters of arm_memcpy_q15

Name	Type	Direction	Description
dst	int16_t *	in, out	Destination pointer
src	const int16_t *	in	Source pointer.
block_size	uint32_t	in	Number of bytes to copy.

function

arm_nn_exp_on_negative_values

C

Fixed-point exp() of a non-positive value.

Include/arm_nnsupportfunctions.h:2865

static int32_t arm_nn_exp_on_negative_values(int32_t val)

Fixed-point exp() of a non-positive value.

Parameters

Parameters of arm_nn_exp_on_negative_values

Name	Type	Direction	Description
val	int32_t	in	Input in Q5.26 fixed point. Must be less than or equal to 0

Returns

Returns of arm_nn_exp_on_negative_values

Description
exp(val) in Q0.31 fixed point. Returns NN_Q31_MAX when `val` is 0.

macro

SELECT_IF_NON_ZERO

C

Include/arm_nnsupportfunctions.h:2878

#define SELECT_IF_NON_ZERO(x) { \ mask = MASK_IF_NON_ZERO(remainder & (1 << shift++)); \ result = SELECT_USING_MASK(mask, MUL_SAT(result, x), result); \ }

Parameters

Parameters of SELECT_IF_NON_ZERO

Name	Description
x

function

arm_nn_mult_by_power_of_two

C

Saturating multiply by a power of two.

Include/arm_nnsupportfunctions.h:2905

static int32_t arm_nn_mult_by_power_of_two(const int32_t val, const int32_t exp)

Saturating multiply by a power of two.

Parameters

Parameters of arm_nn_mult_by_power_of_two

Name	Type	Direction	Description
val	const int32_t	in	Value to be multiplied
exp	const int32_t	in	Exponent. Multiplier = power(2, exp)

Returns

Returns of arm_nn_mult_by_power_of_two

Description
val * 2^exp saturated to the int32 range

function

arm_nn_one_over_one_plus_x_for_x_in_0_1

C

Fixed-point 1 / (1 + x) for x in [0, 1), computed with Newton-Raphson iterations.

Include/arm_nnsupportfunctions.h:2920

static int32_t arm_nn_one_over_one_plus_x_for_x_in_0_1(int32_t val)

Fixed-point 1 / (1 + x) for x in [0, 1), computed with Newton-Raphson iterations.

Parameters

Parameters of arm_nn_one_over_one_plus_x_for_x_in_0_1

Name	Type	Direction	Description
val	int32_t	in	x in Q0.31 fixed point. Range: [0, NN_Q31_MAX]

Returns

Returns of arm_nn_one_over_one_plus_x_for_x_in_0_1

Description
1 / (1 + x) in Q0.31 fixed point

function

arm_nn_write_q15x2_ia

C

Write 2 s16 elements and post increment pointer.

Include/arm_nnsupportfunctions.h:2942

static void arm_nn_write_q15x2_ia(int16_t **dest_q15, int32_t src_q31)

Write 2 s16 elements and post increment pointer.

Parameters

Parameters of arm_nn_write_q15x2_ia

Name	Type	Direction	Description
dest_q15	int16_t **	in, out	Pointer to pointer that holds address of destination. Advanced past the elements written.
src_q31	int32_t	in	Input value to be written.

function

arm_nn_write_s8x2_ia

C

Write 2 s8 elements and post increment pointer.

Include/arm_nnsupportfunctions.h:2955

static void arm_nn_write_s8x2_ia(int8_t **dst, int16_t src)

Write 2 s8 elements and post increment pointer.

Parameters

Parameters of arm_nn_write_s8x2_ia

Name	Type	Direction	Description
dst	int8_t **	in, out	Pointer to pointer that holds address of destination. Advanced past the elements written.
src	int16_t	in	Input value to be written.

function

arm_cmsis_nn_dim_at

C

Get dimension value at specific index.

Include/arm_nnsupportfunctions.h:2969

static int32_t arm_cmsis_nn_dim_at(const cmsis_nn_dims *dims, int32_t index)

Get dimension value at specific index.

Parameters

Parameters of arm_cmsis_nn_dim_at

Name	Type	Direction	Description
dims	const cmsis_nn_dims *	in	Pointer to `cmsis_nn_dims` structure
index	int32_t	in	Index of dimension to get

Returns

Returns of arm_cmsis_nn_dim_at

Description
Dimension value at specified index

function

arm_cmsis_nn_shape_product

C

Calculate the product of all dimensions in a shape array.

Include/arm_nnsupportfunctions.h:2994

static size_t arm_cmsis_nn_shape_product(const int32_t *shape, int32_t length)

Calculate the product of all dimensions in a shape array.

Parameters

Parameters of arm_cmsis_nn_shape_product

Name	Type	Direction	Description
shape	const int32_t *	in	Pointer to array containing shape dimensions
length	int32_t	in	Number of dimensions in the shape array

Returns

Returns of arm_cmsis_nn_shape_product

Description
Product of all dimensions

function

arm_nn_lstm_step_s8

C

Update LSTM function for an iteration step using s8 input and output, and s16 internally.

Include/arm_nnsupportfunctions.h:3022

arm_cmsis_nn_status arm_nn_lstm_step_s8(
    const int8_t *data_in,
    const int8_t *hidden_in,
    int8_t *hidden_out,
    const cmsis_nn_lstm_params *params,
    cmsis_nn_lstm_context *buffers,
    const int32_t batch_offset
)

Update LSTM function for an iteration step using s8 input and output, and s16 internally.

Parameters

Parameters of arm_nn_lstm_step_s8

Name	Type	Direction	Description
data_in	const int8_t *	in	Data input pointer
hidden_in	const int8_t *	in	Hidden state/ recurrent input pointer
hidden_out	int8_t *	out	Hidden state/ recurrent output pointer
params	const cmsis_nn_lstm_params *	in	Struct containg all information about the lstm operator, see arm_nn_types.
buffers	cmsis_nn_lstm_context *	in, out	Struct containg pointers to all temporary scratch buffers needed for the lstm operator, see arm_nn_types.
batch_offset	const int32_t	in	Number of timesteps between consecutive batches. E.g for params->timing_major = true, all batches for t=0 are stored sequentially, so batch offset = 1. For params->time major = false, all time steps are stored continously before the next batch, so batch offset = params->time_steps.

Returns

Returns of arm_nn_lstm_step_s8

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_nn_lstm_step_s16

C

Update LSTM function for an iteration step using s16 input and output, and s16 internally.

Include/arm_nnsupportfunctions.h:3046

arm_cmsis_nn_status arm_nn_lstm_step_s16(
    const int16_t *data_in,
    const int16_t *hidden_in,
    int16_t *hidden_out,
    const cmsis_nn_lstm_params *params,
    cmsis_nn_lstm_context *buffers,
    const int32_t batch_offset
)

Update LSTM function for an iteration step using s16 input and output, and s16 internally.

Parameters

Parameters of arm_nn_lstm_step_s16

Name	Type	Direction	Description
data_in	const int16_t *	in	Data input pointer
hidden_in	const int16_t *	in	Hidden state/ recurrent input pointer
hidden_out	int16_t *	out	Hidden state/ recurrent output pointer
params	const cmsis_nn_lstm_params *	in	Struct containg all information about the lstm operator, see arm_nn_types.
buffers	cmsis_nn_lstm_context *	in, out	Struct containg pointers to all temporary scratch buffers needed for the lstm operator, see arm_nn_types.
batch_offset	const int32_t	in	Number of timesteps between consecutive batches. E.g for params->timing_major = true, all batches for t=0 are stored sequentially, so batch offset = 1. For params->time major = false, all time steps are stored continously before the next batch, so batch offset = params->time_steps.

Returns

Returns of arm_nn_lstm_step_s16

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_nn_lstm_calculate_gate_s8_s16

C

Updates a LSTM gate for an iteration step of LSTM function, int8x816 version.

Include/arm_nnsupportfunctions.h:3067

arm_cmsis_nn_status arm_nn_lstm_calculate_gate_s8_s16(
    const int8_t *data_in,
    const int8_t *hidden_in,
    const cmsis_nn_lstm_gate *gate_data,
    const cmsis_nn_lstm_params *params,
    int16_t *output,
    const int32_t batch_offset
)

Updates a LSTM gate for an iteration step of LSTM function, int8x8_16 version.

Parameters

Parameters of arm_nn_lstm_calculate_gate_s8_s16

Name	Type	Direction	Description
data_in	const int8_t *	in	Data input pointer
hidden_in	const int8_t *	in	Hidden state/ recurrent input pointer
gate_data	const cmsis_nn_lstm_gate *	in	Struct containing all information about the gate caluclation, see arm_nn_types.
params	const cmsis_nn_lstm_params *	in	Struct containing all information about the lstm_operation, see arm_nn_types
output	int16_t *	out	Hidden state/ recurrent output pointer
batch_offset	const int32_t	in	Number of timesteps between consecutive batches, see arm_nn_lstm_step_s8.

Returns

Returns of arm_nn_lstm_calculate_gate_s8_s16

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_nn_lstm_calculate_gate_s16

C

Updates a LSTM gate for an iteration step of LSTM function, int16x816 version.

Include/arm_nnsupportfunctions.h:3088

arm_cmsis_nn_status arm_nn_lstm_calculate_gate_s16(
    const int16_t *data_in,
    const int16_t *hidden_in,
    const cmsis_nn_lstm_gate *gate_data,
    const cmsis_nn_lstm_params *params,
    int16_t *output,
    const int32_t batch_offset
)

Updates a LSTM gate for an iteration step of LSTM function, int16x8_16 version.

Parameters

Parameters of arm_nn_lstm_calculate_gate_s16

Name	Type	Direction	Description
data_in	const int16_t *	in	Data input pointer
hidden_in	const int16_t *	in	Hidden state/ recurrent input pointer
gate_data	const cmsis_nn_lstm_gate *	in	Struct containing all information about the gate caluclation, see arm_nn_types.
params	const cmsis_nn_lstm_params *	in	Struct containing all information about the lstm_operation, see arm_nn_types
output	int16_t *	out	Hidden state/ recurrent output pointer
batch_offset	const int32_t	in	Number of timesteps between consecutive batches, see arm_nn_lstm_step_s16.

Returns

Returns of arm_nn_lstm_calculate_gate_s16

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_nn_vec_mat_mul_result_acc_s8_s16

C

The result of the multiplication is accumulated to the passed result buffer.

Include/arm_nnsupportfunctions.h:3114

arm_cmsis_nn_status arm_nn_vec_mat_mul_result_acc_s8_s16(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *effective_bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t batches,
    const int32_t batch_offset
)

The result of the multiplication is accumulated to the passed result buffer. Multiplies a matrix by a “batched” vector (i.e. a matrix with a batch dimension composed by input vectors independent from each other).

Parameters

Parameters of arm_nn_vec_mat_mul_result_acc_s8_s16

Name	Type	Direction	Description
lhs	const int8_t *	in	Batched vector
rhs	const int8_t *	in	Weights - input matrix (H(Rows)xW(Columns))
effective_bias	const int32_t *	in	Bias + lhs_offset * kernel_sum term precalculated into a constant vector.
dst	int16_t *	out	Output
dst_multiplier	const int32_t	in	Multiplier for quantization
dst_shift	const int32_t	in	Shift for quantization
rhs_cols	const int32_t	in	Vector/matarix column length
rhs_rows	const int32_t	in	Row count of matrix
batches	const int32_t	in	Batch size
batch_offset	const int32_t	in	Number of timesteps between consecutive batches in input, see arm_nn_lstm_step_s8. Note that the output is always stored with sequential batches.

Returns

Returns of arm_nn_vec_mat_mul_result_acc_s8_s16

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_nn_vec_mat_mul_result_acc_s16

C

The result of the multiplication is accumulated to the passed result buffer.

Include/arm_nnsupportfunctions.h:3144

arm_cmsis_nn_status arm_nn_vec_mat_mul_result_acc_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const int64_t *effective_bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t batches,
    const int32_t batch_offset
)

The result of the multiplication is accumulated to the passed result buffer. Multiplies a matrix by a “batched” vector (i.e. a matrix with a batch dimension composed by input vectors independent from each other).

Parameters

Parameters of arm_nn_vec_mat_mul_result_acc_s16

Name	Type	Direction	Description
lhs	const int16_t *	in	Batched vector
rhs	const int8_t *	in	Weights - input matrix (H(Rows)xW(Columns))
effective_bias	const int64_t *	in	Bias + lhs_offset * kernel_sum term precalculated into a constant vector.
dst	int16_t *	out	Output
dst_multiplier	const int32_t	in	Multiplier for quantization
dst_shift	const int32_t	in	Shift for quantization
rhs_cols	const int32_t	in	Vector/matarix column length
rhs_rows	const int32_t	in	Row count of matrix
batches	const int32_t	in	Batch size
batch_offset	const int32_t	in	Number of timesteps between consecutive batches in input, see arm_nn_lstm_step_s16. Note that the output is always stored with sequential batches.

Returns

Returns of arm_nn_vec_mat_mul_result_acc_s16

Description
The function returns `ARM_CMSIS_NN_SUCCESS`

function

arm_elementwise_mul_s16_s8

C

s16 elementwise multiplication with s8 output

Include/arm_nnsupportfunctions.h:3171

arm_cmsis_nn_status arm_elementwise_mul_s16_s8(
    const int16_t *input_1_vect,
    const int16_t *input_2_vect,
    int8_t *output,
    const int32_t out_offset,
    const int32_t out_mult,
    const int32_t out_shift,
    const int32_t block_size,
    const int32_t batch_size,
    const int32_t batch_offset
)

s16 elementwise multiplication with s8 output

Supported framework: TensorFlow Lite micro

Parameters

Parameters of arm_elementwise_mul_s16_s8

Name	Type	Direction	Description
input_1_vect	const int16_t *	in	pointer to input vector 1
input_2_vect	const int16_t *	in	pointer to input vector 2
output	int8_t *	in, out	pointer to output vector
out_offset	const int32_t	in	output offset
out_mult	const int32_t	in	output multiplier
out_shift	const int32_t	in	output shift
block_size	const int32_t	in	number of samples per batch
batch_size	const int32_t	in	number of samples per batch
batch_offset	const int32_t	in	Number of timesteps between consecutive batches in output, see arm_nn_lstm_step_s8. Note that it is assumed that the input is stored with sequential batches.

Returns

Returns of arm_elementwise_mul_s16_s8

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_elementwise_mul_s16_batch_offset

C

s16 elementwise multiplication with s16 output

Include/arm_nnsupportfunctions.h:3197

arm_cmsis_nn_status arm_elementwise_mul_s16_batch_offset(
    const int16_t *input_1_vect,
    const int16_t *input_2_vect,
    int16_t *output,
    const int32_t out_offset,
    const int32_t out_mult,
    const int32_t out_shift,
    const int32_t block_size,
    const int32_t batch_size,
    const int32_t batch_offset
)

s16 elementwise multiplication with s16 output

Supported framework: TensorFlow Lite micro

Parameters

Parameters of arm_elementwise_mul_s16_batch_offset

Name	Type	Direction	Description
input_1_vect	const int16_t *	in	pointer to input vector 1
input_2_vect	const int16_t *	in	pointer to input vector 2
output	int16_t *	in, out	pointer to output vector
out_offset	const int32_t	in	output offset
out_mult	const int32_t	in	output multiplier
out_shift	const int32_t	in	output shift
block_size	const int32_t	in	number of samples per batch
batch_size	const int32_t	in	number of samples per batch
batch_offset	const int32_t	in	Number of timesteps between consecutive batches in output, see arm_nn_lstm_step_s16. Note that it is assumed that the input is stored with sequential batches.

Returns

Returns of arm_elementwise_mul_s16_batch_offset

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_elementwise_mul_acc_s16

C

s16 elementwise multiplication.

Include/arm_nnsupportfunctions.h:3224

arm_cmsis_nn_status arm_elementwise_mul_acc_s16(
    const int16_t *input_1_vect,
    const int16_t *input_2_vect,
    const int32_t input_1_offset,
    const int32_t input_2_offset,
    int16_t *output,
    const int32_t out_offset,
    const int32_t out_mult,
    const int32_t out_shift,
    const int32_t out_activation_min,
    const int32_t out_activation_max,
    const int32_t block_size
)

s16 elementwise multiplication. The result of the multiplication is accumulated to the passed result buffer.

Supported framework: TensorFlow Lite micro

Parameters

Parameters of arm_elementwise_mul_acc_s16

Name	Type	Direction	Description
input_1_vect	const int16_t *	in	pointer to input vector 1
input_2_vect	const int16_t *	in	pointer to input vector 2
input_1_offset	const int32_t	in	offset for input 1. Not used.
input_2_offset	const int32_t	in	offset for input 2. Not used.
output	int16_t *	in, out	pointer to output vector
out_offset	const int32_t	in	output offset. Not used.
out_mult	const int32_t	in	output multiplier
out_shift	const int32_t	in	output shift
out_activation_min	const int32_t	in	minimum value to clamp output to. Min: -32768
out_activation_max	const int32_t	in	maximum value to clamp output to. Max: 32767
block_size	const int32_t	in	number of samples

Returns

Returns of arm_elementwise_mul_acc_s16

Description
The function returns ARM_CMSIS_NN_SUCCESS

function

arm_check_broadcast_required

C

Check if a broadcast is required between 2 cmsisnndims.

Include/arm_nnsupportfunctions.h:3245

static int32_t arm_check_broadcast_required(const cmsis_nn_dims *shape_1, const cmsis_nn_dims *shape_2)

Check if a broadcast is required between 2 cmsis_nn_dims.

Compares each dimension and returns 1 if any dimension does not match. This function does not check that broadcast rules are met.

Parameters

Parameters of arm_check_broadcast_required

Name	Type	Direction	Description
shape_1	const cmsis_nn_dims *	in	pointer to input tensor 1
shape_2	const cmsis_nn_dims *	in	pointer to input tensor 2

Returns

Returns of arm_check_broadcast_required

Description
The function returns 1 if a broadcast is required, or 0 if not.

function

arm_reduce_get_middle_block_from_arrays

C

Reports whether the reduced axes of a 4-D tensor form one contiguous block followed by kept axes, as in a NHWC mean over H and W, and gives the flattened sizes.

Include/arm_nnsupportfunctions.h:3310

static int32_t arm_reduce_get_middle_block_from_arrays(
    const int32_t in_dims,
    const int32_t axis_arr,
    int32_t *outer,
    int32_t *reduce,
    int32_t *inner
)

Reports whether the reduced axes of a 4-D tensor form one contiguous block followed by kept axes, as in a NHWC mean over H and W, and gives the flattened sizes. Axes of size 1 are ignored.

Parameters

Parameters of arm_reduce_get_middle_block_from_arrays

Name	Type	Direction	Description
in_dims	const int32_t	in	4-element array {n, h, w, c}
axis_arr	const int32_t	in	4-element mask {axis_n, axis_h, axis_w, axis_c}
outer	int32_t *	out	Product of the dims before the reduced block
reduce	int32_t *	out	Product of the reduced dims
inner	int32_t *	out	Product of the dims after the reduced block

Returns

Returns of arm_reduce_get_middle_block_from_arrays

Description
1 if the input is [outer, reduce, inner] with the middle dim reduced and inner > 1, otherwise 0

macro

ARM_NN_SQRT_S16_TABLEFREE_SHIFT

C

Include/arm_nnsupportfunctions.h:3364

#define ARM_NN_SQRT_S16_TABLEFREE_SHIFT 14

macro

ARM_NN_SQRT_S16_TABLEFREE_MAGIC

C

Include/arm_nnsupportfunctions.h:3365

#define ARM_NN_SQRT_S16_TABLEFREE_MAGIC UINT32_C(0x5F5FB6C4)

macro

ARM_NN_SQRT_S16_TABLEFREE_K0

C

Include/arm_nnsupportfunctions.h:3366

#define ARM_NN_SQRT_S16_TABLEFREE_K0 (-4.76426697f)

macro

ARM_NN_SQRT_S16_TABLEFREE_K1

C

Include/arm_nnsupportfunctions.h:3367

#define ARM_NN_SQRT_S16_TABLEFREE_K1 (-48.0000114f)

function

arm_nn_sqrt_s16_tablefree_element

C

One element of armsqrts16tablefree(): the float32 chain the MVE path evaluates per lane, so the two agree bit for bit on any IEEE-754 float32 implementation wi…

Include/arm_nnsupportfunctions.h:3381

static int16_t arm_nn_sqrt_s16_tablefree_element(const int32_t value, const float scale)

One element of arm_sqrt_s16_tablefree(): the float32 chain the MVE path evaluates per lane, so the two agree bit for bit on any IEEE-754 float32 implementation with round-to-nearest-even and a fused multiply-add (fmaf). Every product after the pre-scale either has two uses or feeds an fmaf or a conversion, never another lone multiply, so a compiler allowed to reassociate (-ffast-math) still has no chain to reorder, and no product feeds a bare add, so there is nothing to contract.

Parameters

Parameters of arm_nn_sqrt_s16_tablefree_element

Name	Type	Direction	Description
value	const int32_t	in	input code; values <= 0 give 0
scale	const float	in	input_scale / (output_scale * output_scale) as float32

Returns

Returns of arm_nn_sqrt_s16_tablefree_element

Description
trunc(sqrt(value * scale)) saturated to 32767