arm_nnsupportfunctions.h

USE_FAST_DW_CONV_S16_FUNCTIONmacro
LEFT_SHIFTmacro
RIGHT_SHIFTmacro
MASK_IF_ZEROmacro
MASK_IF_NON_ZEROmacro
SELECT_USING_MASKmacro
ARM_NN_MAXmacro
ARM_NN_MINmacro
ARM_NN_CLAMPmacro
arm_nn_min_f16hfunctionMinimum of two scalar f16 values.
arm_nn_max_f16hfunctionMaximum of two scalar f16 values.
arm_nn_propagate_nan_f16hfunctionReturns x when x is NaN, otherwise y.
arm_nn_clamp_f16hfunctionDrop-in equivalent of ARMNNCLAMP(x, h, l) for scalar Float16 operands.
arm_nn_clamp_propagate_nan_f16hfunctionScalar f16 clamp with TFLite NaN semantics: NaN passes through unchanged.
arm_nn_abs_f16hfunctionAbsolute value of a scalar f16 value.
ARM_NN_ROUND_UPmacro
REDUCE_MULTIPLIERmacro
CH_IN_BLOCK_MVEmacro
S4_CH_IN_BLOCK_MVEmacro
MAX_COL_COUNTmacro
REVERSE_TCOL_EFFICIENT_THRESHOLDmacro
CONVERT_DW_CONV_WITH_ONE_INPUT_CH_AND_OUTPUT_CH_ABOVE_THRESHOLDmacro
OPTIONAL_RESTRICT_KEYWORDmacro
arm_nn_size_mulfunctionFold one dimension into a running buffer-size product, reporting overflow as -1.
arm_nn_size_addfunctionAdd to a running buffer-size product, reporting overflow as -1.
arm_nn_transpose_conv_s8_rolling_buffer_sizefunctionRolling scratch bytes shared by the s8 transpose entry and its buffer-size queries.
PACK_S8x4_32x1macrodefinition to pack four 8 bit values.
PACK_Q15x2_32x1macrodefinition to pack two 16 bit values.
GetNearestNeighborfunctionMap an output index to the nearest input index for resize.
arm_nn_requantize_channels_rshift_onlyfunctionWhether every channel requantizes with a right shift only: each has a shift below 0, or a multiplier of 0.
arm_nn_fc_packed_s8_sizefunctionBytes of the stream armfullyconnectedperchannelpackeds8() reads for an accumulation depth k and an output depth n: ceil(n / 4) blocks of four rows of k rounded…
arm_nn_fc_packed_s8_supportedfunctionThe gate of armfullyconnectedperchannelpackeds8(), which a caller selecting the kernel per layer ahead of time evaluates: filter offset 0, a positive accumulat…
arm_nn_is_row_broadcastfunctionThe row-broadcast route of armmuls8() and armadds8(): both inputs have the same depth C > 1, exactly one has W = 1, and the output W is above 1, as in [N, H, W…
arm_nn_is_convolve_1x1functionCheck if convolution parameters correspond to a 1x1 convolution.
arm_nn_is_convolve_1x1_fastfunctionCheck if a 1x1 convolution qualifies for the fast (unit stride) path.
arm_nn_is_convolve_1_x_nfunctionCheck if convolution parameters correspond to a 1xN convolution.
arm_nn_convolve_1_x_n_padding_supportedfunctionCheck that armconvolve1xns4() handles the horizontal padding of a 1xN convolution.
arm_nn_convolve_1_x_n_s8_padding_supportedfunctionCheck that armconvolve1xns8() accepts the padding and output shape of a 1xN convolution.
arm_nn_convolve_1_x_n_padded_columnsfunctionCount the output columns of a 1xN convolution whose window reads padding.
arm_nn_dw_conv_opt_dilation_supportedfunctionCheck if the dilation, stride and padding of a depthwise layer allow the armdepthwiseconvs8opt() or armdepthwiseconvfasts16() route.
arm_q7_to_q15_with_offsetfunctionConverts the elements from a s8 vector to a s16 vector with an added offset.
arm_depthwise_conv_s8_opt_get_buffer_size_mvefunctionGet the required buffer size for optimized s8 depthwise convolution function with constraint that inchannel equals outchannel.
arm_depthwise_conv_s8_opt_get_buffer_size_dspfunctionGet the required buffer size for optimized s8 depthwise convolution function with constraint that inchannel equals outchannel.
arm_nn_depthwise_conv_s8_corefunctionDepthwise conv on an im2col buffer where the input channel equals output channel.
arm_nn_mat_mult_s8functionGeneral Matrix-multiplication function with per-channel requantization.
arm_nn_mat_mult_kernel_s16functionMatrix-multiplication function for convolution with per-channel requantization for 16 bits convolution.
arm_nn_mat_mul_core_1x_s8functionGeneral Vector by Matrix multiplication with requantization and storage of result.
arm_nn_mat_mul_core_1x_s4functionGeneral Vector by Matrix multiplication with requantization, storage of result and int4 weights packed into an int8 buffer.
arm_nn_mat_mul_core_4x_s8functionMatrix-multiplication with requantization & activation function for four rows and one column.
arm_nn_mat_mult_nt_t_s4functionGeneral Matrix-multiplication function with per-channel requantization.
arm_nn_mat_mult_nt_interleaved_t_even_s4functionGeneral Matrix-multiplication function with per-channel requantization.
arm_nn_mat_mult_nt_t_s8functionGeneral Matrix-multiplication function with per-channel requantization.
arm_nn_mat_mult_nt_t_1x1_out_s8functionGeneral Matrix-multiplication function with per-channel requantization.
arm_nn_mat_mult_nt_t_s16functionGeneral Matrix-multiplication function with per-channel requantization and int16 input (LHS) and output.
arm_nn_mat_mult_nt_t_s8_s32functionGeneral Matrix-multiplication function with int8 input and int32 output.
arm_nn_vec_mat_mult_t_s4functions4 Vector by Matrix (transposed) multiplication
arm_nn_vec_mat_mult_t_s8functions8 Vector by Matrix (transposed) multiplication
arm_nn_vec_mat_mult_t_per_ch_s8functions8 Vector by Matrix (transposed) multiplication using per channel quantization for output
arm_nn_vec_mat_mult_t_s16functions16 Vector by s8 Matrix (transposed) multiplication
arm_nn_vec_mat_mult_t_per_ch_s16functions16 vector(lhs) by s8 matrix (transposed) multiplication and per channel quant output
arm_nn_vec_mat_mult_t_s16_s16functions16 Vector by s16 Matrix (transposed) multiplication
arm_nn_vec_mat_mult_t_svdf_s8functions8 Vector by Matrix (transposed) multiplication with s16 output
arm_nn_depthwise_conv_nt_t_padded_s8functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
arm_nn_depthwise_conv_nt_t_s8functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
arm_nn_depthwise_conv_s8_planar_candidatefunctionNecessary conditions of the planar rule that are cheap to test inline: at most 32 channels and stride 1.
arm_nn_is_convolve_s8_1x1_short_kfunctionThe gate of armconvolve1x1s8shortk(): a 1x1 kernel with no padding, unit stride and dilation, filter depth equal to the input depth, an input depth of 1 to 16,…
arm_nn_is_convolve_s8_small_cinfunctionThe gate of armconvolves8smallcin(): upscaledims NULL, input depth 1 to 3 with filter depth equal to it, dilation 1, a kernel of at least 1x1 with kernel width…
arm_nn_is_convolve_s8_3x3_c16_s1functionThe gate of armconvolves83x3c16s1(): upscaledims NULL, input and filter depth 16, a 3x3 kernel, and stride and dilation 1.
arm_nn_convolve_groups_invalidfunctionThe group check of the grouped convolutions: CIN and the filter C are positive, COUT is not negative, CIN is a whole number of filter depths, and COUT a whole…
arm_nn_convolve_s8_args_invalidfunctionThe argument check of armconvolves8() and its direct entries.
arm_nn_depthwise_s16_tap_index_invalidfunctionWhether the generic kernels of armdepthwiseconvs16(), armdepthwiseconvs8() and armdepthwiseconvs4() and the DSP path of armdepthwiseconvfasts16() could wrap th…
arm_nn_depthwise_conv_s16_index_invalidfunctionWhether the int32t index arithmetic of the s16 depthwise kernels could overflow for a layer: the MVE kernel shared by armdepthwiseconvs16() and armdepthwisecon…
arm_nn_depthwise_conv_s16_args_invalidfunctionWhether the generic kernel of armdepthwiseconvs16() cannot take a layer that armnndepthwiseconvs16indexinvalid() accepts, or the generic kernel of armdepthwise…
arm_nn_depthwise_conv_s8_planar_bytesfunctionPlane size in bytes that armnndepthwiseconvs8planar() needs for a layer, or -1 when the layer is not one it takes.
arm_nn_depthwise_conv_s8_planarfunctions8 depthwise convolution with channel multiplier 1 and stride 1, vectorized across the output pixels of one channel plane instead of across channels.
arm_nn_depthwise_conv_nt_t_s4functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
arm_nn_depthwise_conv_nt_t_s16functionDepthwise convolution of transposed rhs matrix with 4 lhs matrices.
arm_nn_transpose_conv_row_s8_s32functionRow of s8 scalars multiplicated with a s8 matrix ad accumulated into a s32 rolling scratch buffer.
arm_nn_read_q15x2_iafunctionRead 2 s16 elements and post increment pointer.
arm_nn_read_s8x4_iafunctionRead 4 s8 from s8 pointer and post increment pointer.
arm_nn_read_s8x2_iafunctionRead 2 s8 from s8 pointer and post increment pointer.
arm_nn_read_s16x2functionRead 2 int16 values from int16 pointer.
arm_nn_read_s8x4functionRead 4 s8 values.
arm_nn_read_s8x2functionRead 2 s8 values.
arm_nn_write_s8x4_iafunctionWrite four s8 to s8 pointer and increment pointer afterwards.
ARM_NN_DEPTH1_STOREmacro
arm_memset_s8functionmemset, a vector loop on MVE and a word loop where ARMNNWORDCOPY is defined
arm_memset_s16functionmemset for 16-bit data, a vector loop on MVE and a word loop where ARMNNWORDCOPY is defined
arm_nn_mat_mult_kernel_s4_s16functionMatrix-multiplication function for convolution with per-channel requantization and 4 bit weights.
arm_nn_mat_mult_kernel_s8_s16functionMatrix-multiplication function for convolution with per-channel requantization.
arm_nn_mat_mult_kernel_row_offset_s8_s16functionMatrix-multiplication function for convolution with per-channel requantization, supporting an address offset between rows.
arm_nn_softmax_common_s8functionCommon softmax function for s8 input and s8 or s16 output.
NN_ROUNDmacromacro for adding rounding offset
MUL_SATmacro
MUL_SAT_MVEmacro
MUL_POW2macro
DIV_POW2macro
DIV_POW2_MVEmacro
EXP_ON_NEGmacro
ONE_OVER1macro
arm_nn_doubling_high_multfunctionSaturating doubling high multiply.
arm_nn_doubling_high_mult_no_satfunctionDoubling high multiply without saturation.
arm_nn_divide_by_power_of_twofunctionRounding divide by power of two.
arm_nn_nonneg_divide_by_pot_s32functionRounding divide by power of two for non-negative values.
arm_nn_mac_wrap_s32_bitsfunctionMultiply-accumulate in int32t that wraps as the hardware does instead of overflowing.
arm_nn_requantizefunctionRequantize a given value.
arm_nn_requantize_positive_shift_s64functionRequantize with a positive scale exponent without narrowing the result.
arm_nn_requantize_s64functionRequantize a given 64 bit value.
arm_nn_sat_lshift_s16functionSaturating left shift for int16t.
arm_nn_sqrdmulh_s16functionSaturating Rounding Doubling High Mul (s16).
arm_nn_sqdmulh_s16functionSaturating Non-rounded Doubling High Mul (s16).
arm_nn_divide_by_power_of_two_s16functionRounding divide by power of two (s16), midpoint away from zero.
arm_memcpy_s8functionmemcpy, a vector loop on MVE and a word loop where ARMNNWORDCOPY is defined
arm_memcpy_s16functionmemcpy of int16t values through armmemcpys8()
arm_memcpy_s32functionmemcpy of int32t values through armmemcpys8()
arm_memcpy_q15functionmemcpy wrapper for int16, a word loop where ARMNNWORDCOPY is defined
arm_nn_exp_on_negative_valuesfunctionFixed-point exp() of a non-positive value.
SELECT_IF_NON_ZEROmacro
arm_nn_mult_by_power_of_twofunctionSaturating multiply by a power of two.
arm_nn_one_over_one_plus_x_for_x_in_0_1functionFixed-point 1 / (1 + x) for x in [0, 1), computed with Newton-Raphson iterations.
arm_nn_softmax_row_scalefunctionReciprocal scale of a softmax row, from the sum of its exponentials.
arm_nn_write_q15x2_iafunctionWrite 2 s16 elements and post increment pointer.
arm_nn_write_s8x2_iafunctionWrite 2 s8 elements and post increment pointer.
arm_cmsis_nn_dim_atfunctionGet dimension value at specific index.
arm_cmsis_nn_shape_productfunctionCalculate the product of all dimensions in a shape array.
arm_nn_lstm_step_s8functionUpdate LSTM function for an iteration step using s8 input and output, and s16 internally.
arm_nn_lstm_step_s16functionUpdate LSTM function for an iteration step using s16 input and output, and s16 internally.
arm_nn_lstm_calculate_gate_s8_s16functionUpdates a LSTM gate for an iteration step of LSTM function, int8x816 version.
arm_nn_lstm_calculate_gate_s16functionUpdates a LSTM gate for an iteration step of LSTM function, int16x816 version.
arm_nn_vec_mat_mul_result_acc_s8_s16functionThe result of the multiplication is accumulated to the passed result buffer.
arm_nn_vec_mat_mul_result_acc_s16functionThe result of the multiplication is accumulated to the passed result buffer.
arm_elementwise_mul_s16_s8functions16 elementwise multiplication with s8 output
arm_elementwise_mul_s16_batch_offsetfunctions16 elementwise multiplication with s16 output
arm_elementwise_mul_acc_s16functions16 elementwise multiplication.
arm_check_broadcast_requiredfunctionCheck if a broadcast is required between 2 cmsisnndims.
arm_reduce_get_middle_block_from_arraysfunctionReports whether the reduced axes of a 4-D tensor form one contiguous block followed by kept axes, as in a NHWC mean over H and W, and gives the flattened sizes.
ARM_NN_SQRT_S16_TABLEFREE_SHIFTmacro
ARM_NN_SQRT_S16_TABLEFREE_MAGICmacro
ARM_NN_SQRT_S16_TABLEFREE_K0macro
ARM_NN_SQRT_S16_TABLEFREE_K1macro
arm_nn_sqrt_s16_tablefree_elementfunctionOne element of armsqrts16tablefree(): the float32 chain the MVE path evaluates per lane, so the two agree bit for bit on any IEEE-754 float32 implementation wi…
arm_nn_f16_bits_to_f32_bitsfunctionInteger widening of one float16 bit pattern, with the result of the FPU's VCVTB at reset FPSCR: exact for every finite half and Inf, and a NaN keeps its sign a…
arm_nn_dequantize_f16_bits_f32functionThe body of armdequantizef16bitsf32() without its argument checks, inline so that a caller with a small or constant size pays no call.

#define USE_FAST_DW_CONV_S16_FUNCTION(dw_conv_params, filter_dims, input_dims, output_dims) (dw_conv_params->ch_mult == 1 && \ arm_nn_dw_conv_opt_dilation_supported(dw_conv_params, input_dims, filter_dims, output_dims) && \ (int64_t)filter_dims->w * filter_dims->h < 512)

Parameters

Parameters of USE_FAST_DW_CONV_S16_FUNCTION
Name	Description
`dw_conv_params`
`filter_dims`
`input_dims`
`output_dims`

#define LEFT_SHIFT(_shift) (_shift > 0 ? _shift : 0)

Parameters

Parameters of LEFT_SHIFT
Name	Description
`_shift`

#define RIGHT_SHIFT(_shift) (_shift > 0 ? 0 : -_shift)

Parameters

Parameters of RIGHT_SHIFT
Name	Description
`_shift`

#define MASK_IF_ZERO(x) (x) == 0 ? ~0 : 0

Parameters

Parameters of MASK_IF_ZERO
Name	Description
`x`

#define MASK_IF_NON_ZERO(x) (x) != 0 ? ~0 : 0

Parameters

Parameters of MASK_IF_NON_ZERO
Name	Description
`x`

#define SELECT_USING_MASK(mask, a, b) ((mask) & (a)) ^ (~(mask) & (b))

Parameters

Parameters of SELECT_USING_MASK
Name	Description
`mask`
`a`
`b`

#define ARM_NN_MAX(A, B) ((A) > (B) ? (A) : (B))

Parameters

Parameters of ARM_NN_MAX
Name	Description
`A`
`B`

#define ARM_NN_MIN(A, B) ((A) < (B) ? (A) : (B))

Parameters

Parameters of ARM_NN_MIN
Name	Description
`A`
`B`

#define ARM_NN_CLAMP(x, h, l) ARM_NN_MAX(ARM_NN_MIN((x), (h)), (l))

Parameters

Parameters of ARM_NN_CLAMP
Name	Description
`x`
`h`
`l`

static _Float16 arm_nn_min_f16h(_Float16 a, _Float16 b)

Minimum of two scalar f16 values.

With ARM_NN_F16_CMOV_WORKAROUND this is IEEE 754 minNum via VMINNM.F16: a NaN operand is suppressed and the non-NaN operand wins. The scalar fallback is ARM_NN_MIN, an ordered compare, so its NaN handling depends on operand order: a NaN b is returned, a NaN a is not. Do not rely on NaN suppression on non-MVE builds.

Parameters

Parameters of arm_nn_min_f16h
Name	Type	Direction	Description
`a`	`_Float16`	in	First operand
`b`	`_Float16`	in	Second operand

Returns

Returns of arm_nn_min_f16h
Description
The smaller of `a` and `b`

static _Float16 arm_nn_max_f16h(_Float16 a, _Float16 b)

Maximum of two scalar f16 values.

With ARM_NN_F16_CMOV_WORKAROUND this is IEEE 754 maxNum via VMAXNM.F16: a NaN operand is suppressed and the non-NaN operand wins. The scalar fallback is ARM_NN_MAX, an ordered compare, so its NaN handling depends on operand order: a NaN b is returned, a NaN a is not. Do not rely on NaN suppression on non-MVE builds.

Parameters

Parameters of arm_nn_max_f16h
Name	Type	Direction	Description
`a`	`_Float16`	in	First operand
`b`	`_Float16`	in	Second operand

Returns

Returns of arm_nn_max_f16h
Description
The larger of `a` and `b`

static _Float16 arm_nn_propagate_nan_f16h(_Float16 x, _Float16 y)

Returns x when x is NaN, otherwise y.

Both the NaN test and the select are performed on the bit patterns: the test is (bits & 0x7FFF) > 0x7C00 (all-ones exponent, non-zero mantissa), which is integer arithmetic that -ffinite-math-only (implied by the shipped -Ofast) has no license to fold, unlike the former floating-point self-compare x != x (#333 / #334); the bit-pattern select neither expands to an HFmode conditional move (PR target/118460) nor quiets/retags the NaN payload. This helper backs the f16 elementwise clamp and, via arm_nn_clamp_scalar_f16 / arm_nn_clamp_propagate_nan_f16h, the other f16 scalar clamp users arm_svdf_f16, arm_max_pool_f16 / arm_avg_pool_f16, the packed f16 matmul (arm_nn_mat_mult_nt_n_packed_f16), the scalar f16 RELU/RELU6/LEAKY_RELU activation legs, and arm_nn_vector_clamp_f16’s scalar leg (conv/depthwise/transpose-conv f16, the 3x3 depthwise, and arm_nn_maxpool1d_f16) so wherever that scalar clamp runs, a NaN passes through it at every optimization level on the gated toolchains. That is a guarantee about the clamp, not the whole kernel: which builds run the scalar clamp, and whether a NaN survives the rest of the kernel to reach it, is per kernel several of these callers clamp with vmaxnmq/vminnmq on MVE builds (a NaN resolves to a bound there), and arm_max_pool_f16’s max reduction drops a NaN before the clamp. The kernels with a NaN

Parameters

Parameters of arm_nn_propagate_nan_f16h
Name	Type	Direction	Description
`x`	`_Float16`	in	Value whose NaN-ness selects the result. Returned unchanged when it is NaN.
`y`	`_Float16`	in	Value returned when `x` is not NaN

Returns

Returns of arm_nn_propagate_nan_f16h
Description
`x` if `x` is NaN, otherwise `y`

static _Float16 arm_nn_clamp_f16h(_Float16 x, _Float16 h, _Float16 l)

Drop-in equivalent of ARM_NN_CLAMP(x, h, l) for scalar _Float16 operands.

Includes the macro’s NaN behaviour: ARM_NN_MIN(NaN, h) is h, so a NaN input resolves to the high bound, exactly as the macro does. Use arm_nn_clamp_propagate_nan_f16h() where TFLite NaN propagation is required.

Parameters

Parameters of arm_nn_clamp_f16h
Name	Type	Direction	Description
`x`	`_Float16`	in	Value to clamp
`h`	`_Float16`	in	Upper bound
`l`	`_Float16`	in	Lower bound

Returns

Returns of arm_nn_clamp_f16h
Description
`x` clamped to [`l`, `h`]

static _Float16 arm_nn_clamp_propagate_nan_f16h(_Float16 x, _Float16 l, _Float16 h)

Scalar f16 clamp with TFLite NaN semantics: NaN passes through unchanged.

Mirrors the MVE idiom in arm_nn_clamp_propagate_nan_mve_f16() (lower bound first, then upper bound, then restore NaN lanes). The NaN restore in arm_nn_propagate_nan_f16h() tests the integer bit pattern, so it holds at every optimization level including the shipped -Ofast; see #333 / #334. Bounds are assumed ordered (l <= h); inverted bounds are unspecified.

Parameters

Parameters of arm_nn_clamp_propagate_nan_f16h
Name	Type	Direction	Description
`x`	`_Float16`	in	Value to clamp
`l`	`_Float16`	in	Lower bound
`h`	`_Float16`	in	Upper bound

Returns

Returns of arm_nn_clamp_propagate_nan_f16h
Description
`x` clamped to [`l`, `h`], or `x` itself when it is NaN

static _Float16 arm_nn_abs_f16h(_Float16 x)

Absolute value of a scalar f16 value.

Parameters

Parameters of arm_nn_abs_f16h
Name	Type	Direction	Description
`x`	`_Float16`	in	Input value

Returns

Returns of arm_nn_abs_f16h
Description
\|`x`\|

#define ARM_NN_ROUND_UP(x, multiple) ((((x) + (multiple) - 1) / (multiple)) * (multiple))

Parameters

Parameters of ARM_NN_ROUND_UP
Name	Description
`x`
`multiple`

#define REDUCE_MULTIPLIER(_mult) ((_mult < 0x7FFF0000) ? ((_mult + (1 << 15)) >> 16) : 0x7FFF)

Parameters

Parameters of REDUCE_MULTIPLIER
Name	Description
`_mult`

#define CH_IN_BLOCK_MVE (124)

#define S4_CH_IN_BLOCK_MVE (124)

#define MAX_COL_COUNT (512)

#define REVERSE_TCOL_EFFICIENT_THRESHOLD (16)

#define CONVERT_DW_CONV_WITH_ONE_INPUT_CH_AND_OUTPUT_CH_ABOVE_THRESHOLD (1)

#define OPTIONAL_RESTRICT_KEYWORD

static int64_t arm_nn_size_mul(const int64_t acc, const int64_t factor)

Fold one dimension into a running buffer-size product, reporting overflow as -1.

Buffer-size queries return an int32_t byte count, so the product of the dimensions they multiply has to be rejected as soon as it cannot fit. Folding one factor at a time keeps the accumulator bounded: an accumulator already known to be <= INT32_MAX times a factor <= INT32_MAX cannot exceed about 2^62, so the int64_t accumulator itself never wraps. Chaining raw (int64_t) casts across three or more int32_t dims does not have that property - 65536 * 65536 * 65536 * 65536 is exactly 2^64 and folds back to 0, which would sail through a trailing “> INT32_MAX” test.

Note

This is the -1 sentinel family, used by the s8/s16 integer buffer-size queries, by the eight SVDF staging queries (arm_svdf_{s8,state_s16_s8,f32,f16}_{input,output}ctx_get_buffer_size) and by the s8/s16 LSTM temp-buffer queries and the GRU temp queries (arm_lstm_unidirectional{s8,s16}_temp{1,2}get_buffer_size, arm_gru_unidirectional{f32,f16}_temp1_get_buffer_size). The four f32/f16 LSTM temp queries have no dimensions to fold (the buffers are unused) and answer -1 only for NULL params, 0 otherwise. It is not interchangeable with the arm_nn_checked_size_mul() / arm_nn_size_to_i32_or_zero() helpers in Source/NNSupportFunctions (shared header for the float sizers), which most f32 and f16 buffer-size queries use and which report an out-of-range size as 0. Mixing the two silently flips a sizer’s out-of-range contract from “must never be used to size a buffer” to “you may pass { NULL, 0 }”, so pick the one the surrounding family already uses.

Parameters

Parameters of arm_nn_size_mul
Name	Type	Direction	Description
`acc`	`const int64_t`	in	Running product, or -1 if an earlier fold already overflowed.
`factor`	`const int64_t`	in	Next factor to fold in.

Returns

Returns of arm_nn_size_mul
Description
acc * factor, or -1 if acc is already -1, factor is negative or out of int32_t range, or the product exceeds INT32_MAX.

static int64_t arm_nn_size_add(const int64_t acc, const int64_t addend)

Add to a running buffer-size product, reporting overflow as -1.

Companion to arm_nn_size_mul() for the sizers that append a fixed slack term.

Parameters

Parameters of arm_nn_size_add
Name	Type	Direction	Description
`acc`	`const int64_t`	in	Running product, or -1 if an earlier step already overflowed.
`addend`	`const int64_t`	in	Value to add. Must be non-negative.

Returns

Returns of arm_nn_size_add
Description
acc + addend, or -1 if acc is already -1 or the sum exceeds INT32_MAX.

static int32_t arm_nn_transpose_conv_s8_rolling_buffer_size(
    const cmsis_nn_transpose_conv_params *params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Rolling scratch bytes shared by the s8 transpose entry and its buffer-size queries.

Parameters

Parameters of arm_nn_transpose_conv_s8_rolling_buffer_size
Name	Type	Direction	Description
`params`	`const cmsis_nn_transpose_conv_params *`	in	Transpose convolution parameters.
`input_dims`	`const cmsis_nn_dims *`	in	Input dimensions.
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions.
`output_dims`	`const cmsis_nn_dims *`	in	Output dimensions.

Returns

Returns of arm_nn_transpose_conv_s8_rolling_buffer_size
Description
Required bytes, or -1 if the rolling geometry or size is out of range.

#define PACK_S8x4_32x1(v0, v1, v2, v3) ((int32_t)((((uint32_t)(v0)) & 0xFFu) | ((((uint32_t)(v1)) & 0xFFu) << 8) | ((((uint32_t)(v2)) & 0xFFu) << 16) | \ ((((uint32_t)(v3)) & 0xFFu) << 24)))

definition to pack four 8 bit values.

Byte lanes are masked and shifted in uint32_t so a negative value never feeds a signed left shift (UB); masking before the shift keeps the same bits the old shift-then-mask form kept. Bit-identical for every input. Deliberate divergence from upstream ARM-software/CMSIS-NN, which still carries the signed-shift form do not paste the upstream text back on a sync (issue #357).

Parameters

Parameters of PACK_S8x4_32x1
Name	Description
`v0`
`v1`
`v2`
`v3`

#define PACK_Q15x2_32x1(v0, v1) ((int32_t)((((uint32_t)(v0)) & 0xFFFFu) | (((uint32_t)(v1)) << 16)))

definition to pack two 16 bit values.

Same treatment: the high half is shifted in uint32_t, not int32_t, so a negative v1 is defined; the low half keeps its mask. Bit-identical for every input. Same deliberate upstream divergence as PACK_S8x4_32x1 above.

Parameters

Parameters of PACK_Q15x2_32x1
Name	Description
`v0`
`v1`

static int32_t GetNearestNeighbor(
    const int input_value,
    const int32_t input_size,
    const float scale,
    const float offset,
    const bool align_corners,
    const bool half_pixel_centers
)

Map an output index to the nearest input index for resize.

This helper follows the TensorFlow Lite nearest-neighbor resize mapping rules.

Parameters

Parameters of GetNearestNeighbor
Name	Type	Direction	Description
`input_value`	`const int`	in	Output index (x or y).
`input_size`	`const int32_t`	in	Input size along the same axis.
`scale`	`const float`	in	Precomputed scaling factor for the axis.
`offset`	`const float`	in	Precomputed offset for the axis.
`align_corners`	`const bool`	in	If true, use align-corners scaling.
`half_pixel_centers`	`const bool`	in	If true, use half-pixel center offset.

Returns

Returns of GetNearestNeighbor
Description
Nearest input index for the given output index.

static int32_t arm_nn_requantize_channels_rshift_only(
    const int32_t *multiplier,
    const int32_t *shift,
    const int32_t num_ch
)

Whether every channel requantizes with a right shift only: each has a shift below 0, or a multiplier of 0. Plain C; it evaluates the same on every build. arm_nn_requantize_rshift_only() adds the build condition.

Parameters

Parameters of arm_nn_requantize_channels_rshift_only
Name	Type	Direction	Description
`multiplier`	`const int32_t *`	in	Per-channel multipliers
`shift`	`const int32_t *`	in	Per-channel shifts
`num_ch`	`const int32_t`	in	Number of channels

Returns

Returns of arm_nn_requantize_channels_rshift_only
Description
1 when every channel qualifies, 0 otherwise.

static int64_t arm_nn_fc_packed_s8_size(const int32_t k, const int32_t n)

Bytes of the stream arm_fully_connected_per_channel_packed_s8() reads for an accumulation depth k and an output depth n: ceil(n / 4) blocks of four rows of k rounded up to 16, plus 48 bytes of parameters.

Parameters

Parameters of arm_nn_fc_packed_s8_size
Name	Type	Direction	Description
`k`	`const int32_t`	in	Accumulation depth, positive
`n`	`const int32_t`	in	Output depth, positive

Returns

Returns of arm_nn_fc_packed_s8_size
Description
The size, in 64 bits so that the caller can bound it.

static int32_t arm_nn_fc_packed_s8_supported(
    const cmsis_nn_fc_params *fc_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const cmsis_nn_dims *filter_dims
)

The gate of arm_fully_connected_per_channel_packed_s8(), which a caller selecting the kernel per layer ahead of time evaluates: filter offset 0, a positive accumulation and output depth, a stream of at most INT32_MAX bytes, and every output channel right-shift-only. Plain C; it evaluates the same on every build. The entry also needs MVE (ARM_MATH_MVEI without ARM_MATH_AUTOVECTORIZE) and the default rounding.

Parameters

Parameters of arm_nn_fc_packed_s8_supported
Name	Type	Direction	Description
`fc_params`	`const cmsis_nn_fc_params *`	in	Fully connected parameters
`quant_params`	`const cmsis_nn_per_channel_quant_params *`	in	Per-channel multipliers and shifts
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions. Format: [N, C]; N is the accumulation depth, C the output depth

Returns

Returns of arm_nn_fc_packed_s8_supported
Description
1 when the layer is in the gate, 0 otherwise.

static int32_t arm_nn_is_row_broadcast(
    const cmsis_nn_dims *input1_dims,
    const cmsis_nn_dims *input2_dims,
    const cmsis_nn_dims *output_dims
)

The row-broadcast route of arm_mul_s8() and arm_add_s8(): both inputs have the same depth C > 1, exactly one has W = 1, and the output W is above 1, as in [N, H, W, C] x [N | 1, H | 1, 1, C] (a squeeze-and-excite scale). Plain C; it evaluates the same on every build.

Parameters

Parameters of arm_nn_is_row_broadcast
Name	Type	Direction	Description
`input1_dims`	`const cmsis_nn_dims *`	in	First input dimensions [N, H, W, C]
`input2_dims`	`const cmsis_nn_dims *`	in	Second input dimensions [N, H, W, C]
`output_dims`	`const cmsis_nn_dims *`	in	Output dimensions [N, H, W, C]

Returns

Returns of arm_nn_is_row_broadcast
Description
1 for the row-broadcast shape, 0 otherwise.

static bool arm_nn_is_convolve_1x1(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims
)

Check if convolution parameters correspond to a 1x1 convolution.

Parameters

Parameters of arm_nn_is_convolve_1x1
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input dimensions
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions

Returns

Returns of arm_nn_is_convolve_1x1
Description
true if parameters describe a 1x1 convolution, false otherwise.

static bool arm_nn_is_convolve_1x1_fast(const cmsis_nn_conv_params *conv_params)

Check if a 1x1 convolution qualifies for the fast (unit stride) path.

Parameters

Parameters of arm_nn_is_convolve_1x1_fast
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters

Returns

Returns of arm_nn_is_convolve_1x1_fast
Description
true if stride is 1x1, false otherwise.

static bool arm_nn_is_convolve_1_x_n(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims
)

Check if convolution parameters correspond to a 1xN convolution.

Parameters

Parameters of arm_nn_is_convolve_1_x_n
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input dimensions
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions

Returns

Returns of arm_nn_is_convolve_1_x_n
Description
true if parameters describe a 1xN convolution, false otherwise.

static bool arm_nn_convolve_1_x_n_padding_supported(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Check that arm_convolve_1_x_n_s4() handles the horizontal padding of a 1xN convolution.

The kernel places pad.w columns on the left and pad.w + (total_pad % 2) on the right, where total_pad = (output W - 1) * stride.w + filter W - input W, and needs the output columns that read padding to fit in output W. Its padded-column code also assumes that each such column reads at least one input column and that the filter is no wider than the input; otherwise it forms input and filter addresses outside the tensors. On MVE builds another pad placement, or too many padded columns, returns ARM_CMSIS_NN_FAILURE. A VALID layer whose stride leaves trailing input unused (negative total_pad) is therefore rejected, and the wrapper routes it to another convolution. The kernel also computes a single output row, so vertical padding or an output height other than 1 is rejected. A non-positive stride.w is left to the kernel’s argument checks.

Parameters

Parameters of arm_nn_convolve_1_x_n_padding_supported
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input dimensions
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions
`output_dims`	`const cmsis_nn_dims *`	in	Output dimensions

Returns

Returns of arm_nn_convolve_1_x_n_padding_supported
Description
true when `arm_convolve_1_x_n_s4()` handles the padding, false otherwise.

static bool arm_nn_convolve_1_x_n_s8_padding_supported(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Check that arm_convolve_1_x_n_s8() accepts the padding and output shape of a 1xN convolution.

The kernel computes a single output row for any pad.w >= 0 and any output width, including an odd total padding, a filter wider than the input and a VALID layer whose stride leaves trailing input unused. It rejects vertical padding, an output height other than 1, a negative pad.w and an empty filter; the wrapper routes those layers to another convolution. A non-positive stride.w is left to the kernel’s argument checks.

Parameters

Parameters of arm_nn_convolve_1_x_n_s8_padding_supported
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions
`output_dims`	`const cmsis_nn_dims *`	in	Output dimensions

Returns

Returns of arm_nn_convolve_1_x_n_s8_padding_supported
Description
true when `arm_convolve_1_x_n_s8()` computes the layer, false otherwise.

static void arm_nn_convolve_1_x_n_padded_columns(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims,
    int64_t *left_num,
    int64_t *right_num
)

Count the output columns of a 1xN convolution whose window reads padding.

Output column j reads input columns j * stride.w - pad.w to j * stride.w - pad.w + filter W - 1. The leading columns whose window starts before the input are left-padded; of the others, the trailing columns whose window ends past the input are right-padded. A window can do both only when it is left-padded.

Parameters

Parameters of arm_nn_convolve_1_x_n_padded_columns
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters. stride.w >= 1 and pad.w >= 0.
`input_dims`	`const cmsis_nn_dims *`	in	Input dimensions. w >= 0.
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions. w >= 1.
`output_dims`	`const cmsis_nn_dims *`	in	Output dimensions. w >= 0.
`left_num`	`int64_t *`	out	Number of left-padded output columns, output W at most.
`right_num`	`int64_t *`	out	Number of right-padded output columns, output W - left_num at most.

static bool arm_nn_dw_conv_opt_dilation_supported(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Check if the dilation, stride and padding of a depthwise layer allow the arm_depthwise_conv_s8_opt() or arm_depthwise_conv_fast_s16() route.

Parameters

Parameters of arm_nn_dw_conv_opt_dilation_supported
Name	Type	Direction	Description
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input dimensions
`filter_dims`	`const cmsis_nn_dims *`	in	Filter dimensions
`output_dims`	`const cmsis_nn_dims *`	in	Output dimensions

Returns

Returns of arm_nn_dw_conv_opt_dilation_supported
Description
true for an undilated layer (dilation 1 in both dimensions), or for a 1D layer dilated along the width only: filter, input and output height 1, stride 1 in both dimensions, no vertical padding, dilation.h == 1 and dilation.w >= 1. false otherwise.

void arm_q7_to_q15_with_offset(const int8_t *src, int16_t *dst, int32_t block_size, int16_t offset)

Converts the elements from a s8 vector to a s16 vector with an added offset.

Output elements are ordered. The equation used for the conversion process is:

dst[n] = (int16_t) src[n] + offset; 0 <= n < block_size.

Parameters

Parameters of arm_q7_to_q15_with_offset
Name	Type	Direction	Description
`src`	`const int8_t *`	in	pointer to the s8 input vector
`dst`	`int16_t *`	out	pointer to the s16 output vector
`block_size`	`int32_t`	in	length of the input vector
`offset`	`int16_t`	in	s16 offset to be added to each input vector element.

int32_t arm_depthwise_conv_s8_opt_get_buffer_size_mve(const cmsis_nn_dims *input_dims, const cmsis_nn_dims *filter_dims)

Get the required buffer size for optimized s8 depthwise convolution function with constraint that in_channel equals out_channel. This is for processors with MVE extension.

The dimensions are checked here, so a negative dimension returns -1 on every build target. The byte count is range-checked inside the selected leg instead, because the Helium and DSP legs use different formulas and the plain-C build needs no buffer at all.

Parameters

Parameters of arm_depthwise_conv_s8_opt_get_buffer_size_mve
Name	Type	Direction	Description
`input_dims`	`const cmsis_nn_dims *`	in	Input (activation) tensor dimensions. Format: [1, H, W, C_IN] Batch argument N is not used.
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]

Returns

Returns of arm_depthwise_conv_s8_opt_get_buffer_size_mve
Description
The function returns required buffer size in bytes, or -1 if any dimension it reads is negative or the required size would not fit in an int32_t

int32_t arm_depthwise_conv_s8_opt_get_buffer_size_dsp(const cmsis_nn_dims *input_dims, const cmsis_nn_dims *filter_dims)

Get the required buffer size for optimized s8 depthwise convolution function with constraint that in_channel equals out_channel. This is for processors with DSP extension.

Parameters

Parameters of arm_depthwise_conv_s8_opt_get_buffer_size_dsp
Name	Type	Direction	Description
`input_dims`	`const cmsis_nn_dims *`	in	Input (activation) tensor dimensions. Format: [1, H, W, C_IN] Batch argument N is not used.
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]

Returns

Returns of arm_depthwise_conv_s8_opt_get_buffer_size_dsp
Description
The function returns required buffer size in bytes, or -1 if any dimension it reads is negative or the required size would not fit in an int32_t

int8_t * arm_nn_depthwise_conv_s8_core(
    const int8_t *row,
    const int16_t *col,
    const uint16_t num_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t kernel_size,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise conv on an im2col buffer where the input channel equals output channel.

Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_depthwise_conv_s8_core
Name	Type	Direction	Description
`row`	`const int8_t *`	in	pointer to row
`col`	`const int16_t *`	in	pointer to im2col buffer, always consists of 2 columns.
`num_ch`	`const uint16_t`	in	number of channels
`out_shift`	`const int32_t *`	in	pointer to per output channel requantization shift parameter.
`out_mult`	`const int32_t *`	in	pointer to per output channel requantization multiplier parameter.
`out_offset`	`const int32_t`	in	output tensor offset.
`activation_min`	`const int32_t`	in	minimum value to clamp the output to. Range : int8
`activation_max`	`const int32_t`	in	maximum value to clamp the output to. Range : int8
`kernel_size`	`const uint16_t`	in	number of elements in one column.
`output_bias`	`const int32_t *const`	in	per output channel bias. Range : int32
`out`	`int8_t *`	out	pointer to output

Returns

Returns of arm_nn_depthwise_conv_s8_core
Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

int8_t * arm_nn_mat_mult_s8(
    const int8_t *input_row,
    const int8_t *input_col,
    const uint16_t output_ch,
    const uint16_t col_batches,
    const int32_t *output_shift,
    const int32_t *output_mult,
    const int32_t out_offset,
    const int32_t col_offset,
    const int32_t row_offset,
    const int16_t out_activation_min,
    const int16_t out_activation_max,
    const uint16_t row_len,
    const int32_t *const bias,
    int8_t *out
)

General Matrix-multiplication function with per-channel requantization.

Supported framework: TensorFlow Lite

Parameters

Parameters of arm_nn_mat_mult_s8
Name	Type	Direction	Description
`input_row`	`const int8_t *`	in	pointer to row operand
`input_col`	`const int8_t *`	in	pointer to col operand
`output_ch`	`const uint16_t`	in	number of rows of input_row
`col_batches`	`const uint16_t`	in	number of column batches. Range: 1 to 4
`output_shift`	`const int32_t *`	in	pointer to per output channel requantization shift parameter.
`output_mult`	`const int32_t *`	in	pointer to per output channel requantization multiplier parameter.
`out_offset`	`const int32_t`	in	output tensor offset.
`col_offset`	`const int32_t`	in	input tensor(col) offset.
`row_offset`	`const int32_t`	in	kernel offset(row). Not used.
`out_activation_min`	`const int16_t`	in	minimum value to clamp the output to. Range : int8
`out_activation_max`	`const int16_t`	in	maximum value to clamp the output to. Range : int8
`row_len`	`const uint16_t`	in	number of elements in each row
`bias`	`const int32_t *const`	in	per output channel bias. Range : int32
`out`	`int8_t *`	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_s8
Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

int16_t * arm_nn_mat_mult_kernel_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const int32_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t num_col_a,
    const cmsis_nn_bias_data *const bias_data,
    int16_t *out_0,
    const int32_t row_address_offset
)

Matrix-multiplication function for convolution with per-channel requantization for 16 bits convolution.

This function does the matrix multiplication of weight matrix for all output channels with 2 columns from im2col and produces two elements/output_channel. The outputs are clamped in the range provided by activation min and max. Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_mat_mult_kernel_s16
Name	Type	Direction	Description
`input_a`	`const int8_t *`	in	pointer to operand A
`input_b`	`const int16_t *`	in	pointer to operand B, always consists of 2 vectors.
`output_ch`	`const int32_t`	in	number of rows of A
`out_shift`	`const int32_t *`	in	pointer to per output channel requantization shift parameter.
`out_mult`	`const int32_t *`	in	pointer to per output channel requantization multiplier parameter.
`activation_min`	`const int32_t`	in	minimum value to clamp the output to. Range : int16
`activation_max`	`const int32_t`	in	maximum value to clamp the output to. Range : int16
`num_col_a`	`const int32_t`	in	number of columns of A
`bias_data`	`const cmsis_nn_bias_data *const`	in	pointer to struct with bias vector. The length of this vector is equal to the number of output columns (or RHS input rows). The vector can be int32 or int64 indicated by a flag in the struct.
`out_0`	`int16_t *`	in, out	pointer to output
`row_address_offset`	`const int32_t`	in	Address offset between rows in output.

Returns

Returns of arm_nn_mat_mult_kernel_s16
Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

arm_cmsis_nn_status arm_nn_mat_mul_core_1x_s8(
    int32_t row_elements,
    const int32_t skipped_row_elements,
    const int8_t *row_base_ref,
    const int8_t *col_base_ref,
    const int32_t out_ch,
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const int32_t *bias,
    int8_t *output
)

General Vector by Matrix multiplication with requantization and storage of result.

Pseudo-code *output = 0 sum_col = 0 for (j = 0; j < out_ch; j++) for (i = 0; i < row_elements; i++) *output += row_base_ref[i] * col_base_ref[i] sum_col += col_base_ref[i] scale sum_col using quant_params and bias store result in ‘output’

Parameters

Parameters of arm_nn_mat_mul_core_1x_s8
Name	Type	Direction	Description
`row_elements`	`int32_t`	in	number of row elements
`skipped_row_elements`	`const int32_t`	in	number of row elements skipped due to padding. row_elements + skipped_row_elements = (kernel_x * kernel_y) * input_ch
`row_base_ref`	`const int8_t *`	in	pointer to row operand
`col_base_ref`	`const int8_t *`	in	pointer to col operand
`out_ch`	`const int32_t`	in	Number of output channels
`conv_params`	`const cmsis_nn_conv_params *`	in	Pointer to convolution parameters like offsets and activation values
`quant_params`	`const cmsis_nn_per_channel_quant_params *`	in	Pointer to per-channel quantization parameters
`bias`	`const int32_t *`	in	Pointer to optional per-channel bias
`output`	`int8_t *`	out	Pointer to output where int8 results are stored.

Returns

Returns of arm_nn_mat_mul_core_1x_s8
Description
The function performs matrix(row_base_ref) multiplication with vector(col_base_ref) and scaled result is stored in memory.

arm_cmsis_nn_status arm_nn_mat_mul_core_1x_s4(
    int32_t row_elements,
    const int32_t skipped_row_elements,
    const int8_t *row_base_ref,
    const int8_t *col_base_ref,
    const int32_t out_ch,
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const int32_t *bias,
    int8_t *output
)

General Vector by Matrix multiplication with requantization, storage of result and int4 weights packed into an int8 buffer.

Pseudo-code as int8 example. Int4 filter data will be unpacked. *output = 0 sum_col = 0 for (j = 0; j < out_ch; j++) for (i = 0; i < row_elements; i++) *output += row_base_ref[i] * col_base_ref[i] sum_col += col_base_ref[i] scale sum_col using quant_params and bias store result in ‘output’

Parameters

Parameters of arm_nn_mat_mul_core_1x_s4
Name	Type	Direction	Description
`row_elements`	`int32_t`	in	number of row elements
`skipped_row_elements`	`const int32_t`	in	number of row elements skipped due to padding. row_elements + skipped_row_elements = (kernel_x * kernel_y) * input_ch
`row_base_ref`	`const int8_t *`	in	pointer to row operand
`col_base_ref`	`const int8_t *`	in	pointer to col operand as packed int4
`out_ch`	`const int32_t`	in	Number of output channels
`conv_params`	`const cmsis_nn_conv_params *`	in	Pointer to convolution parameters like offsets and activation values
`quant_params`	`const cmsis_nn_per_channel_quant_params *`	in	Pointer to per-channel quantization parameters
`bias`	`const int32_t *`	in	Pointer to optional per-channel bias
`output`	`int8_t *`	out	Pointer to output where int8 results are stored.

Returns

Returns of arm_nn_mat_mul_core_1x_s4
Description
The function performs matrix(row_base_ref) multiplication with vector(col_base_ref) and scaled result is stored in memory.

int8_t * arm_nn_mat_mul_core_4x_s8(
    const int32_t row_elements,
    const int32_t offset,
    const int8_t *row_base,
    const int8_t *col_base,
    const int32_t out_ch,
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const int32_t *bias,
    int8_t *output
)

Matrix-multiplication with requantization & activation function for four rows and one column.

Compliant to TFLM int8 specification. MVE implementation only

Parameters

Parameters of arm_nn_mat_mul_core_4x_s8
Name	Type	Direction	Description
`row_elements`	`const int32_t`	in	number of row elements
`offset`	`const int32_t`	in	offset between rows. Can be the same as row_elements. For e.g, in a 1x1 conv scenario with stride as 1.
`row_base`	`const int8_t *`	in	pointer to row operand
`col_base`	`const int8_t *`	in	pointer to col operand
`out_ch`	`const int32_t`	in	Number of output channels
`conv_params`	`const cmsis_nn_conv_params *`	in	Pointer to convolution parameters like offsets and activation values
`quant_params`	`const cmsis_nn_per_channel_quant_params *`	in	Pointer to per-channel quantization parameters
`bias`	`const int32_t *`	in	Pointer to per-channel bias
`output`	`int8_t *`	out	Pointer to output where int8 results are stored.

Returns

Returns of arm_nn_mat_mul_core_4x_s8
Description
The function returns the updated output pointer or NULL if implementation is not available.

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s4(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. This function assumes:

LHS input matrix NOT transposed (nt)
RHS input matrix transposed (t)
RHS is int8 packed with 2x int4
LHS is int8

Parameters

Parameters of arm_nn_mat_mult_nt_t_s4
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Pointer to the LHS input matrix
`rhs`	`const int8_t *`	in	Pointer to the RHS input matrix
`bias`	`const int32_t *`	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst`	`int8_t *`	out	Pointer to the output matrix with "m" rows and "n" columns
`dst_multipliers`	`const int32_t *`	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst_shifts`	`const int32_t *`	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`lhs_rows`	`const int32_t`	in	Number of LHS input rows
`rhs_rows`	`const int32_t`	in	Number of RHS input rows
`rhs_cols`	`const int32_t`	in	Number of LHS/RHS input columns
`lhs_offset`	`const int32_t`	in	Offset to be applied to the LHS input value
`dst_offset`	`const int32_t`	in	Offset to be applied the output result
`activation_min`	`const int32_t`	in	Minimum value to clamp down the output. Range : int8
`activation_max`	`const int32_t`	in	Maximum value to clamp up the output. Range : int8
`lhs_cols_offset`	`const int32_t`	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_t_s4
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_mat_mult_nt_interleaved_t_even_s4(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. This function assumes:

LHS input matrix NOT transposed (nt)
RHS input matrix transposed (t)
RHS is int8 packed with 2x int4
LHS is int8
LHS/RHS input columns must be even numbered
LHS must be interleaved. Compare to arm_nn_mat_mult_nt_t_s4 where LHS is not interleaved.

Parameters

Parameters of arm_nn_mat_mult_nt_interleaved_t_even_s4
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Pointer to the LHS input matrix
`rhs`	`const int8_t *`	in	Pointer to the RHS input matrix
`bias`	`const int32_t *`	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst`	`int8_t *`	out	Pointer to the output matrix with "m" rows and "n" columns
`dst_multipliers`	`const int32_t *`	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst_shifts`	`const int32_t *`	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`lhs_rows`	`const int32_t`	in	Number of LHS input rows
`rhs_rows`	`const int32_t`	in	Number of RHS input rows
`rhs_cols`	`const int32_t`	in	Number of LHS/RHS input columns. Note this must be even.
`lhs_offset`	`const int32_t`	in	Offset to be applied to the LHS input value
`dst_offset`	`const int32_t`	in	Offset to be applied the output result
`activation_min`	`const int32_t`	in	Minimum value to clamp down the output. Range : int8
`activation_max`	`const int32_t`	in	Maximum value to clamp up the output. Range : int8
`lhs_cols_offset`	`const int32_t`	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_interleaved_t_even_s4
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s8(
    const int32_t *weight_sum_buf,
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t row_address_offset,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. This function assumes:

LHS input matrix NOT transposed (nt)
RHS input matrix transposed (t)

Parameters

Parameters of arm_nn_mat_mult_nt_t_s8
Name	Type	Direction	Description
`weight_sum_buf`	`const int32_t *`	in	Pointer to the weight sum multiplied by lhs_offset and summed bias buffer
`lhs`	`const int8_t *`	in	Pointer to the LHS input matrix
`rhs`	`const int8_t *`	in	Pointer to the RHS input matrix
`bias`	`const int32_t *`	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst`	`int8_t *`	out	Pointer to the output matrix with "m" rows and "n" columns
`dst_multipliers`	`const int32_t *`	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst_shifts`	`const int32_t *`	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`lhs_rows`	`const int32_t`	in	Number of LHS input rows
`rhs_rows`	`const int32_t`	in	Number of RHS input rows
`rhs_cols`	`const int32_t`	in	Number of LHS/RHS input columns
`lhs_offset`	`const int32_t`	in	Offset to be applied to the LHS input value
`dst_offset`	`const int32_t`	in	Offset to be applied the output result
`activation_min`	`const int32_t`	in	Minimum value to clamp down the output. Range : int8
`activation_max`	`const int32_t`	in	Maximum value to clamp up the output. Range : int8
`row_address_offset`	`const int32_t`	in	Address offset between rows in output. NOTE: Only used for MVEI extension.
`lhs_cols_offset`	`const int32_t`	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_t_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_1x1_out_s8(
    const int32_t *weight_sum_buf,
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t row_address_offset,
    const int32_t lhs_cols_offset
)

General Matrix-multiplication function with per-channel requantization. Output is calculated with multiple channels in parallel, rather than multiple output indices in a single channel This function assumes:

LHS input matrix NOT transposed (nt)
RHS input matrix transposed (t)

Parameters

Parameters of arm_nn_mat_mult_nt_t_1x1_out_s8
Name	Type	Direction	Description
`weight_sum_buf`	`const int32_t *`	in	Pointer to the weight sum multiplied by lhs_offset and summed bias buffer
`lhs`	`const int8_t *`	in	Pointer to the LHS input matrix
`rhs`	`const int8_t *`	in	Pointer to the RHS input matrix
`bias`	`const int32_t *`	in	Pointer to the bias vector. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst`	`int8_t *`	out	Pointer to the output matrix with "m" rows and "n" columns
`dst_multipliers`	`const int32_t *`	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst_shifts`	`const int32_t *`	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`lhs_rows`	`const int32_t`	in	Number of LHS input rows
`rhs_rows`	`const int32_t`	in	Number of RHS input rows
`rhs_cols`	`const int32_t`	in	Number of LHS/RHS input columns
`lhs_offset`	`const int32_t`	in	Offset to be applied to the LHS input value
`dst_offset`	`const int32_t`	in	Offset to be applied the output result
`activation_min`	`const int32_t`	in	Minimum value to clamp down the output. Range : int8
`activation_max`	`const int32_t`	in	Maximum value to clamp up the output. Range : int8
`row_address_offset`	`const int32_t`	in	Address offset between rows in output. NOTE: Only used for MVEI extension.
`lhs_cols_offset`	`const int32_t`	in	Column offset between subsequent lhs_rows

Returns

Returns of arm_nn_mat_mult_nt_t_1x1_out_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const cmsis_nn_bias_data *bias_data,
    int16_t *dst,
    const int32_t *dst_multipliers,
    const int32_t *dst_shifts,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t row_address_offset
)

General Matrix-multiplication function with per-channel requantization and int16 input (LHS) and output. This function assumes:

LHS input matrix NOT transposed (nt)
RHS input matrix transposed (t)

MVE implementation only.

Parameters

Parameters of arm_nn_mat_mult_nt_t_s16
Name	Type	Direction	Description
`lhs`	`const int16_t *`	in	Pointer to the LHS input matrix
`rhs`	`const int8_t *`	in	Pointer to the RHS input matrix
`bias_data`	`const cmsis_nn_bias_data *`	in	Pointer to struct with bias vector. The length of this vector is equal to the number of output columns (or RHS input rows). The vector can be int32 or int64 indicated by a flag in the struct.
`dst`	`int16_t *`	out	Pointer to the output matrix with "m" rows and "n" columns
`dst_multipliers`	`const int32_t *`	in	Pointer to the multipliers vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`dst_shifts`	`const int32_t *`	in	Pointer to the shifts vector needed for the per-channel requantization. The length of this vector is equal to the number of output columns (or RHS input rows)
`lhs_rows`	`const int32_t`	in	Number of LHS input rows
`rhs_rows`	`const int32_t`	in	Number of RHS input rows
`rhs_cols`	`const int32_t`	in	Number of LHS/RHS input columns
`activation_min`	`const int32_t`	in	Minimum value to clamp down the output. Range : int16
`activation_max`	`const int32_t`	in	Maximum value to clamp up the output. Range : int16
`row_address_offset`	`const int32_t`	in	Address offset between rows in output. NOTE: Only used for MVEI extension.

Returns

Returns of arm_nn_mat_mult_nt_t_s16
Description
The function returns `ARM_CMSIS_NN_SUCCESS` or `ARM_CMSIS_NN_NO_IMPL_ERROR` if not for MVE \|---row_address_offset---\| \|____rhs_rows__________________\| \| \| \| \| --- \| --- \| \| \| \| \| \| \| \| \| \| lhs_rows \| \| \| \| --- \| --- \| \| _______________ \| ______________ \|

arm_cmsis_nn_status arm_nn_mat_mult_nt_t_s8_s32(
    const int8_t *lhs,
    const int8_t *rhs,
    int32_t *dst,
    const int32_t lhs_rows,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t lhs_offset,
    const int32_t dst_idx_offset
)

General Matrix-multiplication function with int8 input and int32 output. This function assumes:

LHS input matrix NOT transposed (nt)
RHS input matrix transposed (t)

Parameters

Parameters of arm_nn_mat_mult_nt_t_s8_s32
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Pointer to the LHS input matrix
`rhs`	`const int8_t *`	in	Pointer to the RHS input matrix
`dst`	`int32_t *`	in, out	Pointer to the output matrix with "m" rows and "n" columns. Accumulated into, so it must be zeroed by the caller before the call
`lhs_rows`	`const int32_t`	in	Number of LHS input rows
`rhs_rows`	`const int32_t`	in	Number of LHS input columns/RHS input rows
`rhs_cols`	`const int32_t`	in	Number of RHS input columns
`lhs_offset`	`const int32_t`	in	Offset to be applied to the LHS input value
`dst_idx_offset`	`const int32_t`	in	Offset between subsequent output results

Returns

Returns of arm_nn_mat_mult_nt_t_s8_s32
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s4(
    const int8_t *lhs,
    const int8_t *packed_rhs,
    const int32_t *bias,
    int8_t *dst,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s4 Vector by Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s4
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side vector
`packed_rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`bias`	`const int32_t *`	in	Input bias
`dst`	`int8_t *`	out	Output vector
`lhs_offset`	`const int32_t`	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
`dst_offset`	`const int32_t`	in	Offset to be added to the output values. Range: -127 to 128
`dst_multiplier`	`const int32_t`	in	Output multiplier
`dst_shift`	`const int32_t`	in	Output shift
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8

Returns

Returns of arm_nn_vec_mat_mult_t_s4
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *kernel_sum,
    const int32_t *bias,
    int8_t *dst,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t address_offset,
    const int32_t rhs_offset
)

s8 Vector by Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s8
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side vector
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`kernel_sum`	`const int32_t *`	in	Kernel sums of the kernels (rhs). See arm_vector_sum_s8 for more info.
`bias`	`const int32_t *`	in	Input bias
`dst`	`int8_t *`	out	Output vector
`lhs_offset`	`const int32_t`	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
`dst_offset`	`const int32_t`	in	Offset to be added to the output values. Range: -127 to 128
`dst_multiplier`	`const int32_t`	in	Output multiplier
`dst_shift`	`const int32_t`	in	Output shift
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8
`address_offset`	`const int32_t`	in	Memory position offset for dst. First output is stored at 'dst', the second at 'dst + address_offset' and so on. Default value is typically 1.
`rhs_offset`	`const int32_t`	in	Offset to be added to the input values of the right-hand side vector. Range: -127 to 128

Returns

Returns of arm_nn_vec_mat_mult_t_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_per_ch_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *kernel_sum,
    const int32_t *bias,
    int8_t *dst,
    const int32_t lhs_offset,
    const int32_t dst_offset,
    const int32_t *dst_multiplier,
    const int32_t *dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t address_offset,
    const int32_t rhs_offset
)

s8 Vector by Matrix (transposed) multiplication using per channel quantization for output

Parameters

Parameters of arm_nn_vec_mat_mult_t_per_ch_s8
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side vector
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`kernel_sum`	`const int32_t *`	in	Kernel sums of the kernels (rhs). See arm_vector_sum_s8 for more info.
`bias`	`const int32_t *`	in	Input bias
`dst`	`int8_t *`	out	Output vector
`lhs_offset`	`const int32_t`	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
`dst_offset`	`const int32_t`	in	Offset to be added to the output values. Range: -127 to 128
`dst_multiplier`	`const int32_t *`	in	Output multipliers
`dst_shift`	`const int32_t *`	in	Output shifts
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8
`address_offset`	`const int32_t`	in	Memory position offset for dst. First output is stored at 'dst', the second at 'dst + address_offset' and so on. Default value is typically 1.
`rhs_offset`	`const int32_t`	in	Offset to be added to the input values of the right-hand side vector. Range: -127 to 128

Returns

Returns of arm_nn_vec_mat_mult_t_per_ch_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const int64_t *bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s16 Vector by s8 Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s16
Name	Type	Direction	Description
`lhs`	`const int16_t *`	in	Input left-hand side vector
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`bias`	`const int64_t *`	in	Input bias
`dst`	`int16_t *`	out	Output vector
`dst_multiplier`	`const int32_t`	in	Output multiplier
`dst_shift`	`const int32_t`	in	Output shift
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int16
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_s16
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_per_ch_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const int64_t *bias,
    int16_t *dst,
    const int32_t *dst_multiplier,
    const int32_t *dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s16 vector(lhs) by s8 matrix (transposed) multiplication and per channel quant output

Parameters

Parameters of arm_nn_vec_mat_mult_t_per_ch_s16
Name	Type	Direction	Description
`lhs`	`const int16_t *`	in	Input left-hand side vector
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`bias`	`const int64_t *`	in	Input bias
`dst`	`int16_t *`	out	Output vector
`dst_multiplier`	`const int32_t *`	in	Per channel output multiplier. Length of vector is equal to rhs_rows
`dst_shift`	`const int32_t *`	in	Per channel output shift. Length of vector is equal to rhs_rows
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int16
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_per_ch_s16
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_s16_s16(
    const int16_t *lhs,
    const int16_t *rhs,
    const int64_t *bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s16 Vector by s16 Matrix (transposed) multiplication

Parameters

Parameters of arm_nn_vec_mat_mult_t_s16_s16
Name	Type	Direction	Description
`lhs`	`const int16_t *`	in	Input left-hand side vector
`rhs`	`const int16_t *`	in	Input right-hand side matrix (transposed)
`bias`	`const int64_t *`	in	Input bias
`dst`	`int16_t *`	out	Output vector
`dst_multiplier`	`const int32_t`	in	Output multiplier
`dst_shift`	`const int32_t`	in	Output shift
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int16
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_s16_s16
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mult_t_svdf_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    int16_t *dst,
    const int32_t lhs_offset,
    const int32_t scatter_offset,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t activation_min,
    const int32_t activation_max
)

s8 Vector by Matrix (transposed) multiplication with s16 output

Parameters

Parameters of arm_nn_vec_mat_mult_t_svdf_s8
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side vector
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`dst`	`int16_t *`	out	Output vector
`lhs_offset`	`const int32_t`	in	Offset to be added to the input values of the left-hand side vector. Range: -127 to 128
`scatter_offset`	`const int32_t`	in	Address offset for dst. First output is stored at 'dst', the second at 'dst + scatter_offset' and so on.
`dst_multiplier`	`const int32_t`	in	Output multiplier
`dst_shift`	`const int32_t`	in	Output shift
`rhs_cols`	`const int32_t`	in	Number of columns in the right-hand side input matrix
`rhs_rows`	`const int32_t`	in	Number of rows in the right-hand side input matrix
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int16
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int16

Returns

Returns of arm_nn_vec_mat_mult_t_svdf_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_depthwise_conv_nt_t_padded_s8(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t lhs_offset,
    const int32_t active_ch,
    const int32_t total_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in padded cases where the padding is -lhs_offset(Range: int8). Dimensions are the same for lhs and rhs.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_padded_s8
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side matrix
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`lhs_offset`	`const int32_t`	in	LHS matrix offset(input offset). Range: -127 to 128
`active_ch`	`const int32_t`	in	Subset of total_ch processed
`total_ch`	`const int32_t`	in	Number of channels in LHS/RHS
`out_shift`	`const int32_t *`	in	Per channel output shift. Length of vector is equal to number of channels
`out_mult`	`const int32_t *`	in	Per channel output multiplier. Length of vector is equal to number of channels
`out_offset`	`const int32_t`	in	Offset to be added to the output values. Range: -127 to 128
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8
`row_x_col`	`const uint16_t`	in	(row_dimension * col_dimension) of LHS/RHS matrix
`output_bias`	`const int32_t *const`	in	Per channel output bias. Length of vector is equal to number of channels
`out`	`int8_t *`	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_padded_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS` if an implementation is available or `ARM_CMSIS_NN_NO_IMPL_ERROR` otherwise

arm_cmsis_nn_status arm_nn_depthwise_conv_nt_t_s8(
    const int32_t *weight_sum_buf,
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t lhs_offset,
    const int32_t active_ch,
    const int32_t total_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in non-padded cases. Dimensions are the same for lhs and rhs.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_s8
Name	Type	Direction	Description
`weight_sum_buf`	`const int32_t *`	in	Pointer to the weight sum multiplied by lhs_offset and summed bias buffer
`lhs`	`const int8_t *`	in	Input left-hand side matrix
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`lhs_offset`	`const int32_t`	in	LHS matrix offset(input offset). Range: -127 to 128
`active_ch`	`const int32_t`	in	Subset of total_ch processed
`total_ch`	`const int32_t`	in	Number of channels in LHS/RHS
`out_shift`	`const int32_t *`	in	Per channel output shift. Length of vector is equal to number of channels.
`out_mult`	`const int32_t *`	in	Per channel output multiplier. Length of vector is equal to number of channels.
`out_offset`	`const int32_t`	in	Offset to be added to the output values. Range: -127 to 128
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8
`row_x_col`	`const uint16_t`	in	(row_dimension * col_dimension) of LHS/RHS matrix
`output_bias`	`const int32_t *const`	in	Per channel output bias. Length of vector is equal to number of channels.
`out`	`int8_t *`	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_s8
Description
The function returns `ARM_CMSIS_NN_SUCCESS` if an implementation is available or `ARM_CMSIS_NN_NO_IMPL_ERROR` otherwise

static int32_t arm_nn_depthwise_conv_s8_planar_candidate(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims
)

Necessary conditions of the planar rule that are cheap to test inline: at most 32 channels and stride 1. A caller can skip arm_nn_depthwise_conv_s8_planar() for layers that fail them without changing which layers it takes.

Parameters

Parameters of arm_nn_depthwise_conv_s8_planar_candidate
Name	Type	Direction	Description
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [1, H, W, C_IN]

Returns

Returns of arm_nn_depthwise_conv_s8_planar_candidate
Description
1 when the layer may take the planar path, 0 when it cannot.

static int32_t arm_nn_is_convolve_s8_1x1_short_k(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims
)

The gate of arm_convolve_1x1_s8_short_k(): a 1x1 kernel with no padding, unit stride and dilation, filter depth equal to the input depth, an input depth of 1 to 16, and N, H and W positive. Plain C; it evaluates the same on every build. The entry also declines C_OUT not positive or unlike the filter’s output depth, and N x H x W x C_OUT, C_OUT x C_IN or C_OUT x 4 (the weight sums’ bytes) above INT32_MAX.

Parameters

Parameters of arm_nn_is_convolve_s8_1x1_short_k
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [C_OUT, 1, 1, C_IN]

Returns

Returns of arm_nn_is_convolve_s8_1x1_short_k
Description
1 when the layer is in the gate, 0 otherwise.

static int32_t arm_nn_is_convolve_s8_small_cin(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims,
    const cmsis_nn_dims *upscale_dims
)

The gate of arm_convolve_s8_small_cin(): upscale_dims NULL, input depth 1 to 3 with filter depth equal to it, dilation 1, a kernel of at least 1x1 with kernel width x depth at most 16 and at most 48 values, and a positive multiple of 4 output channels. Plain C; it evaluates the same on every build.

Parameters

Parameters of arm_nn_is_convolve_s8_small_cin
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [N, H, W, C_OUT]
`upscale_dims`	`const cmsis_nn_dims *`	in	Upscale tensor dimensions, or NULL

Returns

Returns of arm_nn_is_convolve_s8_small_cin
Description
1 when the layer is in the gate, 0 otherwise.

static int32_t arm_nn_is_convolve_s8_3x3_c16_s1(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *upscale_dims
)

The gate of arm_convolve_s8_3x3_c16_s1(): upscale_dims NULL, input and filter depth 16, a 3x3 kernel, and stride and dilation 1. Plain C; it evaluates the same on every build.

Parameters

Parameters of arm_nn_is_convolve_s8_3x3_c16_s1
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
`upscale_dims`	`const cmsis_nn_dims *`	in	Upscale tensor dimensions, or NULL

Returns

Returns of arm_nn_is_convolve_s8_3x3_c16_s1
Description
1 when the layer is in the gate, 0 otherwise.

static int32_t arm_nn_convolve_groups_invalid(
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

The group check of the grouped convolutions: C_IN and the filter C are positive, C_OUT is not negative, C_IN is a whole number of filter depths, and C_OUT a whole number of groups (groups = C_IN / filter C). Nothing is divided before the depths are known to be positive.

Parameters

Parameters of arm_nn_convolve_groups_invalid
Name	Type	Direction	Description
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [N, H, W, C_OUT]

Returns

Returns of arm_nn_convolve_groups_invalid
Description
1 when the dims do not describe whole groups, 0 otherwise.

int32_t arm_nn_convolve_s8_args_invalid(
    const cmsis_nn_conv_params *conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

The argument check of arm_convolve_s8() and its direct entries. arm_nn_convolve_groups_invalid() must return 0, and every value the kernels keep or form in a narrower type must fit it (AmbiqAI/ns-cmsis-nn#707):

C_IN, C_OUT, the input, filter and output W and H, padding and stride are kept as uint16_t. A value is in [0, UINT16_MAX] exactly when no bit above bit 15 is set, so they are checked with one OR; the filter C, a divisor of C_IN, then fits too.
The patch KW * KH * C is at most INT32_MAX / 4, since scratch offsets are formed from it in int32_t, and the filter, C_OUT patches, and the input and output, H * W and H * W * C each, at most INT32_MAX.
A tap index, stride * output index - padding + dilation * kernel index, is formed in int32_t: output W times stride W is at most INT32_MAX / 2 and (KW - 1) times dilation W lies within +-INT32_MAX / 2, and the same for H. Dilation is kept as int32_t, so on a 1x1 kernel any value is accepted.

Parameters

Parameters of arm_nn_convolve_s8_args_invalid
Name	Type	Direction	Description
`conv_params`	`const cmsis_nn_conv_params *`	in	Convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [C_OUT, HK, WK, CK]
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [N, H, W, C_OUT]

Returns

Returns of arm_nn_convolve_s8_args_invalid
Description
1 when `arm_convolve_s8()` reports the arguments as an error, 0 otherwise.

static int32_t arm_nn_depthwise_s16_tap_index_invalid(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *output_dims
)

Whether the generic kernels of arm_depthwise_conv_s16(), arm_depthwise_conv_s8() and arm_depthwise_conv_s4() and the DSP path of arm_depthwise_conv_fast_s16() could wrap the first tap index of an output, stride * output index - padding, which they keep as int16_t (the s16 MVE kernel keeps it as int32_t): it stays in [-INT16_MAX, INT16_MAX] when padding and stride are in [0, INT16_MAX] and (output size - 1) * stride - padding is at most INT16_MAX, in W and in H. See AmbiqAI/ns-cmsis-nn#727 and #786.

Parameters

Parameters of arm_nn_depthwise_s16_tap_index_invalid
Name	Type	Direction	Description
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [N, H, W, C_OUT]

Returns

Returns of arm_nn_depthwise_s16_tap_index_invalid
Description
1 when the index could wrap, 0 otherwise.

static int32_t arm_nn_depthwise_conv_s16_index_invalid(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Whether the int32_t index arithmetic of the s16 depthwise kernels could overflow for a layer: the MVE kernel shared by arm_depthwise_conv_s16() and arm_depthwise_conv_fast_s16(), and the DSP kernel of the latter. Every dimension, ch_mult, padding, stride and dilation must be in [0, INT32_MAX / 4], so a sum of four of them fits int32_t; stride and dilation at least 1; the input and output planes H * W and the input, filter and output (C * ch_mult channels) at most INT32_MAX elements; a kernel size times its dilation and an output size times its stride at most INT32_MAX / 2; C * ch_mult at most INT32_MAX / 2; and the MVE kernel’s other steps at most INT32_MAX: filter W * H, filter W * C * ch_mult, stride W * C, and max(KW, 1) * dilation W * C and max(KH, 1) * dilation H * W * max(C, 1) for its tap rows. The kernels form these in int32_t and divide by the stride and dilation. See AmbiqAI/ns-cmsis-nn#707 and #787.

Parameters

Parameters of arm_nn_depthwise_conv_s16_index_invalid
Name	Type	Direction	Description
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [N, H, W, C_OUT]

Returns

Returns of arm_nn_depthwise_conv_s16_index_invalid
Description
1 when an index could overflow, 0 otherwise.

static int32_t arm_nn_depthwise_conv_s16_args_invalid(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Whether the generic kernel of arm_depthwise_conv_s16() cannot take a layer that arm_nn_depthwise_conv_s16_index_invalid() accepts, or the generic kernel of arm_depthwise_conv_s8() cannot take a layer: they take every dimension, ch_mult, padding, stride and dilation as uint16_t, so each must be in [0, UINT16_MAX]. See AmbiqAI/ns-cmsis-nn#707 and #786.

Parameters

Parameters of arm_nn_depthwise_conv_s16_args_invalid
Name	Type	Direction	Description
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [N, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [N, H, W, C_OUT]

Returns

Returns of arm_nn_depthwise_conv_s16_args_invalid
Description
1 when a value is outside [0, UINT16_MAX], 0 otherwise.

int32_t arm_nn_depthwise_conv_s8_planar_bytes(
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_dims *input_dims,
    const cmsis_nn_dims *filter_dims,
    const cmsis_nn_dims *output_dims
)

Plane size in bytes that arm_nn_depthwise_conv_s8_planar() needs for a layer, or -1 when the layer is not one it takes. The rule is plain C and evaluates the same on every build.

Parameters

Parameters of arm_nn_depthwise_conv_s8_planar_bytes
Name	Type	Direction	Description
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [1, H, W, C_IN]
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [1, H, W, C_OUT]

Returns

Returns of arm_nn_depthwise_conv_s8_planar_bytes
Description
The plane size in bytes, or -1.

arm_cmsis_nn_status arm_nn_depthwise_conv_s8_planar(
    const cmsis_nn_context *ctx,
    const cmsis_nn_context *weight_sum_ctx,
    const cmsis_nn_dw_conv_params *dw_conv_params,
    const cmsis_nn_per_channel_quant_params *quant_params,
    const cmsis_nn_dims *input_dims,
    const int8_t *input,
    const cmsis_nn_dims *filter_dims,
    const int8_t *kernel,
    const cmsis_nn_dims *output_dims,
    int8_t *output
)

s8 depthwise convolution with channel multiplier 1 and stride 1, vectorized across the output pixels of one channel plane instead of across channels. It serves the few-channel and 1xk layers of arm_depthwise_conv_s8_opt(), with the same scratch buffer and weight sums.

Parameters

Parameters of arm_nn_depthwise_conv_s8_planar
Name	Type	Direction	Description
`ctx`	`const cmsis_nn_context *`	in, out	Scratch buffer of `arm_depthwise_conv_s8_opt_get_buffer_size()` bytes
`weight_sum_ctx`	`const cmsis_nn_context *`	in	Per-channel weight sums from `arm_depthwise_convolve_weight_sum()`, bias included
`dw_conv_params`	`const cmsis_nn_dw_conv_params *`	in	Depthwise convolution parameters
`quant_params`	`const cmsis_nn_per_channel_quant_params *`	in	Per-channel quantization parameters
`input_dims`	`const cmsis_nn_dims *`	in	Input tensor dimensions. Format: [1, H, W, C_IN]
`input`	`const int8_t *`	in	Input data pointer
`filter_dims`	`const cmsis_nn_dims *`	in	Filter tensor dimensions. Format: [1, H, W, C_OUT]
`kernel`	`const int8_t *`	in	Filter data pointer
`output_dims`	`const cmsis_nn_dims *`	in	Output tensor dimensions. Format: [1, H, W, C_OUT]
`output`	`int8_t *`	out	Output data pointer

Returns

Returns of arm_nn_depthwise_conv_s8_planar
Description
`ARM_CMSIS_NN_SUCCESS` when the layer was computed, or `ARM_CMSIS_NN_NO_IMPL_ERROR` when it is not one this path takes or its plane does not fit in ctx->size (then nothing is written), or MVE is not available.

arm_cmsis_nn_status arm_nn_depthwise_conv_nt_t_s4(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t lhs_offset,
    const int32_t active_ch,
    const int32_t total_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int32_t *const output_bias,
    int8_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in non-padded cases. rhs consists of packed int4 data. Dimensions are the same for lhs and rhs.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_s4
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side matrix
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed). Consists of int4 data packed in an int8 buffer.
`lhs_offset`	`const int32_t`	in	LHS matrix offset(input offset). Range: -127 to 128
`active_ch`	`const int32_t`	in	Subset of total_ch processed
`total_ch`	`const int32_t`	in	Number of channels in LHS/RHS
`out_shift`	`const int32_t *`	in	Per channel output shift. Length of vector is equal to number of channels.
`out_mult`	`const int32_t *`	in	Per channel output multiplier. Length of vector is equal to number of channels.
`out_offset`	`const int32_t`	in	Offset to be added to the output values. Range: -127 to 128
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8
`row_x_col`	`const uint16_t`	in	(row_dimension * col_dimension) of LHS/RHS matrix
`output_bias`	`const int32_t *const`	in	Per channel output bias. Length of vector is equal to number of channels.
`out`	`int8_t *`	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_s4
Description
The function returns one of the two - Updated output pointer if an implementation is available - NULL if no implementation is available.

int16_t * arm_nn_depthwise_conv_nt_t_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const uint16_t num_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t activation_min,
    const int32_t activation_max,
    const uint16_t row_x_col,
    const int64_t *const output_bias,
    int16_t *out
)

Depthwise convolution of transposed rhs matrix with 4 lhs matrices. To be used in non-padded cases. Dimensions are the same for lhs and rhs.

Parameters

Parameters of arm_nn_depthwise_conv_nt_t_s16
Name	Type	Direction	Description
`lhs`	`const int16_t *`	in	Input left-hand side matrix
`rhs`	`const int8_t *`	in	Input right-hand side matrix (transposed)
`num_ch`	`const uint16_t`	in	Number of channels in LHS/RHS
`out_shift`	`const int32_t *`	in	Per channel output shift. Length of vector is equal to number of channels.
`out_mult`	`const int32_t *`	in	Per channel output multiplier. Length of vector is equal to number of channels.
`activation_min`	`const int32_t`	in	Minimum value to clamp the output to. Range: int8
`activation_max`	`const int32_t`	in	Maximum value to clamp the output to. Range: int8
`row_x_col`	`const uint16_t`	in	(row_dimension * col_dimension) of LHS/RHS matrix
`output_bias`	`const int64_t *const`	in	Per channel output bias. Length of vector is equal to number of channels.
`out`	`int16_t *`	out	Output pointer

Returns

Returns of arm_nn_depthwise_conv_nt_t_s16
Description
The function returns one of the two - Updated output pointer if an implementation is available - NULL if no implementation is available.

arm_cmsis_nn_status arm_nn_transpose_conv_row_s8_s32(
    const int8_t *lhs,
    const int8_t *rhs,
    int32_t *output_start,
    const int32_t output_index,
    const int32_t output_max,
    const int32_t rhs_rows,
    const int32_t rhs_cols,
    const int32_t input_channels,
    const int32_t output_channels,
    const int32_t lhs_offset,
    const int32_t row_offset,
    const int32_t input_x,
    const int32_t stride_x,
    const int32_t skip_row_top,
    const int32_t skip_row_bottom
)

Row of s8 scalars multiplicated with a s8 matrix ad accumulated into a s32 rolling scratch buffer. Helpfunction for transposed convolution.

Parameters

Parameters of arm_nn_transpose_conv_row_s8_s32
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Input left-hand side scalars
`rhs`	`const int8_t *`	in	Input right-hand side matrix
`output_start`	`int32_t *`	out	Output buffer start
`output_index`	`const int32_t`	in	Output buffer current index
`output_max`	`const int32_t`	in	Output buffer size
`rhs_rows`	`const int32_t`	in	Number of rows in rhs matrix
`rhs_cols`	`const int32_t`	in	Number of columns in rhs matrix
`input_channels`	`const int32_t`	in	Number of input channels
`output_channels`	`const int32_t`	in	Number of output channels
`lhs_offset`	`const int32_t`	in	Offset added to lhs before multiplication
`row_offset`	`const int32_t`	in	Address offset between each row of data output
`input_x`	`const int32_t`	in	Length of lhs scalar row.
`stride_x`	`const int32_t`	in	Address offset between each scalar-matrix multiplication result.
`skip_row_top`	`const int32_t`	in	Skip rows on top of the filter, used for padding.
`skip_row_bottom`	`const int32_t`	in	Skip rows in the bottom of the filter, used for padding.

Returns

Returns of arm_nn_transpose_conv_row_s8_s32
Description
The function returns ARM_CMSIS_NN_SUCCESS

static int32_t arm_nn_read_q15x2_ia(const int16_t **in_q15)

Read 2 s16 elements and post increment pointer.

Parameters

Parameters of arm_nn_read_q15x2_ia
Name	Type	Direction	Description
`in_q15`	`const int16_t **`	in, out	Pointer to pointer that holds address of input. Advanced past the elements read.

Returns

Returns of arm_nn_read_q15x2_ia
Description
q31 value

static int32_t arm_nn_read_s8x4_ia(const int8_t **in_s8)

Read 4 s8 from s8 pointer and post increment pointer.

Parameters

Parameters of arm_nn_read_s8x4_ia
Name	Type	Direction	Description
`in_s8`	`const int8_t **`	in, out	Pointer to pointer that holds address of input. Advanced past the elements read.

Returns

Returns of arm_nn_read_s8x4_ia
Description
q31 value

static int32_t arm_nn_read_s8x2_ia(const int8_t **in_s8)

Read 2 s8 from s8 pointer and post increment pointer.

Parameters

Parameters of arm_nn_read_s8x2_ia
Name	Type	Direction	Description
`in_s8`	`const int8_t **`	in, out	Pointer to pointer that holds address of input. Advanced past the elements read.

Returns

Returns of arm_nn_read_s8x2_ia
Description
q31 value

static int32_t arm_nn_read_s16x2(const int16_t *in)

Read 2 int16 values from int16 pointer.

Parameters

Parameters of arm_nn_read_s16x2
Name	Type	Direction	Description
`in`	`const int16_t *`	in	pointer to address of input.

Returns

Returns of arm_nn_read_s16x2
Description
s32 value

static int32_t arm_nn_read_s8x4(const int8_t *in_s8)

Read 4 s8 values.

Parameters

Parameters of arm_nn_read_s8x4
Name	Type	Direction	Description
`in_s8`	`const int8_t *`	in	pointer to address of input.

Returns

Returns of arm_nn_read_s8x4
Description
s32 value

static int32_t arm_nn_read_s8x2(const int8_t *in_s8)

Read 2 s8 values.

Parameters

Parameters of arm_nn_read_s8x2
Name	Type	Direction	Description
`in_s8`	`const int8_t *`	in	pointer to address of input.

Returns

Returns of arm_nn_read_s8x2
Description
s32 value

static void arm_nn_write_s8x4_ia(int8_t **in, int32_t value)

Write four s8 to s8 pointer and increment pointer afterwards.

Parameters

Parameters of arm_nn_write_s8x4_ia
Name	Type	Direction	Description
`in`	`int8_t **`	in, out	Double pointer to destination. Advanced past the bytes written.
`value`	`int32_t`	in	Four bytes to copy

#define ARM_NN_DEPTH1_STORE (0)

static void arm_memset_s8(int8_t *dst, const int8_t val, uint32_t block_size)

memset, a vector loop on MVE and a word loop where ARM_NN_WORD_COPY is defined

Parameters

Parameters of arm_memset_s8
Name	Type	Direction	Description
`dst`	`int8_t *`	in, out	Destination pointer
`val`	`const int8_t`	in	Value to set
`block_size`	`uint32_t`	in	Number of bytes to copy.

static void arm_memset_s16(int16_t *dst, const int16_t val, uint32_t block_size)

memset for 16-bit data, a vector loop on MVE and a word loop where ARM_NN_WORD_COPY is defined

Parameters

Parameters of arm_memset_s16
Name	Type	Direction	Description
`dst`	`int16_t *`	in, out	Destination pointer.
`val`	`const int16_t`	in	16-bit value to set.
`block_size`	`uint32_t`	in	Number of int16_t values to set.

int8_t * arm_nn_mat_mult_kernel_s4_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const uint16_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int32_t activation_min,
    const int32_t activation_max,
    const int32_t num_col_a,
    const int32_t *const output_bias,
    int8_t *out_0
)

Matrix-multiplication function for convolution with per-channel requantization and 4 bit weights.

Parameters

Parameters of arm_nn_mat_mult_kernel_s4_s16
Name	Type	Direction	Description
`input_a`	`const int8_t *`	in	pointer to operand A, int8 packed with 2x int4.
`input_b`	`const int16_t *`	in	pointer to operand B, always consists of 2 vectors.
`output_ch`	`const uint16_t`	in	number of rows of A
`out_shift`	`const int32_t *`	in	pointer to per output channel requantization shift parameter.
`out_mult`	`const int32_t *`	in	pointer to per output channel requantization multiplier parameter.
`out_offset`	`const int32_t`	in	output tensor offset.
`activation_min`	`const int32_t`	in	minimum value to clamp the output to. Range : int8
`activation_max`	`const int32_t`	in	maximum value to clamp the output to. Range : int8
`num_col_a`	`const int32_t`	in	number of columns of A
`output_bias`	`const int32_t *const`	in	per output channel bias. Range : int32
`out_0`	`int8_t *`	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_kernel_s4_s16
Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

int8_t * arm_nn_mat_mult_kernel_s8_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const uint16_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int16_t activation_min,
    const int16_t activation_max,
    const int32_t num_col_a,
    const int32_t aligned_num_col_a,
    const int32_t *const output_bias,
    int8_t *out_0
)

Matrix-multiplication function for convolution with per-channel requantization.

Parameters

Parameters of arm_nn_mat_mult_kernel_s8_s16
Name	Type	Direction	Description
`input_a`	`const int8_t *`	in	pointer to operand A
`input_b`	`const int16_t *`	in	pointer to operand B, always consists of 2 vectors.
`output_ch`	`const uint16_t`	in	number of rows of A
`out_shift`	`const int32_t *`	in	pointer to per output channel requantization shift parameter.
`out_mult`	`const int32_t *`	in	pointer to per output channel requantization multiplier parameter.
`out_offset`	`const int32_t`	in	output tensor offset.
`activation_min`	`const int16_t`	in	minimum value to clamp the output to. Range : int8
`activation_max`	`const int16_t`	in	maximum value to clamp the output to. Range : int8
`num_col_a`	`const int32_t`	in	number of columns of A
`aligned_num_col_a`	`const int32_t`	in	number of columns of A aligned by 4
`output_bias`	`const int32_t *const`	in	per output channel bias. Range : int32
`out_0`	`int8_t *`	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_kernel_s8_s16
Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

int8_t * arm_nn_mat_mult_kernel_row_offset_s8_s16(
    const int8_t *input_a,
    const int16_t *input_b,
    const uint16_t output_ch,
    const int32_t *out_shift,
    const int32_t *out_mult,
    const int32_t out_offset,
    const int16_t activation_min,
    const int16_t activation_max,
    const int32_t num_col_a,
    const int32_t aligned_num_col_a,
    const int32_t *const output_bias,
    const int32_t row_address_offset,
    int8_t *out_0
)

Matrix-multiplication function for convolution with per-channel requantization, supporting an address offset between rows.

This function is slighly less performant than arm_nn_mat_mult_kernel_s8_s16, but allows support for grouped convolution. Supported framework: TensorFlow Lite micro.

Parameters

Parameters of arm_nn_mat_mult_kernel_row_offset_s8_s16
Name	Type	Direction	Description
`input_a`	`const int8_t *`	in	pointer to operand A
`input_b`	`const int16_t *`	in	pointer to operand B, always consists of 2 vectors.
`output_ch`	`const uint16_t`	in	number of rows of A
`out_shift`	`const int32_t *`	in	pointer to per output channel requantization shift parameter.
`out_mult`	`const int32_t *`	in	pointer to per output channel requantization multiplier parameter.
`out_offset`	`const int32_t`	in	output tensor offset.
`activation_min`	`const int16_t`	in	minimum value to clamp the output to. Range : int8
`activation_max`	`const int16_t`	in	maximum value to clamp the output to. Range : int8
`num_col_a`	`const int32_t`	in	number of columns of A
`aligned_num_col_a`	`const int32_t`	in	number of columns of A aligned by 4
`output_bias`	`const int32_t *const`	in	per output channel bias. Range : int32
`row_address_offset`	`const int32_t`	in	address offset between rows in the output
`out_0`	`int8_t *`	in, out	pointer to output

Returns

Returns of arm_nn_mat_mult_kernel_row_offset_s8_s16
Description
The function returns one of the two 1. The incremented output pointer for a successful operation or 2. NULL if implementation is not available.

void arm_nn_softmax_common_s8(
    const int8_t *input,
    const int32_t num_rows,
    const int32_t row_size,
    const int32_t mult,
    const int32_t shift,
    const int32_t diff_min,
    const bool int16_output,
    void *output
)

Common softmax function for s8 input and s8 or s16 output.

Parameters

Parameters of arm_nn_softmax_common_s8
Name	Type	Direction	Description
`input`	`const int8_t *`	in	Pointer to the input tensor
`num_rows`	`const int32_t`	in	Number of rows in the input tensor
`row_size`	`const int32_t`	in	Number of elements in each input row
`mult`	`const int32_t`	in	Input quantization multiplier
`shift`	`const int32_t`	in	Input quantization shift within the range [0, 31]
`diff_min`	`const int32_t`	in	Minimum difference with max in row. Used to check if the quantized exponential operation can be performed
`int16_output`	`const bool`	in	Indicating s8 output if 0 else s16 output
`output`	`void *`	out	Pointer to the output tensor

#define NN_ROUND(out_shift) ((0x1 << out_shift) >> 1)

macro for adding rounding offset

Parameters

Parameters of NN_ROUND
Name	Description
`out_shift`

#define MUL_SAT(a, b) arm_nn_doubling_high_mult((a), (b))

Parameters

Parameters of MUL_SAT
Name	Description
`a`
`b`

#define MUL_SAT_MVE(a, b) arm_doubling_high_mult_mve_32x4((a), (b))

Parameters

Parameters of MUL_SAT_MVE
Name	Description
`a`
`b`

#define MUL_POW2(a, b) arm_nn_mult_by_power_of_two((a), (b))

Parameters

Parameters of MUL_POW2
Name	Description
`a`
`b`

#define DIV_POW2(a, b) arm_nn_divide_by_power_of_two((a), (b))

Parameters

Parameters of DIV_POW2
Name	Description
`a`
`b`

#define DIV_POW2_MVE(a, b) arm_divide_by_power_of_two_mve((a), (b))

Parameters

Parameters of DIV_POW2_MVE
Name	Description
`a`
`b`

#define EXP_ON_NEG(x) arm_nn_exp_on_negative_values((x))

Parameters

Parameters of EXP_ON_NEG
Name	Description
`x`

#define ONE_OVER1(x) arm_nn_one_over_one_plus_x_for_x_in_0_1((x))

Parameters

Parameters of ONE_OVER1
Name	Description
`x`

static int32_t arm_nn_doubling_high_mult(const int32_t m1, const int32_t m2)

Saturating doubling high multiply. Result matches NEON instruction VQRDMULH.

Parameters

Parameters of arm_nn_doubling_high_mult
Name	Type	Direction	Description
`m1`	`const int32_t`	in	Multiplicand. Range: {NN_Q31_MIN, NN_Q31_MAX}
`m2`	`const int32_t`	in	Multiplier. Range: {NN_Q31_MIN, NN_Q31_MAX}

Returns

Returns of arm_nn_doubling_high_mult
Description
Result of multiplication.

static int32_t arm_nn_doubling_high_mult_no_sat(int32_t m1, int32_t m2)

Doubling high multiply without saturation. This is intended for requantization where the scale is a positive integer.

Parameters

Parameters of arm_nn_doubling_high_mult_no_sat
Name	Type	Direction	Description
`m1`	`int32_t`	in	Multiplicand. Range: {NN_Q31_MIN, NN_Q31_MAX}
`m2`	`int32_t`	in	Multiplier Range: {NN_Q31_MIN, NN_Q31_MAX}

Returns

Returns of arm_nn_doubling_high_mult_no_sat
Description
Result of multiplication.

static int32_t arm_nn_divide_by_power_of_two(const int32_t dividend, const int32_t exponent)

Rounding divide by power of two.

Parameters

Parameters of arm_nn_divide_by_power_of_two
Name	Type	Direction	Description
`dividend`	`const int32_t`	in	- Dividend
`exponent`	`const int32_t`	in	- Divisor = power(2, exponent) Range: [0, 31]

Returns

Returns of arm_nn_divide_by_power_of_two
Description
Rounded result of division. Midpoint is rounded away from zero.

static int32_t arm_nn_nonneg_divide_by_pot_s32(int32_t dividend, int32_t exponent)

Rounding divide by power of two for non-negative values.

Parameters

Parameters of arm_nn_nonneg_divide_by_pot_s32
Name	Type	Direction	Description
`dividend`	`int32_t`	in	- Dividend (assumed to be non-negative)
`exponent`	`int32_t`	in	- Divisor = power(2, exponent) Range: [0, 31]

Returns

Returns of arm_nn_nonneg_divide_by_pot_s32
Description
Rounded result of division. Midpoint is rounded away from zero.

static int32_t arm_nn_mac_wrap_s32_bits(const int32_t acc, const int32_t a, const int32_t b)

Multiply-accumulate in int32_t that wraps as the hardware does instead of overflowing.

A start value near INT32_MIN or INT32_MAX, such as a bias, can take a partial sum out of the int32_t range before the full sum returns to it. Adding in uint32_t keeps that defined and gives the same bits.

Parameters

Parameters of arm_nn_mac_wrap_s32_bits
Name	Type	Direction	Description
`acc`	`const int32_t`	in	Accumulator
`a`	`const int32_t`	in	Multiplicand. Product bits are accumulated modulo 2^32
`b`	`const int32_t`	in	Multiplier

Returns

Returns of arm_nn_mac_wrap_s32_bits
Description
acc + a * b, wrapped to int32_t

static int32_t arm_nn_requantize(const int32_t val, const int32_t multiplier, const int32_t shift)

Requantize a given value.

Essentially returns (val * multiplier)/(2 ^ shift) with different rounding depending if CMSIS_NN_USE_SINGLE_ROUNDING is defined or not.

Parameters

Parameters of arm_nn_requantize
Name	Type	Direction	Description
`val`	`const int32_t`	in	Value to be requantized
`multiplier`	`const int32_t`	in	Multiplier. Range {NN_Q31_MIN + 1, Q32_MAX}
`shift`	`const int32_t`	in	Shift. Range: {-31, 30} Default branch: If shift is positive left shift 'val * multiplier' with shift If shift is negative right shift 'val * multiplier' with abs(shift) Single round branch: Input for total_shift in divide by '2 ^ total_shift'

Returns

Returns of arm_nn_requantize
Description
Default branch: Returns (val * multiplier) with rounding divided by (2 ^ shift) with rounding Single round branch: Returns (val * multiplier)/(2 ^ (31 - shift)) with rounding

static int64_t arm_nn_requantize_positive_shift_s64(const int32_t val, const int32_t multiplier, const int32_t shift)

Requantize with a positive scale exponent without narrowing the result.

Parameters

Parameters of arm_nn_requantize_positive_shift_s64
Name	Type	Direction	Description
`val`	`const int32_t`	in	Centered input value.
`multiplier`	`const int32_t`	in	Nonnegative Q31 multiplier in [0, INT32_MAX].
`shift`	`const int32_t`	in	Positive scale exponent in [1, 30].

Returns

Returns of arm_nn_requantize_positive_shift_s64
Description
Rounded result before output zero point and saturation. Both rounding modes agree for this range.

static int32_t arm_nn_requantize_s64(const int64_t val, const int32_t reduced_multiplier, const int32_t shift)

Requantize a given 64 bit value.

Parameters

Parameters of arm_nn_requantize_s64
Name	Type	Direction	Description
`val`	`const int64_t`	in	Value to be requantized in the range {-(1<<47)} to {(1<<47) - 1}
`reduced_multiplier`	`const int32_t`	in	Reduced multiplier in the range {NN_Q31_MIN + 1, Q32_MAX} to {Q16_MIN + 1, Q16_MAX}
`shift`	`const int32_t`	in	Left or right shift for 'val * multiplier' in the range {-31} to {7}

Returns

Returns of arm_nn_requantize_s64
Description
Returns (val * multiplier)/(2 ^ shift)

static int16_t arm_nn_sat_lshift_s16(int16_t x, int shift)

Saturating left shift for int16_t.

Parameters

Parameters of arm_nn_sat_lshift_s16
Name	Type	Direction	Description
`x`	`int16_t`	in	value to be shifted
`shift`	`int`	in	Nonpositive values return x; positive values multiply by 2^shift with s16 saturation.

Returns

Returns of arm_nn_sat_lshift_s16
Description
shifted value

static int16_t arm_nn_sqrdmulh_s16(int16_t a, int16_t b)

Saturating Rounding Doubling High Mul (s16).

Matches NEON SQRDMULH s16

Parameters

Parameters of arm_nn_sqrdmulh_s16
Name	Type	Direction	Description
`a`	`int16_t`	in	Multiplicand
`b`	`int16_t`	in	Multiplier

Returns

Returns of arm_nn_sqrdmulh_s16
Description
Result of multiplication.

static int16_t arm_nn_sqdmulh_s16(int16_t a, int16_t b)

Saturating Non-rounded Doubling High Mul (s16).

Matches NEON SQDMULH s16

Parameters

Parameters of arm_nn_sqdmulh_s16
Name	Type	Direction	Description
`a`	`int16_t`	in	Multiplicand
`b`	`int16_t`	in	Multiplier

Returns

Returns of arm_nn_sqdmulh_s16
Description
Result of multiplication.

static int16_t arm_nn_divide_by_power_of_two_s16(int16_t x, int exponent)

Rounding divide by power of two (s16), midpoint away from zero.

Mirrors arm_nn_divide_by_power_of_two() semantics for s16.

Parameters

Parameters of arm_nn_divide_by_power_of_two_s16
Name	Type	Direction	Description
`x`	`int16_t`	in	Dividend
`exponent`	`int`	in	Divisor = power(2, exponent) Range: [0, 15]

Returns

Returns of arm_nn_divide_by_power_of_two_s16
Description
Rounded result of division. Midpoint is rounded away from zero.

static void arm_memcpy_s8(int8_t *dst, const int8_t *src, uint32_t block_size)

memcpy, a vector loop on MVE and a word loop where ARM_NN_WORD_COPY is defined

Parameters

Parameters of arm_memcpy_s8
Name	Type	Direction	Description
`dst`	`int8_t *`	in, out	Destination pointer
`src`	`const int8_t *`	in	Source pointer.
`block_size`	`uint32_t`	in	Number of bytes to copy.

static void arm_memcpy_s16(int16_t *dst, const int16_t *src, uint32_t block_size)

memcpy of int16_t values through arm_memcpy_s8()

Parameters

Parameters of arm_memcpy_s16
Name	Type	Direction	Description
`dst`	`int16_t *`	in, out	Destination pointer
`src`	`const int16_t *`	in	Source pointer.
`block_size`	`uint32_t`	in	Number of values to copy.

static void arm_memcpy_s32(int32_t *dst, const int32_t *src, uint32_t block_size)

memcpy of int32_t values through arm_memcpy_s8()

Parameters

Parameters of arm_memcpy_s32
Name	Type	Direction	Description
`dst`	`int32_t *`	in, out	Destination pointer
`src`	`const int32_t *`	in	Source pointer.
`block_size`	`uint32_t`	in	Number of values to copy.

static void arm_memcpy_q15(int16_t *dst, const int16_t *src, uint32_t block_size)

memcpy wrapper for int16, a word loop where ARM_NN_WORD_COPY is defined

Parameters

Parameters of arm_memcpy_q15
Name	Type	Direction	Description
`dst`	`int16_t *`	in, out	Destination pointer
`src`	`const int16_t *`	in	Source pointer.
`block_size`	`uint32_t`	in	Number of bytes to copy.

static int32_t arm_nn_exp_on_negative_values(int32_t val)

Fixed-point exp() of a non-positive value.

Parameters

Parameters of arm_nn_exp_on_negative_values
Name	Type	Direction	Description
`val`	`int32_t`	in	Input in Q5.26 fixed point. Must be less than or equal to 0

Returns

Returns of arm_nn_exp_on_negative_values
Description
exp(val) in Q0.31 fixed point. Returns NN_Q31_MAX when `val` is 0.

#define SELECT_IF_NON_ZERO(x) { \ mask = MASK_IF_NON_ZERO(remainder & (1 << shift++)); \ result = SELECT_USING_MASK(mask, MUL_SAT(result, x), result); \ }

Parameters

Parameters of SELECT_IF_NON_ZERO
Name	Description
`x`

static int32_t arm_nn_mult_by_power_of_two(const int32_t val, const int32_t exp)

Saturating multiply by a power of two.

Parameters

Parameters of arm_nn_mult_by_power_of_two
Name	Type	Direction	Description
`val`	`const int32_t`	in	Value to be multiplied
`exp`	`const int32_t`	in	Exponent. Multiplier = power(2, exp)

Returns

Returns of arm_nn_mult_by_power_of_two
Description
val * 2^exp saturated to the int32 range

static int32_t arm_nn_one_over_one_plus_x_for_x_in_0_1(int32_t val)

Fixed-point 1 / (1 + x) for x in [0, 1), computed with Newton-Raphson iterations.

Parameters

Parameters of arm_nn_one_over_one_plus_x_for_x_in_0_1
Name	Type	Direction	Description
`val`	`int32_t`	in	x in Q0.31 fixed point. Range: [0, NN_Q31_MAX]

Returns

Returns of arm_nn_one_over_one_plus_x_for_x_in_0_1
Description
1 / (1 + x) in Q0.31 fixed point

static int32_t arm_nn_softmax_row_scale(const int64_t sum, const int32_t unit_bits, int32_t *bits_over_unit)

Reciprocal scale of a softmax row, from the sum of its exponentials.

A sum past 32 bits normalises from its top 32 bits. An exponent past 31 would round every non-negative quotient to 0, so it returns scale 0 with exponent 31 instead, as it does for an empty row (sum 0).

Parameters

Parameters of arm_nn_softmax_row_scale
Name	Type	Direction	Description
`sum`	`const int64_t`	in	Sum of the row's exponentials, each with the softmax kernels' 12 accumulation integer bits. Range: >= 0
`unit_bits`	`const int32_t`	in	The accumulation integer bits plus 31 less the output width: 12 + 23 for an 8-bit output, 12 + 15 for a 16-bit output
`bits_over_unit`	`int32_t *`	out	Exponent for the final DIV_POW2 of `MUL_SAT(scale, exponential)`. At most 31

Returns

Returns of arm_nn_softmax_row_scale
Description
The scale, 1 / sum in Q0.31 normalised by bits_over_unit

static void arm_nn_write_q15x2_ia(int16_t **dest_q15, int32_t src_q31)

Write 2 s16 elements and post increment pointer.

Parameters

Parameters of arm_nn_write_q15x2_ia
Name	Type	Direction	Description
`dest_q15`	`int16_t **`	in, out	Pointer to pointer that holds address of destination. Advanced past the elements written.
`src_q31`	`int32_t`	in	Input value to be written.

static void arm_nn_write_s8x2_ia(int8_t **dst, int16_t src)

Write 2 s8 elements and post increment pointer.

Parameters

Parameters of arm_nn_write_s8x2_ia
Name	Type	Direction	Description
`dst`	`int8_t **`	in, out	Pointer to pointer that holds address of destination. Advanced past the elements written.
`src`	`int16_t`	in	Input value to be written.

static int32_t arm_cmsis_nn_dim_at(const cmsis_nn_dims *dims, int32_t index)

Get dimension value at specific index.

Parameters

Parameters of arm_cmsis_nn_dim_at
Name	Type	Direction	Description
`dims`	`const cmsis_nn_dims *`	in	Pointer to `cmsis_nn_dims` structure
`index`	`int32_t`	in	Index of dimension to get

Returns

Returns of arm_cmsis_nn_dim_at
Description
Dimension value at specified index

static size_t arm_cmsis_nn_shape_product(const int32_t *shape, int32_t length)

Calculate the product of all dimensions in a shape array.

Parameters

Parameters of arm_cmsis_nn_shape_product
Name	Type	Direction	Description
`shape`	`const int32_t *`	in	Pointer to array containing shape dimensions
`length`	`int32_t`	in	Number of dimensions in the shape array

Returns

Returns of arm_cmsis_nn_shape_product
Description
Product of all dimensions

arm_cmsis_nn_status arm_nn_lstm_step_s8(
    const int8_t *data_in,
    const int8_t *hidden_in,
    int8_t *hidden_out,
    const cmsis_nn_lstm_params *params,
    cmsis_nn_lstm_context *buffers,
    const int32_t batch_offset
)

Update LSTM function for an iteration step using s8 input and output, and s16 internally.

Parameters

Parameters of arm_nn_lstm_step_s8
Name	Type	Direction	Description
`data_in`	`const int8_t *`	in	Data input pointer
`hidden_in`	`const int8_t *`	in	Hidden state/ recurrent input pointer
`hidden_out`	`int8_t *`	out	Hidden state/ recurrent output pointer
`params`	`const cmsis_nn_lstm_params *`	in	Struct containg all information about the lstm operator, see arm_nn_types.
`buffers`	`cmsis_nn_lstm_context *`	in, out	Struct containg pointers to all temporary scratch buffers needed for the lstm operator, see arm_nn_types.
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches. E.g for params->timing_major = true, all batches for t=0 are stored sequentially, so batch offset = 1. For params->time major = false, all time steps are stored continously before the next batch, so batch offset = params->time_steps.

Returns

Returns of arm_nn_lstm_step_s8
Description
The function returns ARM_CMSIS_NN_SUCCESS

arm_cmsis_nn_status arm_nn_lstm_step_s16(
    const int16_t *data_in,
    const int16_t *hidden_in,
    int16_t *hidden_out,
    const cmsis_nn_lstm_params *params,
    cmsis_nn_lstm_context *buffers,
    const int32_t batch_offset
)

Update LSTM function for an iteration step using s16 input and output, and s16 internally.

Parameters

Parameters of arm_nn_lstm_step_s16
Name	Type	Direction	Description
`data_in`	`const int16_t *`	in	Data input pointer
`hidden_in`	`const int16_t *`	in	Hidden state/ recurrent input pointer
`hidden_out`	`int16_t *`	out	Hidden state/ recurrent output pointer
`params`	`const cmsis_nn_lstm_params *`	in	Struct containg all information about the lstm operator, see arm_nn_types.
`buffers`	`cmsis_nn_lstm_context *`	in, out	Struct containg pointers to all temporary scratch buffers needed for the lstm operator, see arm_nn_types.
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches. E.g for params->timing_major = true, all batches for t=0 are stored sequentially, so batch offset = 1. For params->time major = false, all time steps are stored continously before the next batch, so batch offset = params->time_steps.

Returns

Returns of arm_nn_lstm_step_s16
Description
The function returns ARM_CMSIS_NN_SUCCESS

arm_cmsis_nn_status arm_nn_lstm_calculate_gate_s8_s16(
    const int8_t *data_in,
    const int8_t *hidden_in,
    const cmsis_nn_lstm_gate *gate_data,
    const cmsis_nn_lstm_params *params,
    int16_t *output,
    const int32_t batch_offset
)

Updates a LSTM gate for an iteration step of LSTM function, int8x8_16 version.

Parameters

Parameters of arm_nn_lstm_calculate_gate_s8_s16
Name	Type	Direction	Description
`data_in`	`const int8_t *`	in	Data input pointer
`hidden_in`	`const int8_t *`	in	Hidden state/ recurrent input pointer
`gate_data`	`const cmsis_nn_lstm_gate *`	in	Struct containing all information about the gate caluclation, see arm_nn_types.
`params`	`const cmsis_nn_lstm_params *`	in	Struct containing all information about the lstm_operation, see arm_nn_types
`output`	`int16_t *`	out	Hidden state/ recurrent output pointer
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches, see arm_nn_lstm_step_s8.

Returns

Returns of arm_nn_lstm_calculate_gate_s8_s16
Description
The function returns ARM_CMSIS_NN_SUCCESS

arm_cmsis_nn_status arm_nn_lstm_calculate_gate_s16(
    const int16_t *data_in,
    const int16_t *hidden_in,
    const cmsis_nn_lstm_gate *gate_data,
    const cmsis_nn_lstm_params *params,
    int16_t *output,
    const int32_t batch_offset
)

Updates a LSTM gate for an iteration step of LSTM function, int16x8_16 version.

Parameters

Parameters of arm_nn_lstm_calculate_gate_s16
Name	Type	Direction	Description
`data_in`	`const int16_t *`	in	Data input pointer
`hidden_in`	`const int16_t *`	in	Hidden state/ recurrent input pointer
`gate_data`	`const cmsis_nn_lstm_gate *`	in	Struct containing all information about the gate caluclation, see arm_nn_types.
`params`	`const cmsis_nn_lstm_params *`	in	Struct containing all information about the lstm_operation, see arm_nn_types
`output`	`int16_t *`	out	Hidden state/ recurrent output pointer
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches, see arm_nn_lstm_step_s16.

Returns

Returns of arm_nn_lstm_calculate_gate_s16
Description
The function returns ARM_CMSIS_NN_SUCCESS

arm_cmsis_nn_status arm_nn_vec_mat_mul_result_acc_s8_s16(
    const int8_t *lhs,
    const int8_t *rhs,
    const int32_t *effective_bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t batches,
    const int32_t batch_offset
)

The result of the multiplication is accumulated to the passed result buffer. Multiplies a matrix by a “batched” vector (i.e. a matrix with a batch dimension composed by input vectors independent from each other).

Parameters

Parameters of arm_nn_vec_mat_mul_result_acc_s8_s16
Name	Type	Direction	Description
`lhs`	`const int8_t *`	in	Batched vector
`rhs`	`const int8_t *`	in	Weights - input matrix (H(Rows)xW(Columns))
`effective_bias`	`const int32_t *`	in	Bias + lhs_offset * kernel_sum term precalculated into a constant vector.
`dst`	`int16_t *`	out	Output
`dst_multiplier`	`const int32_t`	in	Multiplier for quantization
`dst_shift`	`const int32_t`	in	Shift for quantization
`rhs_cols`	`const int32_t`	in	Vector/matarix column length
`rhs_rows`	`const int32_t`	in	Row count of matrix
`batches`	`const int32_t`	in	Batch size
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches in input, see arm_nn_lstm_step_s8. Note that the output is always stored with sequential batches.

Returns

Returns of arm_nn_vec_mat_mul_result_acc_s8_s16
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_nn_vec_mat_mul_result_acc_s16(
    const int16_t *lhs,
    const int8_t *rhs,
    const int64_t *effective_bias,
    int16_t *dst,
    const int32_t dst_multiplier,
    const int32_t dst_shift,
    const int32_t rhs_cols,
    const int32_t rhs_rows,
    const int32_t batches,
    const int32_t batch_offset
)

Parameters

Parameters of arm_nn_vec_mat_mul_result_acc_s16
Name	Type	Direction	Description
`lhs`	`const int16_t *`	in	Batched vector
`rhs`	`const int8_t *`	in	Weights - input matrix (H(Rows)xW(Columns))
`effective_bias`	`const int64_t *`	in	Bias + lhs_offset * kernel_sum term precalculated into a constant vector.
`dst`	`int16_t *`	out	Output
`dst_multiplier`	`const int32_t`	in	Multiplier for quantization
`dst_shift`	`const int32_t`	in	Shift for quantization
`rhs_cols`	`const int32_t`	in	Vector/matarix column length
`rhs_rows`	`const int32_t`	in	Row count of matrix
`batches`	`const int32_t`	in	Batch size
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches in input, see arm_nn_lstm_step_s16. Note that the output is always stored with sequential batches.

Returns

Returns of arm_nn_vec_mat_mul_result_acc_s16
Description
The function returns `ARM_CMSIS_NN_SUCCESS`

arm_cmsis_nn_status arm_elementwise_mul_s16_s8(
    const int16_t *input_1_vect,
    const int16_t *input_2_vect,
    int8_t *output,
    const int32_t out_offset,
    const int32_t out_mult,
    const int32_t out_shift,
    const int32_t block_size,
    const int32_t batch_size,
    const int32_t batch_offset
)

s16 elementwise multiplication with s8 output

Supported framework: TensorFlow Lite micro

Parameters

Parameters of arm_elementwise_mul_s16_s8
Name	Type	Direction	Description
`input_1_vect`	`const int16_t *`	in	pointer to input vector 1
`input_2_vect`	`const int16_t *`	in	pointer to input vector 2
`output`	`int8_t *`	in, out	pointer to output vector
`out_offset`	`const int32_t`	in	output offset
`out_mult`	`const int32_t`	in	output multiplier
`out_shift`	`const int32_t`	in	output shift
`block_size`	`const int32_t`	in	number of samples per batch
`batch_size`	`const int32_t`	in	number of samples per batch
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches in output, see arm_nn_lstm_step_s8. Note that it is assumed that the input is stored with sequential batches.

Returns

Returns of arm_elementwise_mul_s16_s8
Description
The function returns ARM_CMSIS_NN_SUCCESS

arm_cmsis_nn_status arm_elementwise_mul_s16_batch_offset(
    const int16_t *input_1_vect,
    const int16_t *input_2_vect,
    int16_t *output,
    const int32_t out_offset,
    const int32_t out_mult,
    const int32_t out_shift,
    const int32_t block_size,
    const int32_t batch_size,
    const int32_t batch_offset
)

s16 elementwise multiplication with s16 output

Supported framework: TensorFlow Lite micro

Parameters

Parameters of arm_elementwise_mul_s16_batch_offset
Name	Type	Direction	Description
`input_1_vect`	`const int16_t *`	in	pointer to input vector 1
`input_2_vect`	`const int16_t *`	in	pointer to input vector 2
`output`	`int16_t *`	in, out	pointer to output vector
`out_offset`	`const int32_t`	in	output offset
`out_mult`	`const int32_t`	in	output multiplier
`out_shift`	`const int32_t`	in	output shift
`block_size`	`const int32_t`	in	number of samples per batch
`batch_size`	`const int32_t`	in	number of samples per batch
`batch_offset`	`const int32_t`	in	Number of timesteps between consecutive batches in output, see arm_nn_lstm_step_s16. Note that it is assumed that the input is stored with sequential batches.

Returns

Returns of arm_elementwise_mul_s16_batch_offset
Description
The function returns ARM_CMSIS_NN_SUCCESS

arm_cmsis_nn_status arm_elementwise_mul_acc_s16(
    const int16_t *input_1_vect,
    const int16_t *input_2_vect,
    const int32_t input_1_offset,
    const int32_t input_2_offset,
    int16_t *output,
    const int32_t out_offset,
    const int32_t out_mult,
    const int32_t out_shift,
    const int32_t out_activation_min,
    const int32_t out_activation_max,
    const int32_t block_size
)

s16 elementwise multiplication. The result of the multiplication is accumulated to the passed result buffer.

Supported framework: TensorFlow Lite micro

Parameters

Parameters of arm_elementwise_mul_acc_s16
Name	Type	Direction	Description
`input_1_vect`	`const int16_t *`	in	pointer to input vector 1
`input_2_vect`	`const int16_t *`	in	pointer to input vector 2
`input_1_offset`	`const int32_t`	in	offset for input 1. Not used.
`input_2_offset`	`const int32_t`	in	offset for input 2. Not used.
`output`	`int16_t *`	in, out	pointer to output vector
`out_offset`	`const int32_t`	in	output offset. Not used.
`out_mult`	`const int32_t`	in	output multiplier
`out_shift`	`const int32_t`	in	output shift
`out_activation_min`	`const int32_t`	in	minimum value to clamp output to. Min: -32768
`out_activation_max`	`const int32_t`	in	maximum value to clamp output to. Max: 32767
`block_size`	`const int32_t`	in	number of samples

Returns

Returns of arm_elementwise_mul_acc_s16
Description
The function returns ARM_CMSIS_NN_SUCCESS

static int32_t arm_check_broadcast_required(const cmsis_nn_dims *shape_1, const cmsis_nn_dims *shape_2)

Check if a broadcast is required between 2 cmsis_nn_dims.

Compares each dimension and returns 1 if any dimension does not match. This function does not check that broadcast rules are met.

Parameters

Parameters of arm_check_broadcast_required
Name	Type	Direction	Description
`shape_1`	`const cmsis_nn_dims *`	in	pointer to input tensor 1
`shape_2`	`const cmsis_nn_dims *`	in	pointer to input tensor 2

Returns

Returns of arm_check_broadcast_required
Description
The function returns 1 if a broadcast is required, or 0 if not.

static int32_t arm_reduce_get_middle_block_from_arrays(
    const int32_t in_dims,
    const int32_t axis_arr,
    int32_t *outer,
    int32_t *reduce,
    int32_t *inner
)

Reports whether the reduced axes of a 4-D tensor form one contiguous block followed by kept axes, as in a NHWC mean over H and W, and gives the flattened sizes. Axes of size 1 are ignored.

Parameters

Parameters of arm_reduce_get_middle_block_from_arrays
Name	Type	Direction	Description
`in_dims`	`const int32_t`	in	4-element array {n, h, w, c}
`axis_arr`	`const int32_t`	in	4-element mask {axis_n, axis_h, axis_w, axis_c}
`outer`	`int32_t *`	out	Product of the dims before the reduced block
`reduce`	`int32_t *`	out	Product of the reduced dims
`inner`	`int32_t *`	out	Product of the dims after the reduced block

Returns

Returns of arm_reduce_get_middle_block_from_arrays
Description
1 if the input is [outer, reduce, inner] with the middle dim reduced and inner > 1, otherwise 0

#define ARM_NN_SQRT_S16_TABLEFREE_SHIFT 14

#define ARM_NN_SQRT_S16_TABLEFREE_MAGIC UINT32_C(0x5F5FB6C4)

#define ARM_NN_SQRT_S16_TABLEFREE_K0 (-4.76426697f)

#define ARM_NN_SQRT_S16_TABLEFREE_K1 (-48.0000114f)

static int16_t arm_nn_sqrt_s16_tablefree_element(const int32_t value, const float scale)

One element of arm_sqrt_s16_tablefree(): the float32 chain the MVE path evaluates per lane, so the two agree bit for bit on any IEEE-754 float32 implementation with round-to-nearest-even and a fused multiply-add (fmaf). Every product after the pre-scale either has two uses or feeds an fmaf or a conversion, never another lone multiply, so a compiler allowed to reassociate (-ffast-math) still has no chain to reorder, and no product feeds a bare add, so there is nothing to contract.

Parameters

Parameters of arm_nn_sqrt_s16_tablefree_element
Name	Type	Direction	Description
`value`	`const int32_t`	in	input code; values <= 0 give 0
`scale`	`const float`	in	input_scale / (output_scale * output_scale) as float32

Returns

Returns of arm_nn_sqrt_s16_tablefree_element
Description
trunc(sqrt(value * scale)) saturated to 32767

static uint32_t arm_nn_f16_bits_to_f32_bits(const uint32_t h)

Integer widening of one float16 bit pattern, with the result of the FPU’s VCVTB at reset FPSCR: exact for every finite half and Inf, and a NaN keeps its sign and payload and comes back quiet. Raises no FP flag. Half subnormals are normal singles: renormalize.

Parameters

Parameters of arm_nn_f16_bits_to_f32_bits
Name	Type	Direction	Description
`h`	`const uint32_t`	in	The binary16 bits in the low 16 bits.

Returns

Returns of arm_nn_f16_bits_to_f32_bits
Description
The binary32 bits.

static void arm_nn_dequantize_f16_bits_f32(const uint16_t *input, float *output, const int32_t block_size)

The body of arm_dequantize_f16_bits_f32() without its argument checks, inline so that a caller with a small or constant size pays no call. Each element gives what the path’s hardware half-to-single conversion gives; arm_dequantize_f16_bits_f32() documents the paths.

Parameters

Parameters of arm_nn_dequantize_f16_bits_f32
Name	Type	Direction	Description
`input`	`const uint16_t *`	in	Pointer to the binary16 bit patterns, not NULL unless block_size is 0.
`output`	`float *`	out	Pointer to the float32 output array, 4-byte aligned and not overlapping input.
`block_size`	`const int32_t`	in	Number of elements, not negative (unchecked: the vector path would store block_size & 3 elements for a negative size).