Function arm_batch_matmul_s8

Function Documentation

arm_cmsis_nn_status arm_batch_matmul_s8(const cmsis_nn_context *ctx, const cmsis_nn_bmm_params *bmm_params, const cmsis_nn_per_tensor_quant_params *quant_params, const cmsis_nn_dims *input_lhs_dims, const int8_t *input_lhs, const cmsis_nn_dims *input_rhs_dims, const int8_t *input_rhs, const cmsis_nn_dims *output_dims, int8_t *output)

Batch matmul function with 8 bit input and output.

  1. Supported framework: TensorFlow Lite Micro

  2. Performs row * row matrix multiplication with the RHS transposed.

Parameters:

  • ctx – [inout] Temporary scratch buffer for the per-row kernel sums of the RHS. Mandatory on builds with the MVE extension (ARM_MATH_MVEI), where a NULL ctx->buf is diagnosed with ARM_CMSIS_NN_ARG_ERROR. Unused on every other build, where ctx->buf may be NULL. Sized by arm_batch_matmul_s8_get_buffer_size(input_rhs_dims) - pass the same input_rhs_dims given below. That is input_rhs_dims->w * sizeof(int32_t) where the sums are used, 0 otherwise. Do not size this buffer with arm_fully_connected_s8_get_buffer_size(): it reads a different field, and an allocation short of input_rhs_dims->w words is written past its end. The function fills the buffer itself before each use, so the caller does not need to initialize it. ctx->buf must be aligned to sizeof(int32_t). If ctx->size is non-zero it is validated against the requirement and a buffer too small is rejected with ARM_CMSIS_NN_ARG_ERROR; a ctx->size of 0 skips that check. A negative input_rhs_dims->w, or one large enough that the required size exceeds INT32_MAX, is rejected with ARM_CMSIS_NN_ARG_ERROR regardless of ctx->size. The caller is expected to clear the buffer, if applicable, for security reasons.

  • bmm_params – [in] Batch matmul Parameters Adjoint flags are currently unused and do not transpose either input; callers must supply the tensors in the layouts described below.

  • quant_params – [in] Quantization parameters

  • input_lhs_dims – [in] Input lhs tensor dimensions. This s8 function treats w as the row count and c as the inner dimension. This differs from arm_batch_matmul_f32(), so its dimension mapping must not be reused here.

  • input_lhs – [in] Pointer to input tensor

  • input_rhs_dims – [in] Input rhs tensor dimensions. The RHS must already be transposed, with w as its row count and c equal to input_lhs_dims->c.

  • input_rhs – [in] Pointer to transposed input tensor

  • output_dims – [in] Output tensor dimensions

  • output – [out] Pointer to the output tensor

Returns:

The function returns one of the following:

  • ARM_CMSIS_NN_ARG_ERROR if an MVE build receives an invalid context, a negative or unrepresentable RHS row count, or a declared context size below the requirement.

  • ARM_CMSIS_NN_SUCCESS on success.