Skip to content

converter

LiteRT Converter API

This module handles converting models to LiteRT format.

Classes:

Classes

LiteRTQuantizationType

Supported quantization formats

Attributes:

  • FP32 –

    FP32 quantization

  • FP16 –

    FP16 quantization

  • INT8 –

    INT8 quantization

  • INT16X8 –

    INT16X8 quantization

Functions

from_dtype classmethod 
from_dtype(dtype: dtype)

Get quantization type from dtype string

Parameters:

  • dtype
    (dtype) –

    Dtype object

Returns:

  • LiteRTQuantizationType –

    Corresponding quantization type

LiteRTConversionType

Supported conversion types

Attributes:

  • KERAS –

    Use Keras model directly

  • SAVED_MODEL –

    Use TF Saved model format

  • CONCRETE –

    Lower to TF Concrete functions

LiteRTConverter 

LiteRTConverter(model: Model)

Converts Keras/TF model to LiteRT model.

Parameters:

  • model

    (Model) –

    Keras model

Example:

# Create simple dataset
test_x = np.random.rand(1000, 64).astype(np.float32)
test_y = np.random.randint(0, 10, 1000).astype(np.int32)

# Create a dense model and train
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation="relu", input_shape=(64,)),
    tf.keras.layers.Dense(32, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(test_x, test_y, epochs=1, validation_split=0.2)

# Create converter and convert to LiteRT w/ FP32 quantization
converter = LiteRTKerasConverter(model=model)
model_content = converter.convert(
    test_x,
    quantization=LiteRTQuantizationType.FP32,
    io_type="float32"
)
y_pred_tfl = converter.predict(test_x)
y_pred_tf = model.predict(test_x)
print(np.allclose(y_pred_tf, y_pred_tfl, atol=1e-3))

Functions

convert 
convert(rep_dataset: Callable | None = None, quantization: LiteRTQuantizationType = LiteRTQuantizationType.FP32, io_type: str | None = None, mode: LiteRTConversionType = LiteRTConversionType.KERAS, strict: bool = True, verbose: int = 2) -> str

Convert TF model into LiteRT model content

Parameters:

  • rep_dataset
    (Callable | None, default: None ) –

    Representative dataset generator. Defaults to None.

  • quantization
    (QuantizationType, default: FP32 ) –

    Quantization type. Defaults to QuantizationType.FP32.

  • io_type
    (str | None, default: None ) –

    Input/Output type. Defaults to None.

  • mode
    (ConversionType, default: KERAS ) –

    Conversion mode. Defaults to ConversionType.KERAS.

  • strict
    (bool, default: True ) –

    Strict mode. Defaults to True.

  • verbose
    (int, default: 2 ) –

    Verbosity level (0,1,2). Defaults to 2.

Returns:

  • str ( str ) –

    model content

evaluate 
evaluate(x: NDArray, y: NDArray, input_name: str | None = None, output_name: str | None = None) -> npt.NDArray

Evaluate LiteRT model

Parameters:

  • x
    (NDArray) –

    Input samples

  • y
    (NDArray) –

    Input labels

  • input_name
    (str | None, default: None ) –

    Input layer name. Defaults to None.

  • output_name
    (str | None, default: None ) –

    Output layer name. Defaults to None.

Returns:

  • NDArray –

    npt.NDArray: Loss values

export 
export(save_path: str)

Export LiteRT model content to file

Parameters:

  • save_path
    (str) –

    LiteRT file path

cleanup 
cleanup()

Cleanup temporary files

Functions