Skip to content

metric_utils

Metrics Utils API

This module provides utility functions to compute metrics for classification tasks.

Functions:

Functions

compute_metrics 

compute_metrics(metrics: list[keras.Metric], y_true: keras.KerasTensor, y_pred: keras.KerasTensor) -> dict[str, float]

Compute set of metrics for y_true and y_pred.

Parameters:

  • metrics (list[Metric]) –

    List of metrics

  • y_true (KerasTensor) –

    True labels

  • y_pred (KerasTensor) –

    Predicted labels

Returns:

  • dict ( dict[str, float] ) –

    Dictionary of metric names and values

Example:

metrics = [keras.metrics.Accuracy('acc'), keras.metrics.Precision(0.5, name='precision')]
y_true = np.array([0, 1, 1, 0])
y_pred = np.array([0, 1, 0, 1])
results = nse.metrics.compute_metrics(metrics, y_true, y_pred)
print(results)
Source code in neuralspot_edge/metrics/metric_utils.py 
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
def compute_metrics(
    metrics: list[keras.Metric], y_true: keras.KerasTensor, y_pred: keras.KerasTensor
) -> dict[str, float]:
    """Compute set of metrics for y_true and y_pred.

    Args:
        metrics (list[keras.Metric]): List of metrics
        y_true (keras.KerasTensor): True labels
        y_pred (keras.KerasTensor): Predicted labels

    Returns:
        dict: Dictionary of metric names and values

    Example:

    ```python
    metrics = [keras.metrics.Accuracy('acc'), keras.metrics.Precision(0.5, name='precision')]
    y_true = np.array([0, 1, 1, 0])
    y_pred = np.array([0, 1, 0, 1])
    results = nse.metrics.compute_metrics(metrics, y_true, y_pred)
    print(results)
    ```
    """

    results = {}
    for metric in metrics:
        metric.reset_state()
        metric.update_state(y_true, y_pred)
        results[metric.name] = metric.result().numpy()
    return results

confusion_matrix 

confusion_matrix(labels: keras.KerasTensor, predictions: keras.KerasTensor, num_classes: int, weights: keras.KerasTensor | None = None, dtype: str = 'int32', normalize: Literal['true', 'pred', 'all'] | None = None) -> keras.KerasTensor

Compute confusion matrix using keras w/ addition to normalize.

Normalization modes
  • "true": Normalize by true labels
  • "pred": Normalize by predicted labels
  • "all": Normalize by all labels

Parameters:

  • labels (KerasTensor) –

    True labels

  • predictions (KerasTensor) –

    Predicted labels

  • num_classes (int) –

    Number of classes

  • weights (KerasTensor, default: None ) –

    Weights. Defaults to None.

  • dtype (str, default: 'int32' ) –

    Data type. Defaults to "int32".

  • normalize (Literal['true', 'pred', 'all'], default: None ) –

    Normalization mode. Defaults to None.

Returns:

  • KerasTensor –

    keras.KerasTensor: Confusion matrix

Example:

labels = np.array([0, 1, 1, 0])
predictions = np.array([0, 1, 0, 1])
num_classes = 2
cm = nse.metrics.confusion_matrix(labels, predictions, num_classes, normalize='true')
print(cm)
Source code in neuralspot_edge/metrics/metric_utils.py 
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
def confusion_matrix(
    labels: keras.KerasTensor,
    predictions: keras.KerasTensor,
    num_classes: int,
    weights: keras.KerasTensor | None = None,
    dtype: str = "int32",
    normalize: Literal["true", "pred", "all"] | None = None,
) -> keras.KerasTensor:
    """Compute confusion matrix using keras w/ addition to normalize.

    Normalization modes:
        - "true": Normalize by true labels
        - "pred": Normalize by predicted labels
        - "all": Normalize by all labels

    Args:
        labels (keras.KerasTensor): True labels
        predictions (keras.KerasTensor): Predicted labels
        num_classes (int): Number of classes
        weights (keras.KerasTensor, optional): Weights. Defaults to None.
        dtype (str, optional): Data type. Defaults to "int32".
        normalize (Literal["true", "pred", "all"], optional): Normalization mode. Defaults to None.

    Returns:
        keras.KerasTensor: Confusion matrix

    Example:

    ```python
    labels = np.array([0, 1, 1, 0])
    predictions = np.array([0, 1, 0, 1])
    num_classes = 2
    cm = nse.metrics.confusion_matrix(labels, predictions, num_classes, normalize='true')
    print(cm)
    ```
    """
    cm = metrics_utils.confusion_matrix(
        labels,
        predictions,
        num_classes,
        weights=weights,
        dtype=dtype,
    )

    if normalize == "true":
        cm = keras.ops.divide(cm, keras.ops.sum(cm, axis=1, keepdims=True))
    elif normalize == "pred":
        cm = keras.ops.divide(cm, keras.ops.sum(cm, axis=0, keepdims=True))
    elif normalize == "all":
        cm = keras.ops.divide(cm, keras.ops.sum(cm))
    cm = keras.ops.nan_to_num(cm, posinf=0, neginf=0)
    return cm