Skip to content

threshold

Thresholding Metrics API

This module provides utility functions to threshold model predictions.

Functions:

Functions

get_predicted_threshold_indices 

get_predicted_threshold_indices(y_prob: npt.NDArray, y_pred: npt.NDArray, threshold: float = 0.5) -> npt.NDArray

Get prediction indices that are above threshold (confidence level). This is useful to remove weak predictions that can happen due to noisy data or poor model performance.

Parameters:

  • y_prob (NDArray) –

    Model output as probabilities

  • y_pred (NDArray) –

    Model predictions. Defaults to None.

  • threshold (float, default: 0.5 ) –

    Confidence level

Returns:

  • NDArray –

    npt.NDArray: Indices of y_prob that satisfy threshold

Source code in neuralspot_edge/metrics/threshold.py 
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def get_predicted_threshold_indices(
    y_prob: npt.NDArray,
    y_pred: npt.NDArray,
    threshold: float = 0.5,
) -> npt.NDArray:
    """Get prediction indices that are above threshold (confidence level).
    This is useful to remove weak predictions that can happen due to noisy data or poor model performance.

    Args:
        y_prob (npt.NDArray): Model output as probabilities
        y_pred (npt.NDArray, optional): Model predictions. Defaults to None.
        threshold (float): Confidence level

    Returns:
        npt.NDArray: Indices of y_prob that satisfy threshold
    """

    y_pred_prob = np.take_along_axis(y_prob, np.expand_dims(y_pred, axis=-1), axis=-1).squeeze(axis=-1)
    y_thresh_idx = np.where(y_pred_prob > threshold)[0]
    return y_thresh_idx

threshold_predictions 

threshold_predictions(y_prob: npt.NDArray, y_pred: npt.NDArray, y_true: npt.NDArray, threshold: float = 0.5) -> tuple[npt.NDArray, npt.NDArray, npt.NDArray]

Get prediction indices that are above threshold (confidence level). This is useful to remove weak predictions that can happen due to noisy data or poor model performance.

Parameters:

  • y_prob (NDArray) –

    Model output as probabilities

  • y_pred (NDArray) –

    Model predictions. Defaults to None.

  • y_true (NDArray) –

    True labels

  • threshold (float, default: 0.5 ) –

    Confidence level. Defaults to 0.5.

Returns:

Source code in neuralspot_edge/metrics/threshold.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def threshold_predictions(
    y_prob: npt.NDArray,
    y_pred: npt.NDArray,
    y_true: npt.NDArray,
    threshold: float = 0.5,
) -> tuple[npt.NDArray, npt.NDArray, npt.NDArray]:
    """Get prediction indices that are above threshold (confidence level).
    This is useful to remove weak predictions that can happen due to noisy data or poor model performance.

    Args:
        y_prob (npt.NDArray): Model output as probabilities
        y_pred (npt.NDArray, optional): Model predictions. Defaults to None.
        y_true (npt.NDArray): True labels
        threshold (float): Confidence level. Defaults to 0.5.

    Returns:
        tuple[npt.NDArray, npt.NDArray, npt.NDArray]: Thresholded predictions
    """
    y_thresh_idx = get_predicted_threshold_indices(y_prob, y_pred, threshold)
    y_prob = y_prob[y_thresh_idx]
    y_pred = y_pred[y_thresh_idx]
    y_true = y_true[y_thresh_idx]
    return y_prob, y_pred, y_true