Metrics

The metrics module defines some classes to be used as metrics during model training.

class segmentation.metrics.Dice(*args, **kwargs)

Bases: tensorflow.python.keras.metrics.Metric

Computes the Dice metric per-class.

Dice is a common evaluation metric for semantic image segmentation, obtained by computing the Dice for each semantic class. Dice is defined as follows:

\[Dice = \frac{2*TP}{2*TP + FP + FN}\]

The predictions are accumulated in a confusion matrix, weighted by sample_weight and the metric is then calculated from it. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Standalone usage:

>>> # cm = [[1, 1],
>>> #        [1, 1]]
>>> # sum_row = [2, 2], sum_col = [2, 2], true_positives = [1, 1]
>>> # dice = 2*true_positives / (sum_row + sum_col))
>>> # result = (2 / (2 + 2)) = 0.5
>>> m = segmentation.metrics.Dice(num_classes=2, class_to_return=0)
>>> m.update_state([0, 0, 1, 1], [0, 1, 0, 1])
>>> m.result().numpy()
0.5
>>> m = segmentation.metrics.Dice(num_classes=2, class_to_return=1)
>>> m.update_state([0, 0, 1, 1], [0, 1, 0, 1],
...                sample_weight=[0.3, 0.3, 0.3, 0.1])
>>> m.result().numpy()
0.25

Usage with compile() API:

model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[segmentation.metrics.Dice(num_classes=2)]
    )

Args:

num_classes (int, required): The possible number of labels

the prediction can have.

name (str, optional): string name of the metric instance. Defaults to None. dtype (dtype, optional): data type of the metric result. Defaults to None. class_to_return (int, optional): class for which Dice value is returned. Defaults to 0.

get_config()

Returns the serializable config of the metric.

reset_states()

result()

Compute the mean intersection-over-union via the confusion matrix.

update_state(y_true, y_pred, sample_weight=None)

Accumulates the confusion matrix statistics.

Args:

y_true: The ground truth values. y_pred: The predicted values. sample_weight: Optional weighting of each example. Defaults to 1. Can be a

Tensor whose rank is either 0, or the same rank as y_true, and must be broadcastable to y_true.

Returns:

Update op.

class segmentation.metrics.IoUPerClass(*args, **kwargs)

Bases: tensorflow.python.keras.metrics.Metric

Compute metric IoU for parameter y_true and y_pred only for the specified class. Input y_true and y_pred is supposed to be 5-dimensional: (batch, x, y, z, softmax_probabilities)

result()

This function is only used to assign a name to the given IoU metric.

update_state(y_true, y_pred, sample_weight=None)

Accumulates statistics for the metric.

Note: This function is executed as a graph function in graph mode. This means:

1. Operations on the same resource are executed in textual order. This should make it easier to do things like add the updated value of a variable to another, for example.

2. You don’t need to worry about collecting the update ops to execute. All update ops added to the graph by this function will be executed.

As a result, code should generally work the same way with graph or eager execution.

Args:

*args: **kwargs: A mini-batch of inputs to the Metric.

class segmentation.metrics.MeanDice(*args, **kwargs)

Bases: tensorflow.python.keras.metrics.Metric

Computes the Dice metric average over classes. Dice is a common evaluation metric for semantic image segmentation, obtained by computing the Dice for each semantic class and then by averaging the values. Dice is defined as follows:

\[Dice = \frac{2*TP}{2*TP + FP + FN}\]

Standalone usage:

>>> # cm = [[1, 1],
>>> #        [1, 1]]
>>> # sum_row = [2, 2], sum_col = [2, 2], true_positives = [1, 1]
>>> # dice = 2*true_positives / (sum_row + sum_col - true_positives))
>>> # result = (1 / (2 + 2 - 1) , 1 / (2 + 2 - 1)) = 0.33, 0.33
>>> m = tf.keras.metrics.MeanIoU(num_classes=2)
>>> m.update_state([0, 0, 1, 1], [0, 1, 0, 1])
>>> m.result().numpy()
0.33333334, 0.33333334
>>> m.reset_states()
>>> m.update_state([0, 0, 1, 1], [0, 1, 0, 1],
...                sample_weight=[0.3, 0.3, 0.3, 0.1])
>>> m.result().numpy()
0.33333334, 0.14285715

Usage with compile() API:

model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[segmentation.metrics.MeanDice(num_classes=2)])

The predictions are accumulated in a confusion matrix, weighted by sample_weight and the metric is then calculated from it. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.

Args:

num_classes (int, required): the possible number of labels the prediction task can have. name (str, optional): string name of the metric instance. dtype (dtype, optional): data type of the metric result.

get_config()

Returns the serializable config of the metric.

reset_states()

result()

Compute the mean intersection-over-union via the confusion matrix.

update_state(y_true, y_pred, sample_weight=None)

Accumulates the confusion matrix statistics.

Args:

y_true: The ground truth values. y_pred: The predicted values. sample_weight: Optional weighting of each example. Defaults to 1. Can be a

Tensor whose rank is either 0, or the same rank as y_true, and must be broadcastable to y_true.

Returns:

Update op.

class segmentation.metrics.PerClassIoU(*args, **kwargs)

Bases: tensorflow.python.keras.metrics.Metric

Computes the Intersection-Over-Union metric per-class. This metric is supposed to work only with three-dimensional input. Intersection-Over-Union is a common evaluation metric for semantic image segmentation, obtained by computing the IOU for each semantic class. IOU is defined as follows: .. math:

IOU = \frac{TP}{TP+FP+FN}

The predictions are accumulated in a confusion matrix, weighted by sample_weight and the metric is then calculated from it. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values. Args:

num_classes: The possible number of labels the prediction task can have.

This value must be provided, since a confusion matrix of dimension = [num_classes, num_classes] will be allocated.

name: (Optional) string name of the metric instance. dtype: (Optional) data type of the metric result.

Standalone usage: >>> # cm = [[1, 1], >>> # [1, 1]] >>> # sum_row = [2, 2], sum_col = [2, 2], true_positives = [1, 1] >>> # iou = true_positives / (sum_row + sum_col - true_positives)) >>> # result = (1 / (2 + 2 - 1) , 1 / (2 + 2 - 1)) = 0.33, 0.33 >>> m = segmentation.metrics.PerClassIoU(num_classes=2, class_to_return=1) >>> m.update_state([0, 0, 1, 1], [0, 1, 0, 1]) >>> m.result().numpy() 0.33333334 >>> m.reset_states() >>> m.update_state([0, 0, 1, 1], [0, 1, 0, 1], … sample_weight=[0.3, 0.3, 0.3, 0.1]) >>> m.result().numpy() 0.14285715

Usage with compile() API: ```python model.compile(

optimizer=’sgd’, loss=’mse’, metrics=[segmentation.metrics.PerClassIoU(num_classes=2, class_to_return=0)])

```

get_config()

Returns the serializable config of the metric.

reset_states()

result()

Compute the mean intersection-over-union via the confusion matrix.

update_state(y_true, y_pred, sample_weight=None)

Accumulates the confusion matrix statistics. Args:

y_true: The ground truth values. y_pred: The predicted values. sample_weight: Optional weighting of each example. Defaults to 1. Can be a

Tensor whose rank is either 0, or the same rank as y_true, and must be broadcastable to y_true.

Returns:

Update op.

segmentation.metrics.compute_dice(trueLabel, predictedLabel, return_average=True, classes=None)

segmentation.metrics.compute_dice_class(cl, trueLabel, predictedLabel)

segmentation.metrics.compute_for(trueLabel, predictedLabel, classes=None)

segmentation.metrics.compute_jaccard(trueLabel, predictedLabel, return_average=True, classes=None)

segmentation.metrics.compute_ppv_class(cl, trueLabel, predictedLabel)

segmentation.metrics.compute_precision(trueLabel, predictedLabel, classes=None)

segmentation.metrics.compute_sensitivity(trueLabel, predictedLabel, classes=None)

segmentation.metrics.count_fn(cl, trueLabel, predictedLabel)

Return total number of false negatives for the specified class, given the true and predicted labels.

segmentation.metrics.count_fp(cl, trueLabel, predictedLabel)

Compute total number of false positives for given class.

This function returns the total number of false positives for the specified class, given the true and predicted labels.

Args:

cl (int): The class of interest. trueLabel (array): The true label. predictedLabel (array): The predicted label.

Return:

total number of false positives.

segmentation.metrics.count_tn(cl, trueLabel, predictedLabel)

Compute number of true negatives.

Return total number of true negatives for the specified class, given the true and predicted labels.

Args:

cl (int, required): class to be considered in labels.

segmentation.metrics.count_tp(cl, trueLabel, predictedLabel)

Return total number of true positives for the specified class, given the true and predicted labels.