Classification (ML training)

Micro-accuracy

Micro-average accuracy aggregates the contributions of all classes to compute the average metric. It is the fraction of instances predicted correctly. The micro-average does not take class membership into account. Basically, every sample-class pair contributes equally to the accuracy metric.

The closer to 1.00, the better. In a multi-class classification task, micro-accuracy is preferable over macro-accuracy if you suspect there might be class imbalance (i.e you may have many more examples of one class than of other classes).

Macro-accuracy

Macro-average accuracy is the average accuracy at the class level. The accuracy for each class is computed and the macro-accuracy is the average of these accuracies. Basically, every class contributes equally to the accuracy metric. Minority classes are given equal weight as the larger classes. The macro-average metric gives the same weight to each class, no matter how many instances from that class the dataset contains.

The closer to 1.00, the better. It computes the metric independently for each class and then takes the average (hence treating all classes equally)

Log-loss

Logarithmic loss measures the performance of a classification model where the prediction input is a probability value between 0.00 and 1.00. Log-loss increases as the predicted probability diverges from the actual label.

The closer to 0.00, the better. A perfect model would have a log-loss of 0.00. The goal of our machine learning models is to minimize this value.

Log-loss reduction

Logarithmic loss reduction can be interpreted as the advantage of the classifier over a random prediction.

Ranges from -∞ and 1.00, where 1.00 is perfect predictions and 0.00 indicates mean predictions. For example, if the value equals 0.20, it can be interpreted as "the probability of a correct prediction is 20% better than random guessing"

Top-K accuracy

Top-K accuracy is the percentage of times the true label is among the top K predicted labels when sorted by predicted probability. For example, Top-5 Accuracy measures how often the correct label is among the five highest predicted labels.

The closer to 1.00, the better. Higher values indicate that the correct label frequently appears within the top K predictions, which is useful in scenarios where exact ranking is less important, and we want to measure if the true class is within the most likely options.