召回率 精度 f值_对精度召回率和f分数概念有完整的了解

Developing an efficient machine learning algorithm with a skewed dataset can be tricky. For example, the dataset is about fraudulent activities in the bank or cancer detection. What happens is you will see in the dataset that, 99% of the time there are no fraudulent activities or there is no cancer. You can easily cheat and just predict 0 all the time (predicting 1 if cancer and 0 if no cancer) to get a 99% accuracy. If we do that we will have a 99% accurate machine learning algorithm but we will never detect cancer. If someone has cancer, s/he will never get treatment. In the bank, there will be no action against fraudulent activities. So, accuracy alone cannot decide for a skewed dataset like that if the algorithm is working efficiently or not.

用偏斜的数据集开发有效的机器学习算法可能很棘手。 例如，数据集涉及银行中的欺诈活动或癌症检测。 发生的情况是，您将在数据集中看到99％的时间没有欺诈活动或没有癌症。 您可以很容易地作弊，并且始终可以仅预测0(如果癌症则预测1，如果没有癌症则预测0)以获得99％的准确性。 如果这样做，我们将拥有99％的准确机器学习算法，但我们将永远不会检测到癌症。 如果某人患有癌症，他/他将永远得不到治疗。 在银行中，不会采取任何针对欺诈活动的行动。 因此，像算法是否有效运行一样，仅凭准确性无法决定偏斜的数据集。

背景 (Background)

There are different evaluation matrices that can help with these types of datasets. Those evaluation metrics are called precision-recall evaluation metrics.

有不同的评估矩阵可以帮助处理这些类型的数据集。 这些评估指标称为精确召回评估指标。

To learn the precision and recall, you need to understand the following table and all its terms. Consider a binary classification. It will either return 0 or 1. For a given training data, if the actual class is 1 and the predicted class is also 1, that is called a true positive. If the actual class is 0 and the predicted class comes out to be 1, that is a false positive. If the actual class is 1 but the predicted class is a 0, it is called a false negative. If both the actual class and the predicted class are 0s, that is a true negative.

要了解精度和召回率，您需要了解下表及其所有术语。 考虑二进制分类。 它将返回0或1。对于给定的训练数据，如果实际类别为1而预测类别也为1，则称为true正值 。 如果实际类别为0，而预测类别为1，则为假肯定 。 如果实际类别为1，但预测类别为0，则称为假阴性 。 如果实际类别和预测类别均为0，则为true负数 。

Using all these, we are going to calculate the precision and recall.

使用所有这些，我们将计算精度和召回率。

精确 (Precision)

Precision calculates, what fraction of the transactions we predicted as fraudulent(predicted class 1) are actually fraudulent. Precision can be calculated using the following formula:

Precision会计算出我们预测为欺诈的交易(预测为1类)中有多少实际上是欺诈的。 可以使用以下公式计算精度 ：

Further breaking down, this formula can be written as :

进一步分解，该公式可以写成：

You can see from the formula that higher precision is good. Because higher precision means more true positives. That means when we are saying that this transaction is fraudulent, it is true.

从公式中可以看出，更高的精度是好的。 因为更高的精度意味着更多的真实肯定。 这意味着当我们说此交易是欺诈性的时，这是事实。

召回 (Recall)

Recall tells us, what fraction of all the transactions that are originally fraudulent are detected as fraudulent. That means when a transaction is actually fraudulent if we told the proper authority of the bank to take action. When I first read these definitions of precision and recall, it took me some time to really understand the difference. I hope you are getting it faster. If not, then don’t worry. You are not alone.

回忆告诉我们，在最初欺诈的所有交易中，有百分之几被检测为欺诈。 这意味着如果我们告知银行适当的权力采取行动，那么当某笔交易实际上是欺诈时。 当我第一次阅读这些关于精确度和召回率的定义时，我花了一些时间才能真正理解它们之间的区别。 我希望您能更快地得到它。 如果没有，那就不用担心。 你不是一个人。

Recall can be calculated by the following formula:

召回率可以通过以下公式计算：

Expressing by the term defined in the 2 x 2 table above:

用上面2 x 2表中定义的术语表示：

从精确度和召回率做出决策 (Making Decisions From Precision And Recall)

The precision and recall give a better sense of how an algorithm is actually doing, especially when we have a highly skewed dataset. If we predict 0 all the time and get 99.5% accuracy, the recall and precision both will be 0. Because there are no true positives. So, you know that classifier is not a good classifier. When the precision and recall both are high, that is an indication that the algorithm is doing very well.

精确度和召回率可以更好地了解算法的实际运行方式，尤其是在数据集高度偏斜的情况下。 如果我们一直预测为0并获得99.5％的准确度，则召回率和精确度都将为0。因为没有真正的肯定。 因此，您知道分类器不是一个好的分类器。 当精度和召回率都很高时，表明该算法运行良好。

Suppose we want to predict y = 1 when we are highly confident only. Because there are cases when it is very important. Especially when we are dealing with medical data. Assume that we are detecting if someone has heart disease or cancer. Predicting a false positive can bring a lot of pain in a person’s life. As a reminder, generally, logistic regression predicts 1 if the hypothesis is greater than or equal to 0.5 and predicts 0 if the hypothesis is less than 0.5.

假设仅在高度自信的情况下，我们要预测y = 1。 因为有时候这很重要。 尤其是当我们处理医疗数据时。 假设我们正在检测某人是否患有心脏病或癌症。 预测假阳性会给一个人的生活带来很多痛苦。 提醒一下，通常，逻辑假设如果假设大于或等于0.5则预测1，如果假设小于0.5则预测0。

Predict 1 if hypothesis ≥ 0.5

如果假设≥0.5，则预测1

Predict 0 if hypothesis < 0.5

如果假设<0.5，则预测0

But when we deal with some sensitive situation as mentioned we want to be more sure about our result, we predict 1 if hypothesis ≥ 0.7 and predict 0 if the hypothesis < 0.7. If you want to be even more confident about your result, you can see those values like 0.9. So you will be 90% certain that somebody has cancer or not.

但是，当我们如上所述处理某些敏感情况时，我们想更确定自己的结果，如果假设≥0.7，我们将预测为1，如果假设<0.7，我们将预测为0。 如果您想对结果更有信心，可以看到0.9之类的值。 因此，您将90％地确定某人是否患有癌症。

Now, have a look at the precision and recall formula. Both true positives and false positives will be lower. So, precision will be higher. But on the other hand, false negatives will be higher because we will predict more negatives now. In that case, the recall will be higher. But too many false negatives are also not good. If someone actually has cancer or an account has a fraudulent activity but we are telling them that they do not have cancer or the account does not have fraudulent activity, that could lead to a disaster.

现在，看看精度和召回率公式。 真实肯定和错误肯定都会更低。 因此，精度会更高。 但另一方面，由于我们现在将预测更多的负面因素，因此错误的负面因素会更高。 在这种情况下，召回率会更高。 但是太多的假阴性也不好。 如果某人确实患有癌症，或者某个账户有欺诈行为，但是我们告诉他们他们没有癌症，或者该账户没有欺诈行为，则可能导致灾难。

To avoid false negatives and achieve higher recall, we need to change the threshold to something like this:

为了避免误报并提高召回率，我们需要将阈值更改为以下内容：

Predict 1 if hypothesis ≥ 0.3

如果假设≥0.3，则预测1

Predict 0 if hypothesis < 0.3

如果假设<0.3，则预测0

As opposed to the previous case, we will have higher recall and lower precision.

与以前的情况相反，我们将具有更高的召回率和更低的精度。

How to decide the threshold then? It will depend on what your requirement is. Based on your dataset, you have to decide if you need higher precision or higher recall. Here is the precision-recall curve:

那么如何确定阈值呢？ 这将取决于您的要求。 根据数据集，您必须决定是否需要更高的精度或更高的查全率。 这是精度调用曲线：

The precision-recall curve can be of any shape. So, I am showing three different shapes here. If you cannot decide for yourself if you need higher precision or higher recall, you can use the F1 score.

精确调用曲线可以是任何形状。 因此，我在这里显示三种不同的形状。 如果您不能自己决定是否需要更高的精度或更高的查全率，则可以使用F1分数。

F1分数 (F1 Score)

F1 score is the average of precision and recall. But the formula for average is different. The regular average formula does not work here. Look at the average formula:

F1得分是准确性和召回率的平均值。 但是平均公式却不同。 常规平均公式在这里不起作用。 看一下平均公式：

(Precision + Recall) / 2

(精确+召回)/ 2

Even if the precision is 0 or recall is zero the average is still 0.5. Remember from our previous discussion, what does it mean to have a precision is zero. We can always predict y = 1. So, that should not be acceptable. Because the whole precision-recall idea is to avoid that. The formula F1 score is:

即使精度为0或召回率为零，平均值仍为0.5。 请记住，从我们之前的讨论中可以看出，精度为零是什么意思。 我们总是可以预测y =1。因此，这应该是不可接受的。 因为整个精确调用的想法是避免这种情况。 公式F1得分是：

Here, P is precision and R is the recall. If the precision is zero or recall is zero, the F1 score will be zero. So, you will know that the classifier is not working as we wanted. When the precision and recall both are perfect, that means precision is 1 and recall is also 1, the F1 score will be 1 also. So, the perfect F1 score is 1. It is a good idea to try with different thresholds and calculate the precision, recall, and F1 score to find out the optimum threshold for your machine learning algorithm.

在这里，P是精度，R是召回率。 如果精度为零或召回率为零，则F1分数将为零。 因此，您将知道分类器没有按照我们的期望工作。 当精度和召回率都完美时，这意味着精度为1，召回率也为1，F1分数也将为1。 因此，理想的F1分数是1。 最好尝试使用不同的阈值并计算精度，召回率和F1分数，以找到适合您的机器学习算法的最佳阈值。

结论 (Conclusion)

In this article, you learned how to deal with a skewed dataset. How to choose between precision and recall using an F1 score. I hope it was helpful.

在本文中，您学习了如何处理偏斜的数据集。 如何使用F1分数在精度和召回率之间进行选择。 希望对您有所帮助。

更多阅读 (More Reading)