Python机器学习性能评估

在Python中，我们通常使用scikit-learn库来进行机器学习模型的性能评估。以下是一些常用的性能评估指标和方法：

1. 分类问题

准确率（Accuracy）

from sklearn.metrics import accuracy_score

y_true = [0, 1, 2, 3, 4]
y_pred = [0, 2, 2, 3, 4]

accuracy = accuracy_score(y_true, y_pred)
print(f"Accuracy: {accuracy}")

精确率（Precision）、召回率（Recall）和F1分数（F1 Score）

from sklearn.metrics import precision_score, recall_score, f1_score

y_true = [0, 1, 2, 3, 4]
y_pred = [0, 2, 2, 3, 4]

precision = precision_score(y_true, y_pred, average='macro')
recall = recall_score(y_true, y_pred, average='macro')
f1 = f1_score(y_true, y_pred, average='macro')

print(f"Precision: {precision}")
print(f"Recall: {recall}")
print(f"F1 Score: {f1}")

ROC曲线和AUC值

from sklearn.metrics import roc_curve, auc
import matplotlib.pyplot as plt

y_true = [0, 1, 0, 1, 0, 1]
y_scores = [0.1, 0.4, 0.35, 0.8, 0.7, 0.6]

fpr, tpr, thresholds = roc_curve(y_true, y_scores)
roc_auc = auc(fpr, tpr)

plt.figure()
plt.plot(fpr, tpr, color='darkorange', lw=2, label=f'ROC curve (area = {roc_auc:.2f})')
plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc="lower right")
plt.show()

2. 回归问题

均方误差（Mean Squared Error, MSE）

from sklearn.metrics import mean_squared_error

y_true = [3, -0.5, 2, 7]
y_pred = [2.5, 0.0, 2, 8]

mse = mean_squared_error(y_true, y_pred)
print(f"MSE: {mse}")

均方根误差（Root Mean Squared Error, RMSE）

from sklearn.metrics import mean_squared_error
import numpy as np

rmse = np.sqrt(mean_squared_error(y_true, y_pred))
print(f"RMSE: {rmse}")

平均绝对误差（Mean Absolute Error, MAE）

from sklearn.metrics import mean_absolute_error

mae = mean_absolute_error(y_true, y_pred)
print(f"MAE: {mae}")

R² 分数（R² Score）

from sklearn.metrics import r2_score

r2 = r2_score(y_true, y_pred)
print(f"R² Score: {r2}")

3. 交叉验证

交叉验证是一种评估模型泛化能力的方法，常用的有K折交叉验证。

from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier()
scores = cross_val_score(clf, X, y, cv=5)

print(f"Scores: {scores}")
print(f"Mean Score: {np.mean(scores)}")

4. 混淆矩阵

混淆矩阵可以帮助我们了解模型在不同类别上的表现。

from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt

y_true = [0, 1, 2, 3, 4]
y_pred = [0, 2, 2, 3, 4]

cm = confusion_matrix(y_true, y_pred)
sns.heatmap(cm, annot=True, fmt='d')
plt.xlabel('Predicted')
plt.ylabel('True')
plt.show()

通过这些指标和方法，你可以全面评估机器学习模型的性能，并根据需要进行调整和优化。