05. 多元分类问题的求解

来源：原始 Jupyter Notebook。内容已转换为网页阅读格式；下载原文件。

在上一节中，我们根据梯度下降法得到参数 w 和 b 的更新公式：

w := w - \eta \frac{\partial loss}{\partial w} = w + \eta \sum_{i=1}^n (y_i - \pi(x_i))x_i

b := b - \eta \frac{\partial loss}{\partial b} = b + \eta \sum_{i=1}^n (y_i - \pi(x_i))

这个公式可以用于一元分类的情况，也就是只有一个变量，我们可以进一步将其扩展到二元甚至多元的情况。多元逻辑函数可以写成矩阵的形式：

y = \frac{1}{1 + e^{-\theta^TX}}

比较容易得出，它对应的梯度更新公式为：

\theta_j := \theta_j + \eta \sum_{i=1}^n (y^{(i)} - \pi(x^{(i)}))x_j^{(i)}

其中 $x_j^{(i)}$ 表示的是样本 $x^{(i)}$ 的第 $j$ 个分量，对应权重 $\theta_j$ 。写成矩阵形式就是：

\Theta := \Theta + \eta X^T (Y - \pi(X))

实例一、二元线性分类

Peter Harrington 的《机器学习实战》第 5 章介绍了一个例子，使用梯度上升法（和梯度下降法是一样的）求解二元线性分类问题。

实例二、预测肿瘤的良性和恶性

这是一个多元分类的例子，这个例子来自这里，使用了 UCI 的一份肿瘤数据集，并通过 sklearn 提供的逻辑回归和梯度下降方法来预测肿瘤的良性和恶性。

import pandas as pd
import numpy as np

column_names = [
    'Sample code number',
    'Clump Thickness',
    'Uniformity of Cell Size',
    'Uniformity of Cell Shape',
    'Marginal Adhesion',
    'Single Epithelial Cell Size',
    'Bare Nuclei',
    'Bland Chromatin',
    'Normal Nucleoli',
    'Mitoses',
    'Class']

data = pd.read_csv(
    'https://archive.ics.uci.edu' + 
    '/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data',
    names=column_names)
data = data.replace(to_replace='?',value = np.nan)
data = data.dropna(how='any')
data.shape

运行结果

(683, 11)

data.head(10)

运行结果

	Sample code number	Clump Thickness	Uniformity of Cell Size	Uniformity of Cell Shape	Marginal Adhesion	Single Epithelial Cell Size	Bare Nuclei	Bland Chromatin	Normal Nucleoli	Mitoses	Class
0	1000025	5	1	1	1	2	1	3	1	1	2
1	1002945	5	4	4	5	7	10	3	2	1	2
2	1015425	3	1	1	1	2	2	3	1	1	2
3	1016277	6	8	8	1	3	4	3	7	1	2
4	1017023	4	1	1	3	2	1	3	1	1	2
5	1017122	8	10	10	8	7	10	9	7	1	4
6	1018099	1	1	1	1	2	10	3	1	1	2
7	1018561	2	1	2	1	2	1	3	1	1	2
8	1033078	2	1	1	1	2	1	1	1	5	2
9	1033078	4	2	1	1	2	1	2	1	1	2

对数据进行分割，25%作为测试集，其余作为训练集：

from sklearn.model_selection import train_test_split

X_train,X_test,y_train,y_test = train_test_split(
    data[column_names[1:10]],
    data[column_names[10]],
    test_size = 0.25,
    random_state = 33)
y_train.value_counts()
y_test.value_counts()

运行结果

2    100
4     71
Name: Class, dtype: int64

from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.linear_model import SGDClassifier
from sklearn.metrics import classification_report

ss = StandardScaler()
X_train = ss.fit_transform(X_train)
X_test = ss.transform(X_test)

# 逻辑回归
lr = LogisticRegression()
lr.fit(X_train,y_train)
lr_y_predict = lr.predict(X_test)

# 梯度下降
sgdc = SGDClassifier()
sgdc.fit(X_train,y_train)
sgdc_y_predict = sgdc.predict(X_test)


print('Accuracy of LR Classifier:',lr.score(X_test,y_test))
print(classification_report(y_test,lr_y_predict,target_names=['Benign','Malignant']))

运行结果

Accuracy of LR Classifier: 0.9883040935672515
              precision    recall  f1-score   support

      Benign       0.99      0.99      0.99       100
   Malignant       0.99      0.99      0.99        71

   micro avg       0.99      0.99      0.99       171
   macro avg       0.99      0.99      0.99       171
weighted avg       0.99      0.99      0.99       171

运行结果

/home/aneasystone/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
  FutureWarning)
/home/aneasystone/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/stochastic_gradient.py:166: FutureWarning: max_iter and tol parameters have been added in SGDClassifier in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.
  FutureWarning)

实例一、二元线性分类#

实例二、预测肿瘤的良性和恶性#

实例一、二元线性分类

实例二、预测肿瘤的良性和恶性