{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "在上一节中，我们根据梯度下降法得到参数 w 和 b 的更新公式：\n", "\n", "$$\n", "w := w - \\eta \\frac{\\partial loss}{\\partial w} = w + \\eta \\sum_{i=1}^n (y_i - \\pi(x_i))x_i\n", "$$\n", "\n", "$$\n", "b := b - \\eta \\frac{\\partial loss}{\\partial b} = b + \\eta \\sum_{i=1}^n (y_i - \\pi(x_i))\n", "$$\n", "\n", "这个公式可以用于一元分类的情况，也就是只有一个变量，我们可以进一步将其扩展到二元甚至多元的情况。多元逻辑函数可以写成矩阵的形式：\n", "\n", "$$\n", "y = \\frac{1}{1 + e^{-\\theta^TX}}\n", "$$\n", "\n", "比较容易得出，它对应的梯度更新公式为：\n", "\n", "$$\n", "\\theta_j := \\theta_j + \\eta \\sum_{i=1}^n (y^{(i)} - \\pi(x^{(i)}))x_j^{(i)}\n", "$$\n", "\n", "其中 $x_j^{(i)}$ 表示的是样本 $x^{(i)}$ 的第 $j$ 个分量，对应权重 $\\theta_j$。写成矩阵形式就是：\n", "\n", "$$\n", "\\Theta := \\Theta + \\eta X^T (Y - \\pi(X))\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 实例一、二元线性分类\n", "\n", "Peter Harrington 的 [《机器学习实战》第 5 章](../其他资料/书籍/04.%20机器学习实战.Peter%20Harrington.ipynb#第5章%E3%80%80Logistic回归) 介绍了一个例子，使用梯度上升法（和梯度下降法是一样的）求解二元线性分类问题。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 实例二、预测肿瘤的良性和恶性\n", "\n", "这是一个多元分类的例子，这个例子来自 [这里](https://www.cnblogs.com/chenice/p/7202045.html)，使用了 UCI 的一份肿瘤数据集，并通过 sklearn 提供的逻辑回归和梯度下降方法来预测肿瘤的良性和恶性。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(683, 11)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "import numpy as np\n", "\n", "column_names = [\n", " 'Sample code number',\n", " 'Clump Thickness',\n", " 'Uniformity of Cell Size',\n", " 'Uniformity of Cell Shape',\n", " 'Marginal Adhesion',\n", " 'Single Epithelial Cell Size',\n", " 'Bare Nuclei',\n", " 'Bland Chromatin',\n", " 'Normal Nucleoli',\n", " 'Mitoses',\n", " 'Class']\n", "\n", "data = pd.read_csv(\n", " 'https://archive.ics.uci.edu' + \n", " '/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data',\n", " names=column_names)\n", "data = data.replace(to_replace='?',value = np.nan)\n", "data = data.dropna(how='any')\n", "data.shape" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Sample code numberClump ThicknessUniformity of Cell SizeUniformity of Cell ShapeMarginal AdhesionSingle Epithelial Cell SizeBare NucleiBland ChromatinNormal NucleoliMitosesClass
010000255111213112
1100294554457103212
210154253111223112
310162776881343712
410170234113213112
510171228101087109714
6101809911112103112
710185612121213112
810330782111211152
910330784211212112
\n", "
" ], "text/plain": [ " Sample code number Clump Thickness Uniformity of Cell Size \\\n", "0 1000025 5 1 \n", "1 1002945 5 4 \n", "2 1015425 3 1 \n", "3 1016277 6 8 \n", "4 1017023 4 1 \n", "5 1017122 8 10 \n", "6 1018099 1 1 \n", "7 1018561 2 1 \n", "8 1033078 2 1 \n", "9 1033078 4 2 \n", "\n", " Uniformity of Cell Shape Marginal Adhesion Single Epithelial Cell Size \\\n", "0 1 1 2 \n", "1 4 5 7 \n", "2 1 1 2 \n", "3 8 1 3 \n", "4 1 3 2 \n", "5 10 8 7 \n", "6 1 1 2 \n", "7 2 1 2 \n", "8 1 1 2 \n", "9 1 1 2 \n", "\n", " Bare Nuclei Bland Chromatin Normal Nucleoli Mitoses Class \n", "0 1 3 1 1 2 \n", "1 10 3 2 1 2 \n", "2 2 3 1 1 2 \n", "3 4 3 7 1 2 \n", "4 1 3 1 1 2 \n", "5 10 9 7 1 4 \n", "6 10 3 1 1 2 \n", "7 1 3 1 1 2 \n", "8 1 1 1 5 2 \n", "9 1 2 1 1 2 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.head(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "对数据进行分割，25%作为测试集，其余作为训练集：" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2 100\n", "4 71\n", "Name: Class, dtype: int64" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.model_selection import train_test_split\n", "\n", "X_train,X_test,y_train,y_test = train_test_split(\n", " data[column_names[1:10]],\n", " data[column_names[10]],\n", " test_size = 0.25,\n", " random_state = 33)\n", "y_train.value_counts()\n", "y_test.value_counts()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy of LR Classifier: 0.9883040935672515\n", " precision recall f1-score support\n", "\n", " Benign 0.99 0.99 0.99 100\n", " Malignant 0.99 0.99 0.99 71\n", "\n", " micro avg 0.99 0.99 0.99 171\n", " macro avg 0.99 0.99 0.99 171\n", "weighted avg 0.99 0.99 0.99 171\n", "\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/aneasystone/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n", " FutureWarning)\n", "/home/aneasystone/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/stochastic_gradient.py:166: FutureWarning: max_iter and tol parameters have been added in SGDClassifier in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.\n", " FutureWarning)\n" ] } ], "source": [ "from sklearn.preprocessing import StandardScaler\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.linear_model import SGDClassifier\n", "from sklearn.metrics import classification_report\n", "\n", "ss = StandardScaler()\n", "X_train = ss.fit_transform(X_train)\n", "X_test = ss.transform(X_test)\n", "\n", "# 逻辑回归\n", "lr = LogisticRegression()\n", "lr.fit(X_train,y_train)\n", "lr_y_predict = lr.predict(X_test)\n", "\n", "# 梯度下降\n", "sgdc = SGDClassifier()\n", "sgdc.fit(X_train,y_train)\n", "sgdc_y_predict = sgdc.predict(X_test)\n", "\n", "\n", "print('Accuracy of LR Classifier:',lr.score(X_test,y_test))\n", "print(classification_report(y_test,lr_y_predict,target_names=['Benign','Malignant']))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.1" } }, "nbformat": 4, "nbformat_minor": 2 }