Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "python simulation support vector machine example analysis". In the operation process of actual cases, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!

Brief Description of Ideas

1. Linear separable support vector machine, or hard-spaced support vector machine. It is constructed on the condition that the training data are linearly separable. Its learning strategy is maximum interval method. It can be expressed as a convex quadratic programming problem

2. In reality, the training data is less linearly separable, and the training data is often approximately linearly separable. In this case, linear support vector machines or soft interval support vector machines are used. Linear Support Vector Machine is the most basic support vector machine. For noise or exception, by introducing slack variables, making it "separable,"

3. For nonlinear classification problem in input space, it can be transformed into linear classification problem in some high-dimensional feature space by nonlinear transformation, and linear support vector machine can be learned in high-dimensional feature space. Since the objective function and classification decision function only involve the inner product between instances in the dual problem of linear support vector machine learning, it is not necessary to specify explicitly the nonlinear transformation, but to replace the inner product with kernel function. The kernel function represents the inner product between two instances after a nonlinear transformation.

4. SMO algorithm. SMO algorithm is a fast algorithm for SVM learning. It is characterized by decomposing the original quadratic programming problem into quadratic programming subproblems with only two variables, and solving the subproblems analytically until all variables satisfy KKT condition. In this way, the optimal solution of the original quadratic programming problem is obtained by heuristic method. Because the sub-problem has analytical solution, so each calculation of sub-problem is very fast, although the calculation of sub-problem times are many, but in general or efficient.

The program sklearn already has relevant package files, and directly calls import numpy as npimport pandas as pdfrom sklearn.datasets import load_irisfrom sklearn.model_selection import train_test_splitimport matplotlib.pyplot as pltfrom sklearn.svm import SVCdef create_data(): iris = load_iris() df = pd.DataFrame(iris.data, columns=iris.feature_names) df['label'] = iris.target df.columns = [ 'sepal length', 'sepal width', 'petal length', 'petal width', 'label' ] data = np.array(df.iloc[:100, [0, 1, -1]]) for i in range(len(data)): if data[i, -1] == 0: data[i, -1] = -1 # print(data) return data[:, :2], data[:, -1]X, y = create_data()X_train, X_test, y_train, y_test = train_test_split (X, y, test_size=0.25)clf = SVC()#The parameters below are set for linear simulation. Only after setting can the diagram of simulation function be drawn # clf = SVC (kernel='linear')clf.fit (X_train, y_train)print ("Training Set Score: {}".format(str(clf.score(X_test, y_test)*100)+"%"))### x_points = np. range (4, 8)# y_ = -(clf.coef_[0][0]*x_points + clf.intercept_)/clf.coef_[0][1]# plt.plot(x_points, y_)plt.scatter(X[:50, 0], X[:50, 1], label='-1')plt.scatter(X[50:, 0], X[50:, 1], label='1')plt.legend()plt.show()

Results:

Data points:

Plot of linear simulation results:

Training Set Score: import numpy as npimport pandas as pdfrom sklearn.datasets import load_iris from sklearn.model_selection import train_test_splitimport matplotlib.pyplot as pltdef create_data(): iris = load_iris() df = pd.DataFrame(iris.data, columns=iris.feature_names) df['label'] = iris.target df.columns = [ 'sepal length', 'sepal width', 'petal length', 'petal width', 'label' ] data = np.array(df.iloc[:100, [0, 1, -1]]) for i in range(len(data)): if data[i, -1] == 0: data[i, -1] = -1 # print(data) return data[:, :2], data[:, -1]X, y = create_data()X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)plt.scatter(X[:50, 0], X[:50, 1], label='-1')plt.scatter(X[50:, 0], X[50:, 1], label='1')plt.legend()plt.show()class SVM: def __init__(self, max_iter=100, kernel='linear'): self.max_iter = max_iter self._ kernel = kernel def init_args(self, features, labels): self.m, self.n = features.shape self.X = features self.Y = labels self.b = 0.0 #Save Ei in a list self.alpha = np.ones(self.m) self.E = [self._ E(i) for i in range(self.m)] #Relaxation variables self.C = 1.0 def _KKT(self, i): y_g = self._ g(i) * self.Y[i] if self.alpha[i] == 0: return y_g >= 1 elif 0 < self.alpha[i] < self.C: return y_g == 1 else: return y_g

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.