Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about how to make predictions based on BP neural network in Python. Many people may not know much about it. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something from this article.

I. the advantages of Introduction1 BP neural network

Nonlinear mapping ability: BP neural network essentially realizes a mapping function from input to output. Mathematical theory proves that a three-layer neural network can approach any nonlinear continuous function with arbitrary precision. This makes it especially suitable for solving problems with complex internal mechanism, that is, BP neural network has strong nonlinear mapping ability.

Self-learning and adaptive ability: during training, BP neural network can automatically extract the "reasonable rules" between input and output data through learning, and adaptively memorize the learning content in the weight of the network. That is, BP neural network has a high degree of self-learning and adaptive ability.

Generalization ability: the so-called generalization ability means that when designing a pattern classifier, we should not only consider that the network is ensuring the correct classification of the required classification objects, but also care about whether the network can classify the unseen patterns or patterns with noise pollution after training. In other words, BP neural network has the ability to apply learning achievements to new knowledge.

2 shortcomings of BP neural network

Local minimization problem: from a mathematical point of view, the traditional BP neural network is a local search optimization method, it is to solve a complex non-linear problem, the weight of the network is gradually adjusted along the direction of local improvement, this will make the algorithm fall into the local extremum, the weight converges to the local minimum, resulting in the failure of network training. In addition, the BP neural network is very sensitive to the initial network weight, and initializing the network with different weights tends to converge to different local minima, which is the fundamental reason why different results are obtained in each training.

The convergence speed of BP neural network algorithm is slow: because the BP neural network algorithm is essentially a gradient descent method, the objective function to be optimized is very complex, so it is bound to appear "sawtooth phenomenon", which makes the BP algorithm inefficient. And because the objective function of optimization is very complex, it is bound to appear some flat areas when the neuron output is close to 0 or 1. In these areas, the weight error changes very little, so that the training process almost stops. In the BP neural network model, in order to make the network execute the BP algorithm, the traditional one-dimensional search method can not be used to calculate the step size of each iteration, but the update rule of the step size must be given to the network in advance, which will also cause inefficiency of the algorithm. All these lead to the slow convergence of BP neural network algorithm.

The choice of BP neural network structure is different: up to now, there is no unified and complete theoretical guidance for the selection of BP neural network structure, and generally it can only be selected by experience. The choice of network structure is too large, the efficiency in training is not high, and the phenomenon of over-fitting may occur, resulting in low network performance and decreased fault tolerance. if the choice is too small, the network may not converge. The structure of the network directly affects the approximation ability and generalization property of the network. Therefore, how to choose the appropriate network structure in the application is an important problem.

Implementation process 1: Demo#%% basic array operation library import import numpy as np # drawing library import import matplotlib.pyplot as plt # import 3D display tool from mpl_toolkits.mplot3d import Axes3D# import BP model from sklearn.neural_network import MLPClassifier# import demo data production method from sklearn.datasets import make_classificationfrom sklearn.metrics import classification_report, confusion_matriximport seaborn as snsimport warningsfrom sklearn.exceptions import ConvergenceWarning#%% model training # make five categories of data 1000 samples for each category: train_samples, train_labels = make_classification (n_samples=1000, n_features=3, nasty redundant0 authoritative classrooms 5, n_informative=3, natively clustersclassified pervious classrooms 1 classrooms septic 3, random_state=10) # display the data of five categories in three dimensions fig = plt.figure () ax = Axes3D (fig, rect= [0,0,1,1], elev=20 Azim=20) ax.scatter (train_samples [:, 0], train_samples [:, 1], train_samples [:, 2], marker='o', c=train_labels) plt.title ('Demo Data Map')

#% build BP model, using sgd optimizer Relu nonlinear mapping function BP = MLPClassifier (solver='sgd',activation = 'relu',max_iter = 500 1e-3, hidden_layer_sizes = (32) 32), random_state = 1) # Model training with warnings.catch_warnings (): warnings.filterwarnings ("ignore", category=ConvergenceWarning, module= "sklearn") BP.fit (train_samples Train_labels) # View the parameters of the BP model print (BP) #% for model prediction predict_labels = BP.predict (train_samples) # shows the predicted scatter plot fig = plt.figure () ax = Axes3D (fig, rect= [0,0,1,1], elev=20, azim=20) ax.scatter (train_samples [:, 0], train_samples [:, 1], train_samples [:, 2], marker='o' C=predict_labels) plt.title ('Demo Data Predict Map with BP Model') # shows the prediction score print ("prediction accuracy: {: .4f}" .format (BP.score (train_samples, train_labels)) # Visual prediction data print ("real category:", train_labels [: 10]) print ("prediction category:", predict_labels [: 10]) # accuracy report print (classification_report (train_labels)) Predict_labels) # calculate confusion matrix classes = [0,1,2,3] cofusion_mat = confusion_matrix (train_labels, predict_labels, classes) sns.set () figur, ax= plt.subplots () # draw sns.heatmap (cofusion_mat, cmap= "YlGnBu_r", annot=True, ax=ax) ax.set_title ('confusion matrix') # title ax.set_xticklabels (['] + classes, minor=True) ax.set_yticklabels (['] + classes Minor=True) ax.set_xlabel ('predict') # x-axis ax.set_ylabel (' true') # y-axis plt.show ()

#% # for new test data testing test_sample = np.array ([[- 1,0.1,0.1,0.1]]) print (f "{test_sample} category is:", BP.predict (test_sample)) print (f "{test_sample} category probability is:", BP.predict_proba (test_sample)) test_sample = np.array ([[- 1.2,10]) -91]]) print (f "{test_sample} category is:", BP.predict (test_sample)) print (f "{test_sample} category probability is:", BP.predict_proba (test_sample)) test_sample = np.array ([[- 12,-0.1,-0.1]) print (f "{test_sample} category is:" BP.predict (test_sample)) print (f "{test_sample} category probability is:", BP.predict_proba (test_sample)) test_sample = np.array ([100,90.1,-9.1]]) print (f "{test_sample} category is:", BP.predict (test_sample)) print (f "{test_sample} category probability is:", BP.predict_proba (test_sample))

2 Breast cancer classification prediction based on BP neural network #% breast cancer classification based on BP neural network # basic database import # import breast cancer data set from sklearn.datasets import load_breast_cancer# import BP model from sklearn.neural_network import MLPClassifier# import training set segmentation method from sklearn.model_selection import train_test_split # import prediction index calculation function and confusion matrix calculation function from sklearn.metrics import classification_report Confusion_matrix# Import drawing package import seaborn as snsimport matplotlib.pyplot as plt# Import 3D display tool from mpl_toolkits.mplot3d import Axes3D# Import Breast Cancer dataset cancer = load_breast_cancer () # View dataset information print ('length of breast_cancer dataset is:', len (cancer)) print ('breast_cancer dataset type is:' Type (cancer)) # Segmentation data into training set and test set cancer_data = cancer ['data'] print (' cancer_data data dimension is:', cancer_data.shape) cancer_target = cancer ['target'] print (' cancer_target tag dimension is:', cancer_target.shape) cancer_names = cancer ['feature_names'] cancer_desc = cancer [' DESCR'] # divided into training set and test set cancer_data_train Cancer_data_test = train_test_split (cancer_data,test_size=0.2,random_state=42) # training set cancer_target_train,cancer_target_test = train_test_split (cancer_target,test_size=0.2,random_state=42) # Test set

#%% # build BP model, using Adam optimizer Relu non-linear mapping function BP = MLPClassifier (solver='adam',activation = 'relu',max_iter = 1000 maxim alpha = 1eMub = 1eMub) # Model training BP.fit (cancer_data_train, cancer_target_train) #% Model prediction predict_train_labels = BP.predict (cancer_data_train) # Visualization real data fig = plt.figure () ax = Axes3D (fig, rect= [0,0) 1,1], elev=20, azim=20) ax.scatter (cancer_data_train [:, 0], cancer_data_train [:, 1], cancer_data_train [:, 2], marker='o', c=cancer_target_train) plt.title ('True Label Map') plt.show () # Visualization Forecast data fig = plt.figure () ax = Axes3D (fig, rect= [0,0,1,1], elev=20, azim=20) ax.scatter (cancer_data_train [:, 0], cancer_data_train [: 1], cancer_data_train [:, 2], marker='o', c=predict_train_labels) plt.title ('Cancer with BP Model') plt.show ()

#% display prediction score print ("prediction accuracy: {: .4f}" .format (BP.score (cancer_data_test, cancer_target_test)) # for category prediction of test set data predict_test_labels = BP.predict (cancer_data_test) print ("real tag of test set:\ n", cancer_target_test) print ("prediction tag of test set:\ n" Predict_test_labels) #% statistics of prediction results indicators statistics for each category of prediction accuracy, recall rate, F1 score print (classification_report (cancer_target_test, predict_test_labels))

#% calculate confusion matrix confusion_mat = confusion_matrix (cancer_target_test, predict_test_labels) # print confusion matrix print (confusion_mat) # display confusion matrix in the form of thermal map sns.set () figure, ax = plt.subplots () # draw sns.heatmap (confusion_mat, cmap= "YlGnBu_r", annot=True Ax=ax) # title ax.set_title ('confusion matrix') # x axis is the forecast category ax.set_xlabel (' predict') # y axis actual category ax.set_ylabel ('true') plt.show ()

Note: the census data prediction based on BP neural network has been done before, and the ape friends in need have private information.

III. Keys

The key point of BP neural network is forward propagation and error back propagation to update the parameters and minimize the loss.

It is an iterative algorithm, and the basic idea is:

The state and activation values of each layer are calculated first until the last layer (that is, the signal is propagated forward)

Calculate the error of each layer, and the calculation process of the error is pushed forward from the last layer (back propagation)

Update parameters (the goal is to reduce the error). Iterate through the first two steps until the stop criterion is met (for example, the difference in error between two adjacent iterations is small).

What are the advantages of Python: 1. Easy to use. Compared with traditional languages such as Java, Java and C #, Python has less strict requirements on code format. 2. Python is open source and everyone can see the source code and can be ported to many platforms. 3. Python is object-oriented and can support both process-oriented programming and object-oriented programming. 4. Python is an explanatory language, the program written by Python does not need to be compiled into binary code, and can be run directly from the source code; 5. Python is powerful and has many modules, which can basically achieve all the common functions.

After reading the above, do you have any further understanding of how to make predictions based on BP neural networks in Python? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.