Shulou(Shulou.com)06/03 Report--

This article will explain in detail how to implement a simple three-layer neural network in Python. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

1. Initialization

What we want to initialize includes: the number of neurons in each layer of the neural network (this is based on the actual problem input and output, we set it to a customizable quantity).

The weight value of data transmitted between different layers.

Activation function (simulates natural neurons, where stimulation signals need to reach a certain extent to activate neurons)

The following is the code:

Def _ init__ (self, input_nodes_num, hidden_nodes_num, output_nodes_num, lr): # initialize the number of neurons. You can directly modify self.input_nodes = input_nodes_num self.hidden_nodes = hidden_nodes_num self.output_nodes = output_nodes_num self.learning_rate = lr # initialize the weight value Random initialization using normal distribution function, the mean value is 0 The variance is the square of the number of neurons self.w_input_hidden = numpy.random.normal (0.0, pow (self.hidden_nodes,-0.5), (self.hidden_nodes, self.input_nodes)) self.w_hidden_output = numpy.random.normal (0.0, pow (self.output_nodes,-0.5)) (self.output_nodes, self.hidden_nodes) # initialize the activation function The activation function uses the Sigmoid function, which is smoother and closer to the natural neuron behavior pattern. # lambda defines an anonymous function self.activation_function = lambda x: scipy.special.expit (x) pass.

Let's explain some of the programming knowledge in the above code snippet. The first is _ _ init__ (), which is the constructor of a class, which is called when building an object of a class, so we put the neural network initialization code into this function.

Self.w_input_hidden = numpy.random.normal (0.0, pow (self.hidden_nodes,-0.5), (self.hidden_nodes, self.input_nodes))

This code uses the random.normal () function in the numpy library to initialize the weight value for data transfer between the input layer and the hidden layer, which randomly generates a

The matrix of self.hidden_nodes*self.input_nodes (hidden_nodes and input_nodes represent the number of neurons in the hidden layer and the input layer).

Self.activation_function = lambda x: scipy.special.expit (x)

This code defines an anonymous function using lambda and assigns it to the activation function. The function is the sigmoid function, which is a smooth curve that is close to the way natural neurons respond to stimuli.

two。 Forecast

According to the normal order, the training should be carried out after the initialization is completed, but because the training is more complex and the prediction is relatively simple and easy to implement, we first complete this part of the code. In the prediction part, we need to process the input information and transmit it to the hidden layer neuron after weighted summation. After activation function and weighted summation again, it is transmitted to the output layer and processed by the output layer neuron to get the final result. The code snippet is as follows:

Def query (self, inputs_list): # transpose row vectors into column vectors to better separate each set of data It is convenient for the subsequent matrix point multiplication operation inputs = np.array (inputs_list, ndmin=2). After T # weighted summation, the hidden layer output hidden_inputs = np.dot (self.w_input_hidden) is obtained by sigmoid function. Inputs) hidden_outputs = self.activation_function (hidden_inputs) # after weighted summation, the final output final_inputs = np.dot (self.w_hidden_output, hidden_outputs) final_outputs = self.activation_function (final_inputs) # is obtained through the sigmoid function to get the output data column return final_outputs

This code has nothing to say, it is relatively simple, just follow the above steps of the author. If you don't understand anything, you can read the comments or leave comments.

3. Training

The training problem of neural network is more complex, which involves forward and back propagation of neural network, chain rule of calculus, matrix operation, partial differential derivation and gradient descent algorithm, which are some basic knowledge of machine learning. I won't go into too much detail here. I'll send a new article in detail in a few days. Let's take a look at the main tasks of training code snippets:

Training, like prediction, first reads some inputs and predicts the output. the difference is that in the training phase, we get the data from the training data set, and we know what the correct output is. In the prediction phase, we only know the input and the output needs to be predicted through the model we trained. First, the input is read in the training phase and predicted according to the current model.

The weights between each layer are updated based on the errors of the training prediction results and the labeled actual results.

Here's the code:

Def train (self, inputs_list, targets_list): # convert the data in the training set and test set into column vectors inputs = np.array (inputs_list, ndmin=2). T targets = np.array (targets_list, ndmin=2). T # the input of the hidden layer is the point multiplication of the training set and the weight value The output is the output of the activation function hidden_inputs = np.dot (self.w_input_hidden, inputs) hidden_outputs = self.activation_function (hidden_inputs) # the input of the output layer is the output of the hidden layer The output is final_inputs = np.dot (self.w_hidden_output, hidden_outputs) final_outputs = self.activation_function (final_inputs) # loss function output_errors = targets-final_outputs # the error of the hidden layer is the point multiplication hidden_errors = np.dot of the transpose and output error of the weight matrix (self.w_hidden_output.T Output_errors) # Update the weights self.w_hidden_output + = self.learning_rate * np.dot ((output_errors * final_outputs * (1.0-final_outputs)) Np.transpose (hidden_outputs)) self.w_input_hidden + = self.learning_rate * np.dot ((hidden_errors * hidden_outputs * (1.0-hidden_outputs)) Np.transpose (inputs))

The above code snippet may be a little confused or complex for some students who are new to machine learning or deep learning, but it is only a comprehensive application of back propagation algorithm, chain rule and partial derivative. I will tell you my experience in another essay (which may not be good). If you are interested, you can take a look at it.

4. test

The construction of the three-layer neural network is completed, and I tested it with mnist training set and test set. The code and results are as follows:

# initialize the number of neurons in each layer. The number of input neurons in the middle period depends on the dependent variable read, while the number of output neurons depends on the possibility of classification input_nodes = 784hidden_nodes = 100output_nodes = 1 percent learning rate Learning _ rate = 0.2n = NeuralNetwork (input_nodes, hidden_nodes, output_nodes, learning_rate) # get the training set information training_data_file = open ('data/mnist_train.csv', 'r') training_data_list = training_data_file.readlines () training_data_file.close () for record in training_data_list: all_values = record.split (' ') inputs = (numpy.asfarray (all_ values [1:]) / 255.0 * 0.99) + 0.01targets = numpy.zeros (output_nodes) + 0.01targets [int (all_values [0])] = 0.99n.train (inputs, targets) passprint (' train values') test_file = open ('data/mnist_test.csv' 'r') test_list = test_file.readlines () test_file.close () m = np.size (test_list) j = 0.0for record in test_list: test_values = record.split (',') np.asfarray (test_values) results = n.query (np.asfarray (test_ values [1:]) if results [int (test_values [0])] = = max (results): J + = 1 passprint ("correct rate is "+ str (JGOP))

This is the end of this article on "how to achieve a simple three-layer neural network in Python". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.