Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to use Tensorflow". The explanation content in this article is simple and clear, easy to learn and understand. Please follow the ideas of Xiaobian and go deep into it slowly to study and learn "how to use Tensorflow" together.

Tensorflow graphs

Tensorflow is a graph-based parallel computing model. For an understanding of graph, please refer to the official documentation. For example, if we calculate a=(b+c)<$(c+2)a=(b + c) * (c + 2)a=(b+c)<$(c+2), we can split the formula into one:

d = b + ce = c + 2a = d * e

Converted to graph form:

> It is overkill to say that a simple formula is made like this, but we can find that $d = b + c$and $e = c + 2$are unrelated, that is, they can be calculated in parallel. For more complex CNNs and RNNs, graph's parallel computing power will be better demonstrated.

In practice, a three-layer (single hidden layer) neural network based on Tensorflow is shown in the following figure:

! [Write picture description here](http://adventuresinmachinelearning.com/wp-content/uploads/2017/03/TensorFlow-data-flow-graph.gif) **Tensorflow data flow graph**

In the diagram above, circular or square nodes are called nodes, and the data flow flowing through the nodes is called tensors. For more information about the tensors, see the official documentation.

Tensor of order 0 == Scalar

Tensor of order 1 == Vector (one-dimensional array)

2nd order tensor == 2D array

…

n order tensor == n dimensional array

Relationship between Tensor and Node:

  If the input tensor has dimensions of 5000×645000 \times 645000×64, meaning there are 5000 training samples, each sample has 64 features, then the input layer must have 64 nodes to accept these features.

The three-layer network shown above includes an input layer (input in the figure), a hidden layer (named ReLU layer here to indicate that its activation function is ReLU), and an output layer (Logit Layer in the figure).

As you can see, there are relevant tensors at each level that flow into the Gradient node to compute gradients, and then these gradient tensors flow into the SGD Trainer node to optimize the network (i.e. update network parameters).

Tensorflow represents neural networks through graphs to achieve parallel computation of networks and improve efficiency. Below we will introduce TensorFlow's basic syntax with a simple example.

A Simple TensorFlow example

Calculate a=(b+c)<$(c+2)a = (b + c) * (c +2) a =(b+c)<$(c+2),1. Definition data:

import tensorflow as tf#First, create a TensorFlow constant =>2const = tf.constant(2.0, name ='const ')#Create TensorFlow variables b and cb = tf.Variable(2.0, name ='b')c = tf.Variable(1.0, dtype=tf.float32, name ='c ')

As above, in TensorFlow, constants are defined using tf.constant() and variables are defined using tf.Variable(). Tensorflow can automatically perform data type detection, for example: assign 2.0 to default to tf.floor32, but it is best to explicitly define it. For more information about TensorFlow data types, see the official documentation.

2. Definition of operation (also called TensorFlow operation):

#create operationd = tf.add(b, c, name='d')e = tf.add(c, const, name='e')a = tf.multiply(d, e, name='a')

In TensorFlow,+−× $>+-\times \div+−× $> has its own special function representation. TensorFlow actually defines enough functions to represent all mathematical operations, and of course some mathematical operations are overloaded with operators, but to be on the safe side, I recommend that you use functions instead of operators.

**!! All variables in TensorFlow must be initialized before they can be used. ** The initialization method is divided into two steps:

Definition of initialization operation

Run initialization operation

# 1. definition init operationinit_op = tf.global_variables_initializer()

The TensorFlow graph has been built above. The next step is to calculate and output it.

To run graph, you need to call tf.Session() to create a session. Session is the handle we interact with graph. For more information on sessions, see the official documentation.

# sessionwith tf.Session() as sess: # 2. Run init operation sess.run(init_op) #Calculate a_out = sess.run(a) print("Variable a is {}".format(a_out))

It is worth mentioning that TensorFlow has an excellent visualization tool TensorBoard, which is detailed in the official documentation.

The TensorFlow placeholder

An improvement to the above example: Make variable b accept arbitrary values. The way values are received in TensorFlow is as placeholders, created by tf.placeholder().

#Create placeholder b = tf.placeholder(tf.float32, [None, 1], name ='b ')

The second parameter value is [None, 1], where None means uncertain, i.e. uncertain about the size of the first dimension, which can be any size. In particular, corresponding to the number of tensors (or the number of samples), the number of tensors input may be 32, 64…

Now, if you get the result of the calculation, you need to feed the placeholder b value during the run, specifically changing a_out = sess.run (a) to:

np.newaxis: https://www.jianshu.com/p/78e1e281f698

a_out = sess.run(a, feed_dict={b: np.arange(0, 10)[:, np.newaxis]})

Output:

Variable a is [[ 3.] [ 6.] [ 9.] [ 12.] [ 15.] [ 18.] [ 21.] [ 24.] [ 27.] [ 30.]] A Neural Network Example

An example of a neural network is the MNIST dataset.

1. Loading data:

from tensorflow.examples.tutorials.mnist import input_datamnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

One_hot=True represents one-hot encoding of label, for example label 4 can be represented as [0, 0, 0, 1, 0, 0, 0, 0]. This is the format required by the output layer of the neural network.

Setting things up

2. Defining hyperparameters and placeholders

#Hyperparameterlearning_rate = 0.5epochs = 10batch_size = 100# placeholder#Input picture is 28 x 28 pixels = 784x = tf.placeholder(tf.float32, [None, 784])#Output one-hot codes of 0-9 y = tf.placeholder(tf.float32, [None, 10])

Again, none in [None, 784] means arbitrary value, especially corresponding to the number of tensors.

3. Define parameters w and b

# hidden layer => w, bW1 = tf.Variable(tf.random_normal([784, 300], stddev=0.03), name='W1')b1 = tf.Variable(tf.random_normal([300]), name='b1')# output layer => w, bW2 = tf.Variable(tf.random_normal([300, 10], stddev=0.03), name='W2')b2 = tf.Variable(tf.random_normal([10]), name='b2')

Here, to understand that the two parameters w and b of the fully connected layer need to be initialized randomly, tf.random_normal() generates a normally distributed random number.

4. structural hidden layer network

#Compute output y_ = tf.nn.softmax(tf.add(tf.matmul(hidden_out, W2), b2))

The above code corresponds to the formula:

5. Construct Output (Predicted Value)

#Compute output y_ = tf.nn.softmax(tf.add(tf.matmul(hidden_out, W2), b2))

For single-label multi-classification tasks, the activation function of the output layer is tf.nn.softmax(). For more on softmax see Wikipedia.

6. BP-Definition of loss

The loss is the cross-entropy and the formula is

The formula is divided into two steps:

Compute cross entropy for n labels

averaging m samples

7. BP part-definition optimization algorithm

#Create optimizer, determine optimization goal optimizer = tf.train. Gradient DescentOptimizer (learning_rate=learning_rate).minimizer(cross_entropy)

For more optimization algorithms in TensorFlow, see the official documentation.

8. Define initialization operation and accuracy node

# init operatorinit_op = tf.global_variables_initializer()#create accuracy node correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax (y_, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

correct_prediction returns an m×1m\times 1m×1 tensor, where a value of True/False indicates whether the prediction was correct.

Setting up the training

9. start training

#Create session with tf.Session() as sess: #variable initialization sess.run(init_op) total_batch = int(len(mnist.train.labels) / batch_size) for epoch in range(epochs): avg_cost = 0 for i in range(total_batch): batch_x, batch_y = mnist.train.next_batch(batch_size=batch_size) _, c = sess.run([optimizer, cross_entropy], feed_dict={x: batch_x, y: batch_y}) avg_cost += c / total_batch print("Epoch:", (epoch + 1), "cost = ", "{:.3f}".format(avg_cost)) print(sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels}))

Output:

Epoch: 1 cost = 0.586Epoch: 2 cost = 0.213Epoch: 3 cost = 0.150Epoch: 4 cost = 0.113Epoch: 5 cost = 0.094Epoch: 6 cost = 0.073Epoch: 7 cost = 0.058Epoch: 8 cost = 0.045Epoch: 9 cost = 0.036Epoch: 10 cost = 0.027Training complete! 0.9787

Visualize the training process with TensorBoard:

Thank you for reading, the above is the content of "how to use Tensorflow", after learning this article, I believe everyone has a deeper understanding of how to use Tensorflow, the specific use situation still needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.