Shulou(Shulou.com)06/02 Report--

What is the principle of Autograph mechanism? aiming at this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

First, the mechanism and principle of Autograph

What happens next when we decorate a function with @ tf.function?

For example, let's write down the following code.

Nothing happened back there. The signature of such a function is simply recorded in the Python stack.

What happened next when we first called this function decorated with @ tf.function?

For example, let's write down the following code.

Two things happened.

The first thing is to create a calculation diagram.

That is to create a static calculation chart, track and execute the Python code in the function body, determine the Tensor type of each variable, and add the operator to the calculation diagram according to the execution order.

In this process, if autograph=True is enabled (enabled by default), the Python control flow will be transferred to the control flow in the TensorFlow chart.

The main purpose is to convert if statements into tf.cond operator expressions, while and for loop statements into tf.while_loop operator expressions, and add tf.control_dependencies to specify execution order dependencies when necessary.

This is equivalent to executing a statement similar to the following in tensorflow1.0.

The second thing is to execute the calculation diagram.

This is equivalent to executing the following statement in tensorflow1.0:

So what we see first is the result of the first step: the Python calls the standard output stream to print the "tracing" statement.

Then you see the result of the second step: TensorFlow calls the standard output stream to print 1, 2, and 3.

What happens when we call this function decorated with @ tf.function with the same input parameter type again?

For example, let's write down the following code.

Only one thing will happen, and that is the second step of the above step, performing the calculation diagram.

So this time we didn't see the result of printing "tracing".

What happens when we call this function decorated with @ tf.function with different input parameter types again?

For example, let's write down the following code.

Because the type of the input parameter has changed, the calculation chart that has been created cannot be used again.

There are two things you need to do again: create a new calculation chart and perform a calculation chart.

So what we will see first is the result of the first step: the Python calls the standard output stream to print the "tracing" statement.

Then you see the result of the second step: TensorFlow calls the standard output stream to print 1, 2, and 3.

It is important to note that if the argument entered when calling a function decorated with @ tf.function is not of type Tensor, the graph will be recreated each time.

For example, let's write down the following code. The calculation chart is recreated both times. Therefore, it is generally recommended that you pass in the Tensor type when calling @ tf.function.

Second, re-understand the Autograph coding specification.

By understanding the mechanism of Autograph above, we can understand the three recommendations of the Autograph coding specification.

1. Functions modified by @ tf.function should try to use functions in TensorFlow instead of other functions in Python.

Explanation: the functions in Python will only be used in the stage of tracking the execution of the function to create a static diagram, ordinary Python functions cannot be embedded in the static diagram, so when called again after the construction of the diagram, these Python functions are not calculated, while the functions in TensorFlow can be embedded in the diagram. Using the normal Python function will result in inconsistent output of [eager execution] before being modified by @ tf.function and [static graph execution] modified by @ tf.function.

2. Avoid defining tf.Variable inside @ tf.function-decorated functions.

Explanation: if tf.Variable is defined inside the function, then when [eager executes], this behavior of creating tf.Variable occurs every time the function is called. However, in the case of [static diagram execution], this behavior of creating a tf.Variable will only occur when the first step is to track the logic of the Python code to create a calculation graph, which will result in inconsistent output of [eager execution] before being modified by @ tf.function and [static graph execution] after being modified by @ tf.function. In fact, TensorFlow generally reports an error in this case.

3. A function modified by @ tf.function cannot modify structural type variables such as Python lists or dictionaries outside the function.

Explanation: the static calculation graph is compiled into C++ code and executed in the TensorFlow kernel. The data structure variables such as lists and dictionaries in Python cannot be embedded in the calculation chart, they can only be read when creating the calculation graph, and the data structure variables such as lists or dictionaries in Python cannot be modified when the calculation chart is executed.

The answer to the question about the principle of Autograph mechanism is shared here. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel to learn more about it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.