Shulou(Shulou.com)06/01 Report--

This article shares with you the content of a sample analysis of compiling and building an installation package for TensorFlow source code. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Recently, the company assigned us two virtual machine servers for intensive learning and training. after installing the TensorFlow environment in the virtual environment, we found the following error in import tensorflow:

So I went to Google to search for the reason for this error, and found that it is because our server's CPU does not support the AVX instruction set, and the TensorFlow installed using pip needs to rely on the AVX instruction set. In order to confirm whether our CPU really does not support the AVX instruction set, I used the cat / proc/cpuinfo command to check the current CPU instruction set support and found that our CPU does not support the AVX instruction set. After searching on the Internet, we can use the source code of TensorFlow to compile and build a version of TensorFlow, so we can use TensorFlow on machines that do not support the AVX instruction set. So I followed the steps given on the official website https://www.tensorflow.org/install/source to try to compile and build TensorFlow from the source code myself. Before building TensorFlow, we need to do two things: download the source code of TensorFlow and install the Bazel build tool

When downloading TensorFlow source code, if the network speed is good, you can directly use the following command to clone the TensorFlow code base source code from the GitHub repository:

If the speed of the git clone https://github.com/tensorflow/tensorflow.gitcd tensorflow is worrying, you can download the zip version directly and then upload it to the server. I use the latter.

Note: the code base here defaults to the master development branch.

Download and install Bazel build tools here I would like to say that, in general, there are two ways to install Bazel, one is to use Bazelisk to install, the other is to install manually, I use the latter. You will be told on the official website that you can install any version specified in tensorflow/configure.py between _ TF_MIN_BAZEL_VERSION and _ TF_MAX_BAZEL_VERSION in GitHub, but this is a pit. Because here he said that you can install any version of 3.10 3.99, so I installed a 3.70 version, and when it is really compiled, he will tell you that this version does not match and requires you to install 3.10, so you can only uninstall and reinstall. Next we need to follow bazel, and first install some dependencies:

Apt install curl gnupg curl-fsSL https://bazel.build/bazel-release.pub.gpg | gpg-- dearmor > bazel.gpg mv bazel.gpg / etc/apt/trusted.gpg.d/ echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee / etc/apt/sources.list.d/bazel.list we can install it directly using the apt update & & sudo apt install bazel-3.1.0 command. So far, we have installed the bazel compilation tool and downloaded the source code for TensorFlow, so we are ready to compile and build TensorFlow. Before that, we need to install some corresponding dependencies, using the following command: pip install six numpy wheel setuptools mock 'future > = 0.17.1' pip install keras_applications-- no-deps pip install keras_preprocessing-- no-deps

Then we use gcc-- version to check our version of gcc. Mine is 7.5.0, which comes with 18.04, and is available.

Then comes the real compilation. First we go to the TensorFlow source code downloaded from GitHub and execute. / configure

This time will ask you to do some configuration-related information: Please specify the location of python. [Default is / root/miniconda3/envs/ray/bin/python3]:

This sentence asks you to enter the location of Python, enter directly if it is correct, and enter the directory where your Python is located if you want to modify it; Found possible Python library paths: / root/miniconda3/envs/ray/lib/python3.8/site-packages Please input the desired Python library path to use. Default is [/ root/miniconda3/envs/ray/lib/python3.8/site-packages]

This step allows you to enter the location of the Python library, as above, I enter directly here.

Do you wish to build TensorFlow with ROCm support? [y build N]: n this step is to ask if you want to build a version of TensorFlow that supports ROCm. Here I choose N.

Do you wish to build TensorFlow with CUDA support? [CUDA N]: n this step is to ask whether you want to build a TensorFlow version that supports CUDA, because I am going to use this in the CPU version of the distributed cluster, and this server does not use CUDA, so I choose N.

Do you wish to download a fresh release of clang? (Experimental) [yamp N]: n this step is to ask you if you want a narrow clang to be release version. Here I chose Y at the beginning. Because of my poor Internet speed, I downloaded and reported an error, and then chose N. Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is-march=native-Wno-sign-compare]: this step is to let you enter a compile-time optimizer. Here I will directly use the default and enter directly.

Would you like to interactively configure. / WORKSPACE for Android builds? [y Android SDK N]: n this step is to ask whether you need to support Android. I don't need it on my side, so go straight to N, otherwise it will let you choose your version of Android SDK and so on.

When all this is configured, a configuration message for you will pop up.

Then it's time to really start compiling your TensorFlow source code. Since I'm using a version without CUDA, I can simply use the following command: bazel build-- config=opt / / tensorflow/tools/pip_package:build_pip_package compiles and you will see the following message: (since I didn't have a screenshot and used a screen image, it took me more than 7500 seconds to use the old CPU)

After compiling, we can type the following command in the TensorFlow source directory to build the final pip installation package:. / bazel-bin/tensorflow/tools/pip_package/build_pip_package / tmp/tensorflow_pkg here, / tmp/tensorflow_pkg is the location where I defined the output of the final installation package. When finished, we can cd to the / tmp/tensorflow_pkg directory to see the contents:

As you can see, our final installation version of pip is complete, and then we can type the following command on the command line to install: pip install tensorflow-2.5.0-cp38-cp38-linux_x86_64.whl

After the installation is complete, we can try to import TensorFlow in the Python interactive environment to see if there are any errors:

We found that it can be imported and used normally, and at this point, the entire TensorFlow source code is compiled, built, and installed. Thank you for reading! On the "TensorFlow source code compilation build installation package sample analysis" this article is shared here, I hope the above content can be of some help to you, so that you can learn more knowledge, if you think the article is good, you can share it out for more people to see it!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.