Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "Python uses impala package to connect hive error report". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Since it is an intranet test environment and there is no external network, it is natural to create a python virtual environment locally with conda, install all the required packages and then move them over.

The operation is as fierce as a tiger, and all kinds of holes are filled, and then it is found that the as_pandas in impala.util does not work, because the pandas it depends on does not work, in the final analysis, it is the numpy import that pandas depends on:

ImportError: Something is wrong with the numpy installation.

While importing we detected an older version of numpy in

['/... / anconda3/lib/site-packages/numpy'].

One method of fixing this is to repeatedly uninstall numpy

Until none is found, then reinstall this version.

The positioning problem is the _ _ init__.py section stuck in the numpy source code:

If not (hasattr (multiarray,'_ multiarray_umath') and

Hasattr (umath,'_ multiarray_umath')):

Import sys

Path = sys.modules ['numpy']. _ _ path__

Msg = ("Something is wrong with the numpy installation."

"While importing we detected an older version of"

"numpy in {} .One method of fixing this is to repeatedly uninstall"

"numpy until none is found, then reinstall this version."

Raise ImportError (msg.format (path))

Oddly enough, it is normal to import version, showing that the current numpy version is 1.19.4

From numpy.version import version as _ _ version__

Print (_ _ version__)

The recommendation in the error message is to uninstall numpy completely and then reinstall it. The solutions have been seen all over the Internet, and they are all the same.

Adhering to the spirit of craftsmen and programmers, we must go back to the source and find another way to solve the problem!

Sorry, the tune is too high, in fact, the intranet environment is too inconvenient.

Since the wrong message says that there is an older version numpy, can't you just find a way to clean it up?

First, let's see what old version of numpy was installed before.

This is strange. Version 1.14.3 in the... / lib/site-packages directory has been manually deleted. How can there be any more? Looks like there are still remnants. Turn it over.\ Anaconda3\ conda-meta

Sure enough to find 1.14.3 information, delete these items run conda list numpy, show that there is only the latest version of version, 666 ~

Import numpy again, it is still the initial error report. Is there anything left of older version?

To solve the problem or to go back to the source, let's take a look at the principle of import:

For python, all modules loaded into memory are placed in sys.modules, so when you execute import, you first go to the list to check whether it has been added. If you're already in sys.modules, it's easy, just add the name of the module to the local space where we're calling the module. If you have not already put it in sys.modules, you need to look for the files of the module in order in the directory of all paths to sys.path. These files are generally suffixed with ".py", ".pyo", ".pyc", ".pyd", ".dll". After finding these modules, you can add these modules to sys.modules, and then import module name to the local.

So it's possible that the old and new versions crashed in sys.modules when importing numpy?

First take a look at what's in modules:

Import sys

Sys.modules

The culprit was finally found.

Run to the..\ Anaconda3\ Lib\ site-packages\ numpy\ core__pycache directory to delete these pyd files!

This is the end of the content of "Python uses impala package to connect hive errors". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.