Shulou(Shulou.com)06/01 Report--

This article is about the differences between commercial versions of hadoop. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Comparison of hadoop commercial version selection

I remember that when I first came into contact with hadoop, like most people will complain about the installation and deployment of hadoop, which is a headache for a novice, and it may take a whole day to install and configure the distributed environment. In the first contact with hadoop for a period of time, it can be said that the understanding of hadoop has always been at a relatively superficial level. Later, as I continued to explore and consult and communicate with the older gods in the circle (mainly from the great gods), I became more skilled in the understanding and application of hadoop.

As an experienced person, I'd like to share some questions with newcomers about the choice of hadoop versions. I hope I didn't know that hadoop has 1.0.x and 2.x versions as I foolishly knew at that time.

The current hadoop distribution, in addition to the open version of Apache, Huawei distribution, Intel distribution and Cloudera distribution. The third-party distributions mentioned above have been available for a relatively long time, in addition to the commercial distribution of DKhadoop that has sprung up in recent years.

The Hadoop distributions launched by most domestic companies are paid for, while the free distributions are mainly foreign, such as Apache distributions, Cloudera distributions and so on. Faced with so many hadoop versions, it's hard to choose. Let's briefly compare the advantages and disadvantages of some of these different versions, hoping to be helpful to beginners.

Apache distribution:

Advantages: the advantages of Apache distribution mainly focus on its completely open source free, community activity and detailed documentation.

Disadvantages: there are relatively many disadvantages of Apache distribution, which are shown in the following aspects:

1. Complex version management. Version management is chaotic, various versions emerge one after another, so that users do not know what to do.

two。 Complex cluster deployment, installation, configuration. Usually according to the needs of the cluster, a large number of configuration files are written and distributed to each node, which is prone to error and inefficient.

3. Complex cluster operation and maintenance. For the monitoring of the cluster, operation and maintenance, it is difficult to install other third-party software, such as ganglia,nagois, etc.

4. Complex ecological environment. In the Hadoop ecosystem, the selection and use of components, such as Hive,Mahout,Sqoop,Flume,Spark,Oozie, need to consider a lot of compatibility, whether the version is compatible, whether the components conflict, whether the compilation can pass, and so on. Often a lot of time is wasted to compile components to resolve version conflicts.

Advantages and disadvantages of third-party distributions: (e.g. CDH,HDP,MapR, etc.)

Advantages: the main advantages of third-party distributions are as follows:

1. Based on Apache protocol, 100% open source

two。 Compared with the native hadoop, it has improved in terms of compatibility, security and stability.

3. Clear version management and faster updates

4. Deployment, installation and configuration tools are provided, which greatly improves the efficiency of cluster deployment, and the cluster can be deployed in a few hours.

5. The operation and maintenance is simple. It provides tools for management, monitoring, diagnosis and configuration modification, which makes the management and configuration convenient, the positioning problem fast and accurate, and makes the operation and maintenance work simple and effective.

Disadvantages: the main drawback of third-party hadoop distributions is that it involves vendor locking, but this problem can be solved technically.

DKhadoop distribution:

The Dkhadoop distribution is the one I currently have access to and use. Compared with other third-party distributions in the market, it is more integrated, but it also retains all the advantages of open source systems. From the perspective of current use, the performance is much better than some third-party hadoop distributions used in the past! With regard to the issue of the DKhadoop distribution, those who are interested can consult and collect some information by themselves.

Thank you for reading! This is the end of the article on "what are the differences between the commercial versions of hadoop?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.