Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to build CentOS7.0 system configuration in Hadoop2.X/YARN environment". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to build CentOS7.0 system configuration in Hadoop2.X/YARN environment.

Why did I choose CentOS7.0

The official version of CentOS 7.0.1406 was released at 17:39:42 on July 7, 14. I have used many Linux. The reasons for choosing CentOS7.0 for the environment configuration of Hadoop2.X/YARN are as follows:

1. The interface uses RHEL7.0 's new GNOME interface style, which is not comparable to CentOS6.5/RHEL6.5! (of course, Fedora has adopted this style for a long time, but now the lack of Fedora package is not what it looks like.)

2, once, I also used RHEL7.0, its biggest problem is that YUM can not be used, and there will always be Warning prompt to sign up for purchase! To this end, it is necessary to modify the YUM source, as well as modify other files so that it can not be prompted to use, read so troublesome, there must be a lot of children's shoes dripping heart has been cold 4 big 7 bar!

3. Of course, I have used Ubuntu14.04/12.04 and elementaryOS in Ubuntu series. For some reason, I feel so ugly when I use Ubuntu14.04. But many people say that CentOS and RHEL are uglier, but they don't know which is the enterprise class and which is the overlord.

On the other hand, if you want visual effects, it is recommended to install an elementaryOS to try, of course, elementaryOS also has defects: it has not been updated for a long time, the 12-year-old system has not released a new version for more than 2 years, and the website has not been updated. At the same time, the elementaryOS model Ubuntu has long been updated, so it has exposed a lot of problems, among which the trouble is not to update the system at will, otherwise the black screen Then it is really pure command line, the beautiful system is suddenly hit back to the prototype, the heart is sure to be cool, and its updated apt-get is unable to install some matches, so elementaryOS is only for appreciation!

4. Some people have used openSUSE, which is said to be the most gorgeous system. In my opinion, when the gaudy KDE opens the file / software, it tickles in my heart. Why are you doing something useless and ugly? What?? In addition, the Chinese design of the latest openSUSE is really not satisfactory. In addition, the environment of openSUSE is not familiar with, the European design of GNU/Linux is still a little difficult to adapt, of course, I do not intend to change the flag to the European SUSE.

In fact, there are many versions of GNU/Linux, it is enough to choose a suitable one to play and use it all the time. There are many versions of Linux with different designs, but the use is more or less the same, just understand the "differences"! Of course, a word of advice: beginners just run the virtual machine to see the effect, do not use each model! After all, experience is also one of the driving forces for learning!

Second, experience the "difference" between CentOS7 and the previous version

Initial start

After installing the machine, the interface for booting into the system was changed. At first glance, I thought it was two cores, but there was originally one selected by rescue, and after pressing e, I would find that all the grub.conf information had come out. It was not easy to find the place where the kernel started, wrote a 1 (to enter single-user mode), and then entered rescue mode.

Character interface

After entering the system, is the graphical interface, want to enter the character interface, the result found / etc/inittab, found that it is almost an empty file, the document prompts you want to change runlevel, you can connect / lib/systemd/system/runlevel*.target soft to / etc/systemd/system/default.target (of course, the test, overlay is also OK), try it is OK, can be changed to the character interface runlevel3.

Configure network and hostname

Then, want to configure the network, enter / etc/sysconfig/network-script/ a look, I rely on, the network card renamed enp1s5, very interesting, after configuring the network to change the host name, to / etc/sysconfig/network to change the name, found that this file is also empty, according to the original version 6 to modify, restart after no effect, man a hostname, found that 7 renamed to / etc/hostname to change the name.

Local yum source and mount

Next, you want to configure a local yum source, hang up the CD after matching it, hang it manually, and then directly echo "mount." In rc.local, after all, I started the execution script, but I didn't hang it up after rebooting. I went into rc.local to have a look. 7 had to manually add chmod + x rc.local. Yes, I had to add execution permission manually. Could it be that I installed 6 before this system had to manually add execution permission? I don't remember! Of course, after adding permissions, the boot starts and executes the mount command of rc.local.

LVM and xfs

Later, I wanted to dress up a lvm for fun, so, ah, I partitioned, formatted (specially formatted to xfs), pv,vg,lv step by step, no problem, and then thought about zooming in and out, and then something went wrong. When I executed resize2fs, how did I keep reporting an error of superblock or something? what's going on? then man resize2fs, I found that this command only supports the file system of ext. Oh, my God, then how do I install the system is lvm? I haven't solved this problem yet. There should be a solution.

Dhcp and Servic

Then, want to dress up a dhcp to play, surprised to find that after all the packaging of dhcp, can not service dhcpd start, and then found that there is not a dhcp similar thing in / etc/init.d/, after all, there is dhcpd,dhcpd6,dhcrelay before 6, and then found that / sbin/dhcpd has startup files, is it, is it that the future services want to start service, all have to manually compile their own? No, it's in / usr/lib/systemd/system/dhcpd.service, which needs to be modified a lot, and then add permissions to execute the service service restart/stop. Yes, but it points to the systemctl restart/start/stop service .service

Iptables

It is also found that iptables is not under / etc/init.d/ or / sbin as a service this time.

The firewall has been changed from iptables to firewalld

Third, the system configuration turns off the firewall

Systemctl stop firewalld.service # stop firewall

Systemctl disable firewalld.service # prevents firewall from booting

The basic operation format of the systemctl command is: systemctl action service name.service action: start,stop,restart,status,enable,disable,is-enabled

4. Configure Linux network

1. Modify hostname

The hostname of CentOS 7.0is modified by root users in / etc/hostname:

[root@hadoop1 ~] # vim / etc/hostname # Edit / etc/hostname [root@hadoop1 ~] # cat / etc/hostname # View / etc/hostnamehadoop1 [root@hadoop1 ~] #

Previous versions of CentOS were modified by the root user in / etc/sysconfig/network:

[root@hadoop1 ~] # vim / etcsysconfig/network

The setting items for "/ etc/sysconfig/network" are as follows:

Whether the NETWORKING uses the network GATEWAY default gateway to match the default gateway in the properties file of the next configuration IP. The interface name of the IPGATEWAYDEV default gateway HOSTNAME hostname DOMAIN domain name

Hostname takes effect after reboot

2. Modify the IP of the current machine

Step1: enter the configuration file directory

Now that we have root permissions, we cd to the directory of the network configuration files and list the files in the directory.

Step2: modifying configuration fil

In this directory, the file name "ifcfg-ens192" file is the main file of the network configuration. Vi it!

Modify or add:

ONBOOT=yes BOOTPROTO=static IPADDR=IP address NETMASK=255.255.255.0 # Subnet Mask GATEWAY=192.168.30.1 # Gateway address

Step3: restart network services

After modifying the configuration file,: wq! Save and exit! Then restart the network.

Step4: testing

Use the ifconfig command to see if ip is set up successfully, and then ping the network. It should be a success at this time!

3. Modify DNS (optional, depending on the situation)

If the network environment has DNS requirements, then configure, otherwise, do not match!

Format:

Nameserver DNS address

[root@hadoop1] # vi / etc/resolv.conf nameserver 202.131.80.1 # on demand nameserver 202.131.80.5 # on demand

4. Configure the hosts file

The file "/ etc/hosts" is used to configure the DNS server information that the host will use, and is used to record the corresponding [HostName and IP] of each host connected within the LAN. When users are connecting to the network, they first look for the file to find the corresponding IP address of the corresponding host name (or domain name).

We want to test the connectivity between the two machines, generally using "IP of the ping machine". If you want to use the "host name of the ping machine" to find the machine with that name, the solution is to modify the "/ etc/hosts" file. By writing the IP address of each host in the LAN and the one-to-one correspondence of HostName into this file, you can solve the problem.

For example, the machine is "hadoop1:59.67.107.80" and the machine is "hadoop2:59.67.107.79" with the command "ping" memory connection test. The test results are as follows:

From the values in the above figure, directly test the IP address and be able to ping, but test the hostname. If there is no ping connection, it will prompt "unknown host-- unknown host". At this time, check the "/ etc/hosts" file content of "hadoop1".

If the ping hadoop1 is not available, it will be displayed:

If the ping hadoop1 General rules show:

If it is found that there is no "59.67.107.79 hadoop2" content, so this machine can not parse the host name of the machine as "hadoop2".

In the Hadoop cluster configuration, you need to add the IP and hostnames of all machines in the cluster to the "/ etc/hosts" file, so that Master and all Slave machines can communicate not only through IP, but also through hostnames. So add the following to the end of the "/ etc/hosts" file on all machines:

Because this article is described in a configured environment, my / etc/hosts shows:

It is generally under the private network, so add it as follows (192.168.1.1 is the gateway):

192.168.1.2 hadoop1192.168.1.3 hadoop2192.168.1.4 hadoop3192.168.1.5 hadoop4

Add with the following command:

Vim / etc/hosts

Now we are doing a ping test on the hostname of the machine "hadoop2" to see if the test is successful.

From the figure above, we have been able to ping with the hostname, indicating that the content we just added can be parsed for DNS in the local area network, so what is left is the same configuration on the rest of the Slave machines. And then test it.

Thank you for your reading, the above is the content of "how to build CentOS7.0 system configuration in Hadoop2.X/YARN environment". After the study of this article, I believe you have a deeper understanding of how to build CentOS7.0 system configuration in Hadoop2.X/YARN environment, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.