Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of Hadoop-Centos7 installation and Hadoop storage planning, which is very detailed and has a certain reference value. Interested friends must read it!

1. Install VMware12 and CentOS7:

1. VMware 12 Professional Edition permanent license key: 5A02H-AU243-TZJ49-GTC7K-3C61N

2. Install Centos7:

Create a new virtual machine: typical (recommended)-install the operating system later-Linux (L)-version CentOS 64-bit-Virtual Machine name: centos64- stores the virtual disk as a single file-Custom hardware-Network connection: bridge mode-Select ISO image: CentOS-7-x86_64

2.2. Enable this virtual machine-Direction key selection-InstallingCentos7 (first item)-enter-Select English-Next-

Set installation options:

Install software options:

Manual zoning:

Storage planning: / boot: 200Mrecoery swap: generally 1x or 2x memory, remaining all root partitions /

Set the network and name:

Open the network:

Setting name:

2.3. Restart after installation is complete

Special note: the ifconfig view network command is not available after the installation is completed. There is no such command after the new version, so you can use the ip addr command to view it.

How to enable and use the ifconfig command (if the server can be connected to the Internet):

2.4.1. Find out which package provides the ifconfig command, and enter the following command: yum provides ifconfig

2.4.2. Net-tools package provides ifconfig command. Install net-tools package and enter the following command: yum install net-tools

2.4.3. After installation, you can use the ifconfig command.

Network configuration: vim / etc/sysconfig/network-scripts/ifcfg-eno16777736

Restart the Nic after configuration: / etc/init.d/network restart

2.5.1. Configure static IP address:

/ etc/sysconfig/network-scripts

Vim ifcfg-eno16777736 (name may be different)

Restart the network card:

Systemctl restart network.service

Ip addr

Install vim:yum install vim-enhanced-y

II. CentOS7 system optimization

1. Create a hadoop user

Useradd hadoop

Echo "hadoop123" >

2. Check whether the hadoop user has been added successfully

Su-hadoop

Su-hadoop

3. Exit and return to the root user

Exit

Exit

4. View the installed files on the system

Yum grouplist

Yum groupinstall "Compatibility Libraries"Base"Development tools"

Yum gruopinstall "debugging Tools"Dial-up Networking Support"

You can view the specific installed components through the yum groupinfo package group.

Run yum groupinstall "Compatibility libraries" to report an error:

[root@server1 mnt] # yum groupinstall Compatibility libraries

Loaded plugins: fastestmirror

Loading mirror speeds from cached hostfile

* base: mirrors.btte.net

* extras: mirrors.skyshe.cn

* updates: mirrors.yun-idc.com

Warning: group Compatibility does not exist.

Warning: group libraries does not exist.

Maybe run: yum groups mark install (see man yum)

No packages in any requested group available to install or update

Solution:

Yum clean all emptying information

Yum makecache create cache

Sudo yum groups install "Compatibility libraries"

You can view the specific installed components through the yum groupinfo package group.

5. Close SELINUX:

Sed-I's setting SELINUXFORCING'/ etc/selinux/config

Setenforce 0

Grep SELINUX=disabled / etc/selinux/config

6. View the default running level of the system: runlevel

7. Check which services are started automatically under the runlevel 3

Old directive:

Chkconfig

Chkconfig-- list | grep 3:on

Chkconfig-- list | grep 3:on | cut-d ""-F1

New directive:

Systemctl list-units-type=service

Systemctl list-unit-files | grep enabled

Systemctl list-unit-files-- type=service | grep enabled

8. Turn off the autorun service:

Old directive:

Chkconfig network (service name) off (turn off auto start) / on (turn on auto start)

New directive:

Systemctl disable (disable auto start) / enable (turn on auto start) network.service (service name)

9. Set the following services to start automatically:

Crond, ntpdate, lvm2-monitor,blk-availability, rsyslog, sshd,network

10. Close the firewall:

View status: iptables-L-n

Systemctl list-unit-files-- type=service | grep enabled

11. Rights management: elevate privileges to hadoop users (through sudo, you can execute commands that hadoop users do not have permission to execute)

Echo 'hadoop ALL= (ALL) NOPASSWD:ALL' > > / etc/sudoers

Su-hadoop

Sudo-l

Sudo su-(you can log in to root users without a password)

Exit

12. Increase the file descriptor

Echo'*-nofile 65535'> > / etc/security/limits.conf

Ulimit-a

Logout

Ulimit-a

This is a global change. You can log in to hadoop to verify:

Su-hadoop

Ulimit-a

Exit

3. Vmware clone virtual machine:

1. Manage-Clone-next-current status in the virtual machine-create full clone-virtual machine name

2. Configure IP address

3. Modify the hostname: check the hostname hostname and hostnamectl

First: hostname hostname

Hostname host name

In this way, only the temporary hostname can be modified, and when the machine is rebooted, the hostname changes back.

The second kind: hostnamectl set-hostname

Enter on the command line

Hostnamectl set-hostname

By modifying it in this way, you can permanently modify the host name!

The centos6 virtual machine eth0 failed to start the resolution:

In fact, it is because the format of the virtual machine network card used to connect the wireless network card and the physical line to the computer is different. The wireless network card uses eth0, and the physical line uses eth2.

Restart the network card

/ etc/init.d/network restart

Rm-f / etc/udev/rules.d/70-persistent-ipoib.rules

Cat / etc/sysconfig/network

Delete the two ifcfg-eth0 configurations and restart the system.

The above is all the contents of the article "sample Analysis of Hadoop-Centos7 installation and Hadoop Storage Planning". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.