In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-03-29 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >
Share
Shulou(Shulou.com)06/03 Report--
I. Node preparation
1. Node configuration
hostname
g0
matada node
g1
matadata node g2
matadata node g3
data node
g4
data node g5
data node
2. Install ntp
yum install ntp -yservice ntpd startsystemctl enable ntpd
Second, install gpfs
1. Set ssh to login without password
salt-ssh -i '*' cp.get_file salt://id_rsa /root/.ssh/id_rsasalt-ssh -i '*' cp.get_file salt://id_rsa.pub /root/.ssh/id_rsa.pubsalt-ssh -i '*' cmd.run 'chmod 600 /root/.ssh/id_rsa'salt-ssh -i '*' cmd.run 'chmod 600 /root/.ssh/id_rsa.pub'
2. Install gpfs
salt-ssh -i 'g*' cmd.run 'cd /root/gpfs && tar xvzf GPFS_4.1_ADV_LSX.tar.gz'salt-ssh -i 'g*' cmd.run 'cd /root/gpfs/gpfs_install-4.1.0-0_x86_64 --silent'salt-ssh -i 'g*' cmd.run 'yum localinstall /usr/lpp/mmfs/4.1/*rpm -y'salt-ssh -i 'g*' cmd.run 'bash /root/gpfs/Spectrum_Scale_Advanced-4.1.1.3-x86_64-Linux-update'salt-ssh -i 'g*' cmd.run 'mv /usr/lpp/mmfs/4.1.1.3/gpfs.hadoop-connector-2.7.0-2.x86_64.rpm /usr/lpp/mmfs/4.1.1.3/gpfs.hadoop-connector-2.7.0-2.x86_64.rpm.orig'salt-ssh -i 'g*' cmd.run 'yum localinstall /usr/lpp/mmfs/4.1.1.3/*rpm -y'
Compile the gpfs kernel module and execute it on one of the nodes
3.1 Compile kernel module
cd /usr/lpp/mmfs/src/yum install kernel-devel cpp gcc gcc-c++make LINUX_DISTRIBUTION=REDHAT_AS_LINUX Autoconfigmake Worldmake InstallImages
3.2 Compile the gpfs kernel module into an rpm package
yum install rpm-buildmake rpm
3.3 Copy the compiled rpm package to other nodes and install it.
4. Set path variable
salt-ssh -i 'g*' file.append /etc/profile 'export PATH=$PATH:/usr/lpp/mmfs/bin/'
Create a gpfs cluster
1. Create a gpfs node
Edit the profile used to create the node
cat /root/gpfs_config/nodefileg0:quorum-managerg1:quorum-managerg2:quorum-managerg3:managerg4:managerg5:managerEOF
Create a gpfs cluster
mmcrcluster -N /root/gpfs_config/nodefile -p g0 -s g1 -C gpfscluster -A -r /usr/bin/ssh -R /usr/bin/scp Parameters: -N indicates the node's configuration file-p indicates the primary node-s indicates the Secondary cluster configuration server-C cluster name-A indicates that the GPFS daemon does not shut down when a node restarts. - R and-R are communication protocols.
2. Authorized license
mmchlicense server --accept -N g0,g1,g2,g3,g4,g5
3. Add NSD
Edit nsdfile.cfg
%pool: pool=system blockSize=512K layoutMap=cluster%pool: pool=datapool blockSize=2M layoutMap=cluster writeAffinityDepth=1 blockGroupFactor=256%nsd: device=/dev/vdb servers=g0 nsd=metansd00 usage=metadataOnly pool=system%nsd: device=/dev/vdb servers=g1 nsd=metansd01 usage=metadataOnly pool=system%nsd: device=/dev/vdb servers=g2 nsd=metansd02 usage=metadataOnly pool=system %nsd: device=/dev/vdb servers=g3 nsd=datansd00 usage=dataOnly pool=datapool%nsd: device=/dev/vdb servers=g4 nsd=datansd01 usage=dataOnly pool=datapool%nsd: device=/dev/vdb servers=g5 nsd=datansd02 usage=dataOnly pool=datapool
Add NSD
mmcrnsd -F nsdfile.cfg
4. Start the cluster
mmstartup -a#View cluster status: mmgetstate -La
5. Create a gpfs file system
# mmcrfs /gpfs gpfs -F nsdfile.cfg -A yes -v nommcrfs gpfs -F nsdfile.cfg -A yes -B 1M -j hcluster -m 2 -r 2 -M 3 -R 3 -T /gpfs --metadata-block-size 256K-v no Parameters: gpfs: Denotes the name of the file system-F: Specifies the NSD configuration file-A: Set this item to yes-B: Specifies the size of the data block, which must be 16k, 64k, 128k, 256k, 512k, 1M, 2M, 4M, 8M, 16M.- j: Specify mapping type-m: Default value for number of metadata copies-M: Maximum value for number of metadata copies-r: Default value for number of data copies-R: Maximum value for number of data copies-T: Mount point--metadata-block-size: Size of metadata node block-v Verify that the disk created already has a file system Yes No mmshutdown -ammchconfig maxblocksize=2048Kmmstartup -ammcrfs gpfs -F nsdfile.cfg -A yes -j hcluster -m 2 -r 2 -M 3 -R 3 -T /gpfs -v no
6. Mount the file system
mmmount gpfs /gpfs -N all
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.