In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-03-17 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/03 Report--
Big data is defined as a large amount of unstructured data. The quantity should be large and unstructured.
Hadoop is composed of three parts, 1.hdfs hadoop distributed file system. 2.MapReduce distributed computing. 3.hive distributed storage.
Operating system: centos6.5 64
Environment building: 1. Install Hadoop,2. Install mysql, 3. Install hive,4, install jdk
Hadoop installation:
Download Hadoop1.0.4,hadoop-1.0.4.tar.gz.
Wget http://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4.tar.gz
Decompress:
Tar xzvf hadoop-1.0.4.tar.gz-C / usr
Modify 3 configuration files
Cd / usr/hadoop-1.0.4/conf/
Vi core-site.xml
Fs.default.name
Hdfs://localhost:9000
Vi mapred-site.xml
Mapred.job.tracker
Localhost:9001
Vi hdfs-site.xml
Dfs.name.dir
/ home/longlong/temp/log1,/home/longlong/temp/log2
Dfs.data.dir
/ home/longlong/temp/data1,/home/longlong/temp/data2
Dfs.replication
two
Configure the environment variable, where the environment variable of java,hive,hadoop is pasted
Vi / etc/profile
JAVA_HOME=/home/Hadoop/jdk1.6.0_45
CLASSPATH=$JAVA_HOME/jre/lib/rt.jar
HADOOP_HOME=/usr/hadoop-1.0.4
HIVE_HOME=/usr/hive
PATH=$HADOOP_HOME/bin:$PATH:$JAVA_HOME/bin:$HIVE_HOME/bin
Install jdk:
Download jdk
Wget http://download.oracle.com/otn-pub/java/jdk/6u45-b06/jdk-6u45-linux-x64.bin
Chmod + x jdk-6u45-linux-x64.bin
. / jdk-6u45-linux-x64.bin
Configure vi hadoop-env.sh and add jdkhome at the end
Export JAVA_HOME=/home/Hadoop/jdk1.6.0_45
Enter the bin directory to configure vi hadoop-config.sh
Export HADOOP_HOME=$ {HADOOP_PREFIX}
Export HADOOP_HOME_WARN_SUPPRESS=1
Formatting
. / hadoop namenode-format
2.mysql installation
3.hive installation
download
Wget http://mirrors.cnnic.cn/apache/hive/hive-0.13.1/apache-hive-0.13.1-bin.tar.gz
Tar-xf apache-hive-0.13.1-bin.tar.gz
Mv apache-hive-0.13.1-bin hive
Replace Metabase
Cd conf/
Touch hive-site.xml
Vi hive-site.xml
Javax.jdo.option.ConnectionURL
Jdbc:mysql://localhost:3306/hive
Javax.jdo.option.ConnectionDriverName
Com.mysql.jdbc.Driver
Javax.jdo.option.ConnectionUserName
Root
Javax.jdo.option.ConnectionPassword
Root
Download the mysql driver
Wget http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.0.8.tar.gz
Tar-xzvf mysql-connector-java-5.0.8.tar.gz
Cd mysql-connector-java-5.0.8
Cp mysql-connector-java-5.0.8-bin.jar / usr/hive/lib/
Start
1.hdfs start
Cd / usr/hadoop-1.0.4/bin
. / start-all.sh
Password/ operating system password
2.mysql start
Service mysqld start
Mysql-uroot-proot
3.hive start
Cd / usr/hive/bin
. / hive
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.