Shulou(Shulou.com)06/03 Report--

First, download

Http://archive.cloudera.com/kafka/parcels/2.2.0/

Wget http://archive.cloudera.com/kafka/parcels/2.2.0/KAFKA-2.2.0-1.2.2.0.p0.68-el6.parcelwget http://archive.cloudera.com/kafka/parcels/2.2.0/KAFKA-2.2.0-1.2.2.0.p0.68-el6.parcel.sha1 II. Verify [hadoop@hadoop003 softwares] $sha1sum KAFKA-2.2.0-1.2.2.0.p0.68-el6.parcel359509e028ae91a2a082adfad5f64596b63ea750 KAFKA-2.2.0-1.2.2.0.p0.68-el6.parcel [hadoop@hadoop003 softwares] $cat KAFKA-2.2.0-1.2.2.0.p0.68-el6.parcel.sha1359509e028ae91a2a082adfad5f64596b63ea750

The check code is the same, indicating that the file is not damaged in the download process and can be used normally.

Third, decompress and set the soft connection [hadoop@hadoop003 softwares] $tar-zxf KAFKA-2.2.0-1.2.2.0.p0.68-el6.parcel-C / app [hadoop@hadoop003 app] $ln-s / home/hadoop/app/KAFKA-2.2.0-1.2.2.0.p0.68/ / home/hadoop/app/kafka IV. Important directory description [hadoop@hadoop003 kafka] $pwd/home/hadoop/app/kafka [hadoop@hadoop003 kafka] $lltotal 20drwxr-xr-x 2 hadoop hadoop 4096 Jul 7 2017 bindrwxr-xr-x 5 hadoop hadoop 4096 Jul 7 2017 etcdrwxr-xr-x 3 hadoop hadoop 4096 Jul 7 2017 libdrwxr-xr-x 2 hadoop hadoop 4096 Jul 7 2017 meta### kafka configuration file directory We modify the configuration file here to modify [hadoop@hadoop003 kafka] $ll etc/kafka/conf.dist/total 48 hadoop Jul 7 2017 connect-file-sink 1 hadoop hadoop 906 Jul 7 2017 connect-console-sink.properties-rw-r--r-- 1 hadoop hadoop 909 Jul 7 2017 connect-console-source.properties-rw-r--r-- 1 hadoop hadoop 7 2017 connect-distributed.properties-rw-r--r-- 1 hadoop hadoop 883 Jul 7 2017 connect-file-sink .properties-rw-r--r-- 1 hadoop hadoop 881 Jul 7 2017 connect-file-source.properties-rw-r--r-- 1 hadoop hadoop 1074 Jul 7 2017 connect-log4j.properties-rw-r--r-- 1 hadoop hadoop 2061 Jul 7 2017 connect-standalone.properties-rw-r--r-- 1 hadoop hadoop 4369 Jul 7 2017 log4j.properties-rw-r--r-- 1 hadoop hadoop 5679 Jun 1 01:24 server.properties-rw-r--r-- 1 hadoop hadoop 1032 Jul 7 2017 tools-log4j.properties### kafka feature directory [hadoop@hadoop003 kafka] $ll lib/kafka/total 112drwxr-xr-x 2 hadoop hadoop 4096 Jul 7 2017 bindrwxr-xr-x 2 hadoop hadoop 4096 Jul 7 2017 clouderalrwxrwxrwx 1 hadoop hadoop 43 Jun 1 02:11 config-> / etc/kafka/conf # Note this is red-rw-rw-r-- 1 hadoop hadoop 48428 Jun 1 02:17 KAFKA-2.2.0-1.2.0.p0.68- El6.parceldrwxr-xr-x 2 hadoop hadoop 12288 Jul 7 2017 libs-rwxr-xr-x 1 hadoop hadoop 28824 Jul 7 2017 LICENSEdrwxrwxr-x 2 hadoop hadoop 4096 Jun 1 01:39 logs-rwxr-xr-x 1 hadoop hadoop 336 Jul 7 2017 NOTICEdrwxr-xr-x 2 hadoop hadoop 4096 Jul 7 2017 site-docs### config soft Connect configuration file for getaway that is now linked by default That is, the configuration file of the CM client, because we do not use cm, there is no automatic generation of / etc/kafka/conf error flashing red # bin directory is the kafka related script For example, server startup shutdown & & consumer&&producer startup script 5. Modify the configuration file # first step: [hadoop@hadoop003 kafka] cd etc/kafka/conf.dist# step 2: vim server.properties# step 3: (mainly modify 6 parameters) broker.id=0 # identifier log.dirs = / home/hadoop/app/kafka/logs # location where the data is saved log.retention.hours=168 # data retention time (168hours=7 days) zookeeper.connect=hadoop001:2181 Hadoop002:2181 The location where hadoop003:2181/kafka# zookeeper stores kafka data delete.topic.enable=true # can delete created topics VI. Start Kafka [Hadoop @ hadoop003 kafka] $lib/kafka/bin/kafka-server-start.sh / home/hadoop/app/kafka/etc/kafka/conf.dist/server.properties SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/home/hadoop/app/KAFKA-2.2.0-1.2.2.0.p0.68/lib/kafka/libs/slf4j-log4j12-1.7.21.jarring] SLF4J: Found binding in [jar:file:/home/hadoop/app/KAFKA-2.2.0-1.2.2.0.p0.68/lib/kafka/libs/slf4j-log4j12-1.7.5.jar http://www.slf4j.org/codes.html#multiple_bindings for an explanation.log4j:ERROR Could not read configuration file from URL [file:lib/kafka/bin/../config/log4j.properties] .class] SLF4J: See Logger for an explanation.log4j:ERROR Could not read configuration file from URL [file:lib/kafka/bin/../config/log4j.properties] .class .FileNotFoundException: lib/kafka/bin/../config/log4j.properties (No such file or directory) at java.io.FileInputStream.open0 (Native Method) at java.io.FileInputStream.open (FileInputStream.java:195) at java.io.FileInputStream. (FileInputStream.java:138) at java.io.FileInputStream. (FileInputStream.java:93) at sun.net.www.protocol.file.FileURLConnection.connect (FileURLConnection.java:90) at sun.net.www.protocol .file.FileURLConnection.getInputStream (FileURLConnection.java:188) at org.apache.log4j.PropertyConfigurator.doConfigure (PropertyConfigurator.java:557) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure (OptionConverter.java:526) at org.apache.log4j.LogManager. (LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory. (Log4jLoggerFactory.java:66) at org.slf4j.impl.StaticLoggerBinder. (StaticLoggerBinder.java:72) at org.slf4j.impl.StaticLoggerBinder. (StaticLoggerBinder .java: 45) at org.slf4j.LoggerFactory.bind (LoggerFactory.java:150) at org.slf4j.LoggerFactory.performInitialization (LoggerFactory.java:124) at org.slf4j.LoggerFactory.getILoggerFactory (LoggerFactory.java:412) at org.slf4j.LoggerFactory.getLogger (LoggerFactory.java:357) at org.slf4j.LoggerFactory.getLogger (LoggerFactory.java:383) at org.apache.kafka.common.utils.Utils. (Utils.java:59) at kafka.Kafka$.getPropsFromArgs (Kafka.scala:41) at com.cloudera.kafka.wrap.Kafka$.main (Kafka.scala:72) at com.cloudera.kafka.wrap.Kafka.main (Kafka.scala) log4j:ERROR Ignoring configuration file [file:lib/kafka/bin/../config/log4j.properties]. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] log4j:WARN No appenders could be found for logger (kafka.server.KafkaConfig). Log4j:WARN Please initialize the log4j system properly.log4j:WARN See Http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Bug can't find the configuration file

Java.io.FileNotFoundException: lib/kafka/bin/../config/log4j.properties

Specified as etc/kafka/conf.dist/ because lrwxrwxrwx 1 hadoop hadoop 43 Jun 1 02:11 config-> / etc/kafka/conf cannot be found

[hadoop@hadoop003 kafka] $rm lib/kafka/config [hadoop@hadoop003 kafka] $ln-s / home/hadoop/app/kafka/etc/kafka/conf.dist/ / home/hadoop/app/kafka/lib/kafka/config

Restart

[hadoop@hadoop003 kafka] $nohup kafka-server-start.sh / home/hadoop/app/kafka/etc/kafka/conf.dist/server.properties > / home/hadoop/app/kafka/server-logs/kafka-server.log 2 > & 1 &

The wrong information was not reported.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.