In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-03-17 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/03 Report--
Tool preparation: jdk1.8, scala-2.11.11, gradle-3.1, zookeeper-3.4.5, kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz
install the JDK
Install Scala
Building Zookeeper
kafka source code build
Extract kafka-0.10.0.1-src.tgz, run kafka-0.10.0.1-src from the command line, and execute gradle idea
Note:
1)gradle idea, with foreign library build download speed is very slow, in the project inside, find the build.gradle file modification library for Ali mirror library
buildscript {
repositories {
maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}
}
apply from: file('gradle/buildscript.gradle'), to: buildscript
dependencies {
// For Apache Rat plugin to ignore non-Git files
classpath "org.ajoberstar:grgit:1.5.0"
classpath 'com.github.ben-manes:gradle-versions-plugin:0.12.0'
classpath 'org.scoverage:gradle-scoverage:2.0.1'
}
}
allprojects {
apply plugin: 'idea'
apply plugin: 'eclipse'
apply plugin: "jacoco"
allprojects {
repositories {
maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}
}
}
2)Error when executing gradle idea
* What went wrong:
A problem occurred evaluating root project 'kafka-0.10.0.1-src'.
> Failed to apply plugin [class 'org.gradle.api.plugins.scala.ScalaBasePlugin']
> No such property: useAnt for class: org.gradle.api.tasks.scala.ScalaCompileOptions
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Solution: Open the build.gradle file in kafka-0.10.0.1-src directory
Add the following line
[plain] view plain copy
ScalaCompileOptions.metaClass.daemonServer = true
ScalaCompileOptions.metaClass.fork = true
ScalaCompileOptions.metaClass.useAnt = false
ScalaCompileOptions.metaClass.useCompileDaemon = false
7. Open kafka with intellij idea-0.10.0.1-src
8. Copy the log4j.properties file to src/main/scala path of the core module, so that the log can be displayed normally.
9. Add scala-sdk
10. Configure startup parameters
11. startup project
12. verification
Unzip the file kafka_2.11-0.10.0.1.tgz and enter bin/windows on the command line
1) Creating a topic
C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --
partitions 1 --topic testMsg
Created topic "testMsg".
The console logs are as follows:
2) Execute producer commands to generate messages:
3) Executing consumer commands, consuming messages:
Question:
ERROR Processor got uncaught exception. (kafka.network.Processor)
java.nio.BufferUnderflowException
Solution:
This is caused by the inconsistency between the version numbers of the client jar package and the cluster in the minor version, so keep the version numbers consistent.
scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.