Shulou(Shulou.com)06/03 Report--

Tool preparation: jdk1.8, scala-2.11.11, gradle-3.1, zookeeper-3.4.5, kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

install the JDK

Install Scala

Building Zookeeper

kafka source code build

Extract kafka-0.10.0.1-src.tgz, run kafka-0.10.0.1-src from the command line, and execute gradle idea

Note:

1)gradle idea, with foreign library build download speed is very slow, in the project inside, find the build.gradle file modification library for Ali mirror library

buildscript {

repositories {

maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}

}

apply from: file('gradle/buildscript.gradle'), to: buildscript

dependencies {

// For Apache Rat plugin to ignore non-Git files

classpath "org.ajoberstar:grgit:1.5.0"

classpath 'com.github.ben-manes:gradle-versions-plugin:0.12.0'

classpath 'org.scoverage:gradle-scoverage:2.0.1'

}

}

allprojects {

apply plugin: 'idea'

apply plugin: 'eclipse'

apply plugin: "jacoco"

allprojects {

repositories {

maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}

}

}

2)Error when executing gradle idea

* What went wrong:

A problem occurred evaluating root project 'kafka-0.10.0.1-src'.

> Failed to apply plugin [class 'org.gradle.api.plugins.scala.ScalaBasePlugin']

> No such property: useAnt for class: org.gradle.api.tasks.scala.ScalaCompileOptions

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Solution: Open the build.gradle file in kafka-0.10.0.1-src directory

Add the following line

[plain] view plain copy

ScalaCompileOptions.metaClass.daemonServer = true

ScalaCompileOptions.metaClass.fork = true

ScalaCompileOptions.metaClass.useAnt = false

ScalaCompileOptions.metaClass.useCompileDaemon = false

7. Open kafka with intellij idea-0.10.0.1-src

8. Copy the log4j.properties file to src/main/scala path of the core module, so that the log can be displayed normally.

9. Add scala-sdk

10. Configure startup parameters

11. startup project

12. verification

Unzip the file kafka_2.11-0.10.0.1.tgz and enter bin/windows on the command line

1) Creating a topic

C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --

partitions 1 --topic testMsg

Created topic "testMsg".

The console logs are as follows:

2) Execute producer commands to generate messages:

3) Executing consumer commands, consuming messages:

Question:

ERROR Processor got uncaught exception. (kafka.network.Processor)

java.nio.BufferUnderflowException

Solution:

This is caused by the inconsistency between the version numbers of the client jar package and the cluster in the minor version, so keep the version numbers consistent.

scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.