Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Construction of kafka source code reading environment under windows

2025-03-17 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >

Share

Shulou(Shulou.com)06/03 Report--

Tool preparation: jdk1.8, scala-2.11.11, gradle-3.1, zookeeper-3.4.5, kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

install the JDK

Install Scala

Building Zookeeper

kafka source code build

Extract kafka-0.10.0.1-src.tgz, run kafka-0.10.0.1-src from the command line, and execute gradle idea

Note:

1)gradle idea, with foreign library build download speed is very slow, in the project inside, find the build.gradle file modification library for Ali mirror library

buildscript {

repositories {

maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}

}

apply from: file('gradle/buildscript.gradle'), to: buildscript

dependencies {

// For Apache Rat plugin to ignore non-Git files

classpath "org.ajoberstar:grgit:1.5.0"

classpath 'com.github.ben-manes:gradle-versions-plugin:0.12.0'

classpath 'org.scoverage:gradle-scoverage:2.0.1'

}

}

allprojects {

apply plugin: 'idea'

apply plugin: 'eclipse'

apply plugin: "jacoco"

allprojects {

repositories {

maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}

}

}

2)Error when executing gradle idea

* What went wrong:

A problem occurred evaluating root project 'kafka-0.10.0.1-src'.

> Failed to apply plugin [class 'org.gradle.api.plugins.scala.ScalaBasePlugin']

> No such property: useAnt for class: org.gradle.api.tasks.scala.ScalaCompileOptions

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Solution: Open the build.gradle file in kafka-0.10.0.1-src directory

Add the following line

[plain] view plain copy

ScalaCompileOptions.metaClass.daemonServer = true

ScalaCompileOptions.metaClass.fork = true

ScalaCompileOptions.metaClass.useAnt = false

ScalaCompileOptions.metaClass.useCompileDaemon = false

7. Open kafka with intellij idea-0.10.0.1-src

8. Copy the log4j.properties file to src/main/scala path of the core module, so that the log can be displayed normally.

9. Add scala-sdk

10. Configure startup parameters

11. startup project

12. verification

Unzip the file kafka_2.11-0.10.0.1.tgz and enter bin/windows on the command line

1) Creating a topic

C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --

partitions 1 --topic testMsg

Created topic "testMsg".

The console logs are as follows:

2) Execute producer commands to generate messages:

3) Executing consumer commands, consuming messages:

Question:

ERROR Processor got uncaught exception. (kafka.network.Processor)

java.nio.BufferUnderflowException

Solution:

This is caused by the inconsistency between the version numbers of the client jar package and the cluster in the minor version, so keep the version numbers consistent.

scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Internet Technology

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report