Shulou(Shulou.com)06/03 Report--

Add: you already have a kafka dependency package in flume's lib, so you can just use flume's configuration file.

Here is a typical scenario of kafka+flume: (Kafka-sink)

Source: netcatchannel:memorysink:kafka sink

Netcat-kafka.conf

# alias a1.sources=s1a1.channels=c1a1.sinks=k1#sourcea1.sources.s1.type=netcata1.sources.s1.bind=192.168.5.0a1.sources.s1.port=6666#channela1.channels.c1.type=memory#sinka1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSinka1.sinks.k1.kafka.topic = kafka_test # incoming theme a1.sinks.k1.kafka.bootstrap.servers = hadoop02:9092 # kafka cluster entry a1.sinks.k1.kafka.producer.acks = 1 # message acknowledgement level # specifies the type of compression a1.sinks.k1.kafka.producer.compression.type = snappya1.sources.s1.channels=c1a1.sinks.k1.channel=c1

Finally, start the configuration file of flume:

Bin/flume-ng agent-n A1-c conf-f conf/myconf/netcat-kafka.conf-Dflume.root.logger=INFO,console

Official website address: http://flume.apache.org/FlumeUserGuide.html#kafka-sink

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.