In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-04-08 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/02 Report--
Kafkaflume.conf
Agent.sources = s1agent.channels = c1agent.sinks = k1agent.sources.s1.type=execagent.sources.s1.command=tail-F / usr/local/src/flume/testflume2.logagent.sources.s1.channels=c1agent.channels.c1.type=memoryagent.channels.c1.capacity=10000agent.channels.c1.transactionCapacity=100# set Kafka receiver # agent.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink## set broker address and port number of Kafka agent.sinks.k1.kafka.bootstrap.servers = master:9092,node01:9092 Node02:9092## sets Topicagent.sinks.k1.kafka.topic of Kafka = mydemo5## sets serialization mode # # agent.sinks.k1.kafka.flumeBatchSize = 20agent.sinks.k1.kafka.producer.acks = 1agent.sinks.k1.kafka.producer.linger.ms = 1agent.sinks.ki.kafka.producer.compression.type = snappyagent.sinks.k1.channel=c1
Create a topic
Kafka-topics-create-zookeeper master:2181/kafka-replication-factor 2-partitions 3-topic mydemo5
Start flume as a message producer to write on kafka
Flume-ng agent-c conf-f kafkaflume.conf-n agent-Dflume.root.logger=INFO,console
Consume kafka data
Kafka-console-consumer-bootstrap-server master:9092-topic mydemo5-from-beginning
Read the contents of a file and write to append to a new file
#! / bin/bashwhile read linedo echo $line sleep 0.01echo-e $line > > / usr/local/src/flume/testflume2.logdone < / etc/sudo-ldap.conf
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.