Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to use failover and load balance of Flume". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn how to use Flume's failover and load balance.

1. Load Balancing Sink Processor

= > agent_2

Source = > agent_1 = > agent_3 = = > Sink

= > agent_4

Agent_2,agent_3,agent_4 shares the data pressure of agent_1

[root@hftest0001 conf] # pwd/opt/apache-flume-1.6.0-bin/conf [root@hftest0001 conf] # vi s-exec_c-m_s-avro_sp-lb.confagent.sinkgrous.lb-1.sinks = 155-avroSink 154-avroSinkagent.sinkgrous.lb-1.processor.type = load_balance#agent.sinkgrous.lb-1.processor.backoff = trueagent.sinkgrous.lb-1.processor.selector = round_robin#agent.sinkgrous.lb-1.processor.selector.maxTimeOut = 3000agent.sinks.155-avroSink.type = avroagent.sinks.155-avroSink.hostname = ${ip-155} agent.sinks.155-avroSink.port = 55555agent.sinks.155-avroSink.channel = memoryChannelagent.sinks.154-avroSink.type = avroagent.sinks.154-avroSink.hostname = ${ip-154} agent.sinks.154-avroSink.port = 55555agent.sinks.154-avroSink.channel = memoryChannelagent.channels.memoryChannel.type = memoryagent.channels.memoryChannel.capacity = 100

II. Failover Sink Processor

[root@hftest0001 conf] # vi s-exec_c-m_s-avro_sp-fo.conf agent.sinkgrous.lb-1.sinks = 155-avroSink 154-avroSinkagent.sinkgrous.lb-1.processor.type = failoveragent.sinkgrous.lb-1.processor.priority.155-avroSink = 5agent.sinkgrous.lb-1.processor.priority.154-avroSink = 10 = > set the weight agent.sinkgrous.lb-1.processor.maxpenalty = 10000agent.sinks.155-avroSink.type = avroagent.sinks.155 -avroSink.hostname = ${ip-155} agent.sinks.155-avroSink.port = 55555agent.sinks.155-avroSink.channel = memoryChannelagent.sinks.154-avroSink.type = avroagent.sinks.154-avroSink.hostname = ${ip-154} agent.sinks.154-avroSink.port = 55555agent.sinks.154-avroSink.channel = memoryChannelagent.channels.memoryChannel.type = memoryagent.channels.memoryChannel.capacity = 100so far I believe you have a deeper understanding of "how to use Flume's failover and load balance", so you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.