Shulou(Shulou.com)05/31 Report--

This article will give you a detailed explanation on how to achieve Storm-kafka. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

The implementation details are as follows:

Public static Iterable generateTuples (KafkaConfig kafkaConfig, Message msg) {Iterable tups; ByteBuffer payload = msg.payload (); ByteBuffer key = msg.key () If (key! = null & & kafkaConfig.scheme instanceof KeyValueSchemeAsMultiScheme) {tups = ((KeyValueSchemeAsMultiScheme) kafkaConfig.scheme) .normalializeKeyAndValue (Utils.toByteArray (key), Utils.toByteArray (payload)) } else {tups = kafkaConfig.scheme.deserialize (Utils.toByteArray (payload));} return tups;}

1: tups object of type Iterable inside the method

2: the Message object is the Message of kafka. Including Key, as well as palyload load

3: when generating tuples, you need to get the Scheme in kafkaConfig.scheme. Once Scheme instanceof

KeyValueSchemeAsMultiScheme, convert mandatory to KeyValueSchemeAsMultiScheme, and call the deserializeKeyAndValue method to deserialize List

In detail, let's take a look at the KeyValueSchemeAsMultiScheme object

Package com.mixbox.storm.kafka;import backtype.storm.spout.SchemeAsMultiScheme;import java.util.Arrays;import java.util.List;@SuppressWarnings ("serial") public class KeyValueSchemeAsMultiScheme extends SchemeAsMultiScheme {public KeyValueSchemeAsMultiScheme (KeyValueScheme scheme) {super (scheme);} / / deserialize a key value public Iterable deserializeKeyAndValue (final byte [] key, final byte [] value) {List o = ((KeyValueScheme) scheme) .normalializeKeyAndValue (key, value) If (o = null) return null; else return Arrays.asList (o);}}

First of all, KeyValueSchemeAsMultiScheme inherits from SchemeAsMultiScheme,SchemeAsMultiScheme is a class implementation that comes with Storm itself, and the above two class implement [MultiScheme] in Storm.

This is the end of the article on "how to achieve Storm-kafka". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.