Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to customize the EventSerializer serialization class in hdfssink in flume. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Because I had done the serialization class of hbasesink before, I thought it would be easy to write hdfs, but I didn't expect it to be different. Hdfs does not directly configure the serialization class option to select the corresponding serialization class according to fileType, the type of datastream we use, the corresponding class is HDFSDataStream, the default serialization class TEXT of this class (this is an enumeration type)

SerializerType = context.getString ("serializer", "TEXT")

The classes of the enumeration are as follows:

Public enum EventSerializerType {TEXT (BodyTextEventSerializer.Builder.class), HEADER_AND_TEXT (HeaderAndBodyTextEventSerializer.Builder.class), AVRO_EVENT (FlumeEventAvroEventSerializer.Builder.class), CUSTOM (CUSTOMEventSerializer.Builder.class), / / Custom serialization class OTHER (null); private final Class

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.