Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how to solve the problem caused by the return of operator function NULL. Many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.

In the operator function, return null

Return actionRDD.mapToPair (new PairFunction () {private static final long serialVersionUID = 1L; @ Override public Tuple2 call (Row row) throws Exception {return new Tuple2 ("- 999", RowFactory.createRow ("- 999"));}})

As you can see, in some operator functions, we need to have a return value. However, sometimes, we may be interested in certain values

I just don't want to have any return value. If we return to NULL directly, we will report an error!

Scala.Math (NULL), exception

If you do not want a return value for certain values, there is a solution:

When returning, return some special values. Do not return null, such as "- 999".

After obtaining a RDD through the operator, you can perform filter operations on the RDD to filter the data. Filter, you can determine the data, if it is-999, then return false, to filter out it.

Don't forget that the operator we talked about before optimizes the coalesce operator inside. After filter, you can use the coalesce operator to compress the number of partition of RDD to make the data of each partition more compact. It can also improve some performance.

After reading the above, do you have any further understanding of how to solve the problem caused by the operator function returning NULL? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.