Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to solve the situation of be downtime caused by empty Drois Stream load JSON. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

Two days ago, the cluster suddenly experienced a large-scale outage of BE under normal operation, and began to think that it was caused by a query. After stopping the query API, it was found that it still appeared. The following exception information was found through the be.out log:

When loading json data like `[]` (an empty array). BE will crash with stack: ```* Aborted at 1612273824 (unix time) try "date-d @ 1612273824" if you are using GNU date * PC: @ 0xe0cce7 rapidjson::GenericValue::Accept () * SIGSEGV (@ 0xe) received by PID 36798 (TID 0x7f7812114700) from PID 14 Stack trace: * * @ 0x7f791b74b470 (unknown) @ 0xe0cce7 rapidjson::GenericValue::Accept () @ 0x169ff79 _ ZN5doris10JsonReader17_print_json_valueB5cxx11ERKN9rapidjson12GenericValueINS1_4UTF8IcEENS1_19MemoryPoolAllocatorINS1_12CrtAllocatorEEEEE @ 0x16a0689 doris::JsonReader::_write_values_by_jsonpath () @ 0x16a2cb4 doris::JsonReader::_handle_flat_array_complex_json () @ 0x16a3761 doris: : JsonScanner::get_next () @ 0x1659bd4 doris::BrokerScanNode::scanner_scan () @ 0x165a671 doris::BrokerScanNode::scanner_worker () @ 0x281f67f execute_native_thread_routine @ 0x7f791b5001c3 start_thread @ 0x7f791b7fd12d _ _ clone

Finally, it is found that the Stream Load JSON data string is empty. If it is through FE, it will not cause this problem, but performing Stream Load operation directly through BE will throw the above exception, resulting in be downtime.

Finally, under the guidance of the community god Chen Mingyu, someone in the community has submitted this Patch, and after putting on this patch, the problem has been solved.

Fix bug that BE core will loading empty json array

Because this Patch is not in the release yet, in order to facilitate your use, I compiled a version and put it on the Baidu network disk, which you can download according to your own needs.

You use Stream load to import data (communicate with BE directly). If an empty json string leads to BE down, please put on this patch as soon as possible.

The above is how to solve the problem of be outage caused by empty Drois Stream load JSON. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.