Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how to parse the core source code of SparkContext. Many people may not know much about it. In order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

After obtaining a series of initialization information, SparkContext starts to create and start the TaskScheduler instance:

Enter the createTaskScheduler method:

First instantiate a TaskSchedulerImpl in the above code:

Then create the key backend:

Go back to the code of SparkContext's Standalone mode:

The mode that handles scheduling modes such as FIFO and Fair is shown in the above code.

The instantiated backend, scheduler, is returned at the end of the code block:

After reading the above, do you have any further understanding of how to parse the core source code of SparkContext? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.