Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to understand the use of MaxCompute Studio. Many people may not know much about it. In order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

MaxCompute Studio is a big data integrated development environment tool installed on the developer client provided by the MaxCompute platform. It is a development plug-in based on the popular integrated development platform IntelliJ IDEA, which can help you to develop data conveniently. Let's take a look at the simple use of Studio.

User login

Studio provides user login function. The general login entry is in the Sign In in the upper right corner of Intellij toolbar. After logging in, you can directly use the answering robot, add D2 items, synchronize D2 and other functions.

Question answering robot

Click the "FAQ Robot" button on the right to enter the answering robot page, as shown in the following figure. The answering robot supports both self-help answering and manual answering, and supports uploading pictures and files. If you are not satisfied with the results of the self-answering, you can send a "manual" call to the ODPS on duty in the chat box. Or send pictures or files directly to evoke human services.

Resource management

One of the core functions of Studio is to browse the resources of MaxCompute project space (Project), including Table, UDF, Resource and so on. In order to be able to access them in Studio, as well as many other functions, we first need to create a new project connection. Click "+" in Project Explorer to add connections. Currently, Studio supports synchronizing D2 projects and adding project connections through accessId/Key.

After establishing the project space connection, you can browse the project resources such as Table&View, Function, Resource and so on. Double-click the corresponding element to view the resource details in Intellij.

Table

Double-click to open a table, as shown in the following figure. There are two tab pages, and the Table Information page is shown below, which shows the basic information of table, schema information, and provides data preview functions.

Select the corresponding partition, set the number of preview rows, and click the "Preview" button to preview the data. At the same time, right-click the header to support the data import / export function.

The Partition Information page is shown below, which displays the details of the partition and provides query and refresh functions for the partition. At the same time, it provides functions such as exporting partition information, importing and exporting partition data and so on.

When you right-click a table in project explorer, you can see a series of practical functions such as table building, table modification, data operation and so on.

Right-click a table and select "open table editor" to modify the table information accordingly. As shown in the following figure, Studio can convert user actions into corresponding SQL statements.

Functions

Functions is divided into BuiltIn Function and UserDefied Function. Double-click the corresponding function to expand the details:

Resources

Double-click the corresponding resource to open the file in editor. For cases where the number of UDF files is greater than 5, only the first five class are opened.

You can add resource in the following ways:

SQL Editor and Job submission

How to help users write SQL efficiently and happily is the core mission of MaxCompute studio. Let's take a look at the use of the SQL editor. First, you need to create a MaxCompute Script Module, as shown in the following figure:

After the Module is created, the new SQL script file is shown in the following figure.

Finally, double-click the SQL file to develop our script, where you can set the mode (single sentence mode | script mode), system type (legacy system type | MaxCompute system | Hive system), compiler type (default compiler | experimental compiler), and so on. You need to select the corresponding project in the upper right corner.

The three buttons shown in the following figure represent synchronizing SQL to D2, opening ODPS CMD in Cosonle, and opening SQL History.

After editing the SQL, click the graph tab button below to display the execution plan of the SQL. Double-clicking the Task node will expand the corresponding Operator level information.

Click the green submit button, and Studio will compile the script first, then submit the Server, then display the running information and open the Job analysis page, as shown below:

Job details

Above we talked about how to edit the SQL and submit the job, and then take a look at how to view the job details. The job details page can be opened through multiple paths. For example, submitting a job mentioned above will automatically open the job details page. Secondly, you can also open the job details page through Job Explorer, and if you already know logview, you can open the job details page through logview, as shown below:

The job details page is divided into two parts, the left side is some basic information of the job, and the right side contains several tab pages, including graph information, sequence diagram, details, script, summary, results, analysis and so on. The graph diagram, sequence diagram, details page, and job analysis are further introduced below.

Graph page

The graph page shows the Job | Task | Operator three-tier structure of the job. Click the navigation in the upper left corner to switch the views of different level, double-click the Job node to enter the Task view, and double-click the Task node to enter the Operator view.

Graph page users can zoom the image through the mouse wheel or the left zoom button, while providing functions such as eagle eye and drag and drop. Click the task node to display the task level information, and click on the connection between the task to display the schema information.

Right-click the node in the task level view and select "expand all" to expand all operator.

At the same time, clicking the table node will display the basic information and partition information of the table, and double-clicking the table node will jump to the corresponding table details page.

Time sequence diagram

The timing diagram shows the Gantt chart of all the Fuxi Instance, and the running time of the job can be analyzed in detail through the sequence chart. The Filter on the left can filter the instances. Hovering the mouse over the corresponding instance will display the corresponding information. Double-click will jump to the "details" tab page and select the corresponding instance, which is very convenient for user analysis. At the same time, the timing diagram supports zooming function.

Details page

The details page shows Task-level and Instance-level details.

Job analysis page

The job analysis page provides the conclusions of job analysis, such as long-tailed nodes, data skew, and so on. Scatter chart, long tail chart and data tilt chart are provided for user analysis.

Job queue

Maxcompute jobs need to go through a lot of states from submission to execution, such as compiling, optimizing, queuing for resources, and so on. The job queue provides the ability to view the current detailed status. For jobs that are queued for scheduling, you can also view information such as queued locations and preordered jobs from the queue window. Click "Job Explorer" to open the job queue page, which provides the project name, status, date and other Filter. After selecting the corresponding conditions, you will get the corresponding job queue. Double-clicking a job will open the corresponding job details page.

After reading the above, do you have any further understanding of how to understand the use of MaxCompute Studio? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.