Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "Nebula Graph source code analysis". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Guide reading

For some of you who are new to the Nebula Graph open source library, you may want to improve yourself at first, just like me, look at the code of the gods and try to do something, maybe you can fix a Bug that doesn't seem so difficult. But faced with so much code, I cracked and didn't know how to do it. Finally, I took a hard look at the code again and again, and after running one use case after another, I finally got something.

The following is to share the process of personal learning Nebula Graph open source code, but also hope that friends who have just come into contact with Nebula Graph can take fewer detours and get started quickly. In addition, Nebula Graph itself uses some open source libraries, details of which can be found in the appendix.

In this article, we will quickly learn Nebula Graph through data flow. Take the user entering a nGQL statement SHOW SPACES on the client as an example, and use the GDB tracking statement to track how Nebula Graph is called and run when input.

Overall architecture

A complete Nebula Graph contains three services, namely, Query Service,Storage Service and Meta Service. Each service has its own executable binaries.

Query Service is mainly responsible for

Management of client connection

Parse the nGQL statement from the client into the abstract syntax tree AST, and parse the abstract tree AST into a series of execution actions.

Optimize the execution action

Execute the optimized execution plan

Storage Service is mainly responsible for

Distributed storage of data

Meta Service is mainly responsible for

Add, delete, check and modify the figure schema

Cluster management

User authentication

This time, we mainly analyze Query Service.

Directory structure

At the beginning, you can get a source package and decompress it. You can first look at the hierarchical relationship of the code. What are the main functions of different packages? the src directory is listed below:

| |-- src |-- client / / client code |-- common / / provide some commonly used basic components |-- console |-- daemons |-- dataman |-- graph / / contains most of the Query Service code |-- interface / / mainly some meta, Definition of communication interface between storage and graph |-- jni |-- kvstore |-- meta / / related to metadata management |-- parser / / is mainly responsible for lexical and syntax analysis |-- storage / / related to storage layer |-- tools |-- webservice code tracking

Start the metad and storaged services through the scripts in the scripts directory:

Check the current service status via nebula.service status all after startup

Then gdb runs the nebula-graphd binary in the bin directory

Gdb > set args-- flagfile / home/mingquan.ji/1.0/nebula-install/etc/nebula-graphd.conf / / set function input parameter gdb > set follow-fork-mode child / / since it is a daemon, gdb continues to track the child process gdb > b main / / break point at the mian entry after the fork child process

Type run in gdb to run the nebula-graphd program, and then you can run it step by step through next until you encounter gServer- > serve (); / / Blocking wait until shut down via gServer- > stop (), when all threads of nebula-graphd block and wait for the client to connect. In this case, you need to find out which function will handle the request from the client.

Since Nebula Graph uses FBThrift to define and generate communication codes for different services, you can see the definition of GraphService interface in the src/interface/graph.thrift file as follows:

Service GraphService {AuthResponse authenticate (1: string username, 2: string password) oneway void signout (1: i64 sessionId) ExecutionResponse execute (1: i64 sessionId, 2: string stmt)}

Before gServer- > serve ()

Auto interface = std::make_shared (); status = interface- > init (ioThreadPool); gServer- > setInterface (std::move (interface)); gServer- > setAddress (localIP, FLAGS_port)

You know that it is the GraphService object that handles client connections and requests, so you can break the point at GraphService.cpp: ``future _ execute to track subsequent processing.

At this point, re-open a terminal to enter the nebula installation directory, connect to the nebula service through. / nebule-u=root-p=nebula, and then enter SHOW SPACES on the client. The client does not respond because the server is still blocking and debugging. Enter continue at the server, as shown below:

After session verification, enter executionEngine- > execute (), and step enters the interior of the function.

Auto plan = new ExecutionPlan (std::move (ectx)); plan- > execute ()

Continue step to enter inside the execute function of ExecutionPlan, and then execute to

Auto result = GQLParser () .parse (rctx- > query ())

Parse mainly uses flex & bison to construct objects to abstract syntax trees for lexical analysis and syntax parsing. Its lexical file is src/parser/scanner.lex, and its grammar file is src/parser/parser.yy. Its lexical analysis is similar to regular expressions. Examples of syntax analysis are as follows:

Go_sentence: KW_GO step_clause from_clause over_clause where_clause yield_clause {auto go = new GoSentence (); go- > setStepClause ($2); go- > setFromClause ($3); go- > setOverClause ($4); go- > setWhereClause ($5); if ($6 = = nullptr) {auto * cols = new YieldColumns () For (auto e: $4-> edges ()) {if (e-> isOverAll ()) {continue;} auto * edge = new std::string (* e-> edge ()); auto * expr = new EdgeDstId_Expression (edge); auto * col = new YieldColumn (expr) Cols- > addColumn (col);} $6 = new YieldClause (cols);} go- > setYieldClause ($6); $$= go;}

When it matches to the corresponding go statement, the corresponding node is constructed, then processed by bison, and finally an abstract syntax tree is generated.

After lexical and grammatical analysis, we start the execution module, continue gdb, enter the excute function, and keep step until we enter the ShowExecutor::execute function.

Continue next until showSpaces (), step enters this function

Auto future = ectx ()-> getMetaClient ()-> listSpaces (); auto * runner = ectx ()-> rctx ()-> runner (); 'std::move (future) .via (runner) .thenValue (cb) .thenError (error)

At this point, Query Service gets the spaces data through the communication between metaClient and Meta Service, and then sends back the obtained data through the callback function cb. At this point, the nGQL statement SHOW SPACES; has been executed, and other complex statements can also be used.

If it is a running service, you can first find out the process ID of the service, and then debug the process through gdb attach PID

If you do not want to start the server and the client to debug, there is a test directory in each folder under the src directory, which is a unit test for the corresponding module or function. You can compile the corresponding unit module directly, and then track the run. The methods are as follows:

Find the corresponding module name through the CMakeLists.txt file in the corresponding directory

Name the make module in the build directory and generate the corresponding binary program in the build/bin/test directory

Gdb trace debug the program

"Nebula Graph source code analysis" content is introduced here, thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.