Shulou(Shulou.com)06/02 Report--

What this article shares with you is about how to use C++ lexical analysis and design methods. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article. Let's take a look at it with the editor.

The following article from the perspective of C++ lexical analysis, in fact, a language is a collection of sentences, and a sentence is a nonlinear structure composed of marks returned by the lexical analyzer. The task of C++ grammatical analysis is on the basis of lexical analysis. According to the grammatical rules of the language, lexical symbols are divided into various grammatical units.

The method to reflect the sentence structure is tree, and the commonly used methods are analysis tree and syntax tree. There are two basic methods for analyzing grammatical structures: top-down analysis and bottom-up analysis. Top-down analysis builds an analysis tree from root to leaf, while bottom-up analysis is just the opposite. In both cases, the parser scans the input from left to right, reading one token at a time. Similar to C++ lexical analysis, grammatical analysis has dual meanings:

① specifies the rules for the formation of sentences, also known as grammatical rules. Most of the grammatical rules of programming language can be described by ContextFreeGrammar (CFG for short).

② recognizes the comment structure in the token stream according to syntax rules, also known as parsing. The most effective top-down and bottom-up analysis methods can only deal with subclasses of context-free grammars, such as LL grammars and LR methods, but they are sufficient to cope with most grammatical phenomena in programming review.

I. Task and purpose

The main contents are as follows: 1. The C++ lexical analyzer of this function drawing language is written by using CumberCraft + programming language and recursive descending subroutine. And require the design of a parser test Mini Program to call their own parser to test a variety of different inputs.

2. The task of grammar analysis is to decompose lexical symbols into various grammatical units on the basis of C++ lexical analysis and according to the grammatical rules of the language. Grammatical analysis is based on the grammatical rules of the language, which are usually described by context-free grammars.

Through self-written parser, master normal formula and regular grammar, context-free grammar (CFG), basic concepts with derivation (derivation, parsing tree and syntax tree, elimination of ambiguity and ambiguity), top-down analysis (recursive descending subroutine method, predictive analysis table method, LL (1) grammar), bottom-up analysis. Understand how to combine theory with practice and understand the difference between theory and practice.

II. Analysis and design

The syntax analysis program generally has the following functions: syntax analysis of word symbol strings (derivation and specification according to semantic rules), identification of all kinds of grammatical units in the program, and finally determine whether the input string constitutes a grammatically correct "program".

Here we use the recursive descent analysis method: directly simulate the process of producing language in a programmatic way. Its basic design idea is that a subroutine is constructed for each non-Terminator, the process body of each subroutine is expanded according to the candidate of the production, and the Terminator is directly matched. When a non-Terminator is encountered, the corresponding non-Terminator subroutine is called.

The analysis starts with calling the subroutine of the grammar start symbol until all non-terminators are expanded to finalizers and a match is obtained. If you reach this step in the analysis process, it indicates that the analysis is successful, otherwise it indicates that there are grammatical errors in the input. The limitation of recursive descent analysis on grammar is that there can be no common left factor and left recursion. Because grammars are recursively defined, subroutines are also recursive.

For relatively small languages, the recursive descending subroutine method is very effective. It is simple, flexible and easy to construct. Its disadvantage is that the program is directly related to the grammar, and any changes to the grammar need to modify the program accordingly.

Here is a general design method of C++ lexical analysis program:

1. Write the context-free grammar G for parsing according to the requirements.

2. Eliminate the ambiguity of context-free grammar G

3. Eliminate (direct) left recursion of context-free grammar G and extract left factor.

4. Construct the state transition diagram of the grammar and simplify it

5. Convert the conversion diagram to EBNF representation

6. Construct a recursive descending subroutine from EBNF

The following is a more detailed design:

Overall structure and module division

Grammar Test Module (parsermain.cpp)

Parser module (parser.h & parser.cpp) drawing language interpreter entry recursive subassembly preorder traversal and printing expression syntax tree error handling module lexical analyzer module (scanner.h & scanner.cpp) the lexical analyzer identifies the smallest grammatical unit with independent meaning the auxiliary module is how to use C++ lexical analysis and design method The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.