Shulou(Shulou.com)06/02 Report--

This article mainly explains "C++ template > > example Analysis of compilation problems, which interested friends may wish to have a look at. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "C++ template > > compilation problem example analysis" bar!

In compilation theory, the compilation process is usually abstracted into five main stages: lexical analysis (Lexical Analysis), syntax analysis (Parsing), semantic analysis (Semantic Analysis), optimization (Optimization), code generation (Code Generation). These five phases are similar to the Unix pipeline model, with the output of the previous phase as the input to the next phase. Among them, lexical analysis is based on the input source code text stream, divide words, identify categories, and generate lexical elements (Token) streams, such as:

Int a = 10

After lexical analysis, you will get [(Type, "int"), (Identifier, "a"), (AssignOperator, "="), (IntLiteral, 10)]. In the subsequent stage of syntax analysis, the corresponding grammar rules will be matched according to these lexical elements. When I learned the principle of compilation, the introduction of lexical analysis in textbooks is mainly based on regular expressions, which means that the rules of words and regulations in common languages can be described by regular expressions. For example, the C language variable name rule is "contain letters, numbers, or underscores, and start with letters or underscores", which can be expressed in the regular expression [a-zA-Z] [a-zA-Z0-9] *. However, in practice, I have found that both the mainstream language and the self-designed DSL have a large number of examples that can not be analyzed simply through regular expressions. Let's take a look at an example of a template for Clover 98:

Map

The above code will be reported a syntax error in the compiler of Cnotify 98 because it recognizes "> >" as a bit right shift operator rather than two template right parentheses, which must be written with a space between the two parentheses.

Map

In addition to the C++ template, as far as I know, there are a lot of lexical ambiguities in the grammatical rules of the classical FORTRAN language.

I think in essence, the root of this kind of problem is that lexical analysis is based on simple lexical rules and does not have all grammatical information, and lexical ambiguity must be eliminated in grammatical rules. Therefore, when I design some DSL, I simply combine lexical analysis with grammatical analysis, which is equivalent to allowing syntax analysis to be carried out at the character level, rather than at the classical lexical element level, this is the so-called Scannerless Parsing. Examples of this approach are not uncommon, and parsers in languages such as TeX, Wiki, Makefile and Perl 6 all fall into this category.

Scannerless Parsing method makes up for the problem that lexical rules can not be disambiguated, but it also destroys the simple and clear pipeline structure of lexical and grammatical analysis, and generally increases the complexity of implementation and understanding. In addition, for a large language like C++, if there is lexical analysis at the beginning, it would be too exaggerated to turn the whole language into Scannerless Parsing when it comes to a slight ambiguity. This problem has perplexed me for a long time, and it was not until recently that I found a satisfactory solution. Or take the above "> >" as an example, we know that Category 11 now allows no spaces, so how does the compiler deal with this lexical ambiguity? The answer is: since the lexical analysis stage is not good at "> >", let's not analyze it at all, and give ">" directly to the parser for analysis, while others without lexical ambiguity remain the same. When I knew about this plan, I couldn't help sighing: wonderful! In theory, lexical analysis can do nothing, and there is no problem in handing all the characters to the parser, so let the lexical analysis only do the sure part, and leave the unsolved ones to the parser. In this way, it not only retains the pipeline structure, but also solves lexical ambiguity.

Let's take a look at the definition of this issue in the Category 11 specification:

14.2 Names of template specializations [temp.names] #

After name lookup (3.4) finds that a name is a template-name or that an operator-function-id or a literal-operator-id refers to a set of overloaded functions any member of which is a function template if this is followed by an is taken as the ending delimiter rather than a greater-than operator. Similarly, the first non-nested > > is treated as two consecutive but distinct > tokens, the first of which is taken as the end of the template-argument-list and completes the template-id. [Note: The second > token produced by this replacement rule may terminate an enclosing template-id construct or it may be part of a different construct (e.g. A cast).-end note]

It can be seen that in Category 11, the lexical analyzer passes the "> >" directly to the parser as two ">", and then if the template- argument-lis syntax is matched in the parsing, the * * ">" symbol will be directly regarded as the template Terminator, not the greater than, nor the displacement symbol. Based on this definition, I constructed an example:

Template class Foo {}; Foo > 1 > foo

This example compiles correctly in Category 98, and "> >" is interpreted as a displacement operation, but it cannot be compiled in Category 11 because it is interpreted as a template parameter Terminator according to the specification. If you want to compile in Category 11, you need to explicitly add parentheses:

Foo > 1) > foo; so far, I believe you have a deeper understanding of "C++ template > > compilation problem example Analysis". You might as well come to practice it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.