Shulou(Shulou.com)06/02 Report--

This article focuses on "how to understand IDE support for programming languages". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to understand IDE support for programming languages.

Grammar analysis

For development tools, parsing has several important functions:

Syntax highlighting is an editor feature that displays different colors and fonts according to the category of terms to enhance readability.

Achieve IntelliSense

Implement jump and reference analysis

From my rough survey, it can be roughly divided into four categories:

Parsing based on regular expressions

Regular matching method of Sublime Text based on YAML form: Sublime Syntax files

Regular matching method of Textmate and VS Code based on JSON: Language Grammars

Generate intermediate code based on parsers such as BNF

The way Jetbrins generates code based on BNF: Grammar and Parser

Self-made DSL for grammar parsing

Vim based on regular + self-made DSL:Vim documentation: syntax, Rust example

Handwritten parsing grammar

Eclipse IDE provides a JFace editor, but it seems to be handwritten: FAQ How do I provide syntax coloring in an editor?

Emacs Mode: ModeTutorial

Each category has its own advantages and disadvantages and difficulty in writing. But, on the whole, none of the ways are simple.

Regular implementation of syntax analysis

For the regular approach, both Sublime Text and Textmate and VS Code based on Textmate syntax rules have a significant disadvantage: long, such as VCode's java.tmLanguage.json, in terms of length, the version I see has 1831 lines. The expression is also a bit cumbersome:

"comments": {"patterns": [{"captures": {"0": {"name": "punctuation.definition.comment.java"}, "match": "/\ *\ * /", "name": "comment.block.empty.java"}, {"include": "# comments-inline"}]}

Among them, there are all kinds of include relations. The same is true for Sublime Text:

Comments:-match: /\ *\ * / scope: comment.block.empty.java punctuation.definition.comment.java-include: scope:text.html.javadoc-include: comments-inline

If you look at it, you will suspect that they have established a grammatical alliance.

However, yaml and json are programming language independent things. Therefore, VS Code and Atom can quickly establish lexical analysis for mainstream languages based on Textmate grammar rules, thus establishing syntax highlighting support.

We can also say that BNF is a programming language independent thing. However, in fact, when we operate, we add some programming language-specific elements.

Parser parsing

Thanks to the previous writing of the system analysis tool Coca and the general parser Chapi, I am also quite familiar with the morphology of BNF-actually it is not difficult. The only trouble is that after writing it, we have to write code to do some transformations, so let's take a look at an example of the Jetbrians plug-in:

COMMENT = 'regexp:// [^\ r\ n] *' BLOCK_COMMENT = 'regexp: [/] [*] [^ *] * [*] + ([^ / *] [^ *] * [*] +) * [/]'

This is not much different from antlr:

WS: [\ t\ r\ n\ u000C] +-> channel (HIDDEN); COMMENT:'/ *'. *?'* /'- > channel (HIDDEN); LINE_COMMENT:'/ /'~ [\ r\ n] *-> channel (HIDDEN)

Then, it is to design and analyze the morphology:

FunctionParameters:: = LPAREN inputParameters RPAREN outputParameters? | IN SUB GT inputParameters | outputParameters

Then, in IDEA, we can generate the corresponding Lexer files and code through this BNF file. For lexicology written in Antlr, the code size of the Java section is about 800.

However, from the comparison of the two reading experiences, it is clear that BNF will be a little more friendly.

Self-made DSL grammar parsing

Unfortunately, I haven't written any Vim plug-ins yet, but luckily I know how Vim is returned. I use Vim as the editor for git, and I'm familiar with some common shortcuts for Vim editing. Therefore, the syntax highlighting section mainly refers to the documentation and code examples of Vim. Here I found a good Chinese translation: grammar highlighting

Generally speaking, the grammar rules are: syn vim keyword matching rules, such as:

Syn region rustCommentLine start= "/ /" end= "$" contains=rustTodo,@Spell syn region rustCommentLineDoc start= "/ /\% (/ /\ @!\ |!\)" end= "$" contains=rustTodo,@Spell syn region rustCommentLineDocError start= "/ /\% (/ /\ @!\ |!\)" end= "$" contains=rustTodo,@Spell contained syn region rustCommentBlock matchgroup=rustCommentBlock start= "/\ *\% (!\ |\ * [* /]\ @!\)\ @!" End= "\ * /" contains=rustTodo

It still looks like a regular match, such as Float:

Syn match rustFloat display "\

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.