Shulou(Shulou.com)06/01 Report--

What is NLP?

In the computer field, NLP (Natural Language Processing), which is often called "natural language processing", is the study of how to make computers understand human language. This includes that the computer should not only understand the meaning of the natural language text, but also use the natural language text to express the given deep intention, thought and so on. Therefore, this technology often embodies the highest task and realm of artificial intelligence, that is to say, only when the computer has the ability to understand natural language can the machine achieve real intelligence. However, because Chinese words are made up of ever-changing Chinese characters, the "natural language processing" in the Chinese field is particularly complex. Therefore, the research in this field will involve natural language, that is, the language that people use every day, so it is closely related to the study of linguistics, but there are important differences. Natural language processing is not to study natural language in general, but to develop a computer system, especially the software system, which can effectively realize natural language communication. So it's part of computer science. Natural language processing (NLP) is a field of computer science, artificial intelligence and linguistics concerned with the interaction between computers and human (natural) language.

Focusing on the field of "natural language processing" for many years, the technology and services of NLP have been improving with the rapid development. From automatic translation, information retrieval, automatic indexing, automatic summarization, automatic story writing and other fields, we can use our tool class DKNLPBase to deal with, NLP technology is no longer simply stay at the conceptual level, but gradually and successfully applied in various fields.

Why do you need NLP

For example, in our daily life, we always come across some obscure words that we don't know how to read, and then we often go to search engines, such as "what are you going to read?" We found that the search results must show you what the "four words" look like, accompanied by pinyin and notes, rather than the lonely words "what to read" or their superficial matching results.

This is actually an embodiment of NLP technology. Through this technology, people do not have to spend a lot of effort to learn and understand the difficult computer language, but use the language they are most accustomed to to use the computer, and further understand the meaning behind it.

What on earth can NLP be used for?

The big fast NLP module is a component of the big fast big data integrated platform. Users can refer to this component to effectively carry out natural language processing, such as summarizing articles, semantic discrimination and improving the accuracy and effectiveness of content retrieval.

Nowadays, natural language processing is studied not only as the core topic of artificial intelligence, but also as the core topic of the new generation of computers. From the point of view of knowledge industry, expert system, database, knowledge base, computer aided design system (CAD), computer aided instruction system (CAI), computer aided decision system, office automation management system, intelligent robot and so on all need to be processed by natural language. The natural language understanding system with text comprehension ability can be used in the fields of automatic machine translation, information retrieval, automatic indexing, automatic summarization, automatic story writing and so on, all of which can be processed by our tool class DKNLPBase.

Standard participle

Method signature: List StandardTokenizer.segment (String txt)

Return: list of participles.

Signature parameter description: txt: the statement to be participle.

Example: the following example verifies that the fifth participle in a paragraph is AlphaGo.

Public void testSegment () throws Exception

{

String text = "goods and services"

List termList = DKNLPBase.segment (text)

AssertEquals ("Commodity", termList.get (0) .word)

AssertEquals ("and", termList.get (1) .word)

AssertEquals ("Services", termList.get (2) .word)

Text = "Ke Jie explains that Lee se-dol VS AlphaGo's second inning ends like this."

TermList = DKNLPBase.segment (text)

AssertEquals ("AlphaGo", termList.get (5) .word); / / able to identify "AlphaGo"

}

Keyword extraction

Method signature: List extractKeyword (String txt,int keySum)

Return: keyword list.

Signature parameter description: txt: the statement to extract keywords, and the number of keywords to be extracted by keySum

Example: give a paragraph to extract a key word is "programmer".

Public void testExtractKeyword () throws Exception

{

String content = "programmers (English Programmer) are professionals engaged in program development and maintenance." +

Programmers are generally divided into programmers and programmers.

"but the line between the two is not very clear, especially in China." +

"Software practitioners are divided into junior programmers, senior programmers and systems" +

"there are four categories of analysts and project managers."

List keyword = DKNLPBase.extractKeyword (content, 1)

AssertEquals (1, keyword.size ())

AssertEquals (programmer, keyword.get (0))

}

Phrase extraction

Method signature: List extractPhrase (String txt, int phSum)

Return: phrase

Signature parameter description: txt: statement to extract phrase, number of phSum phrase

Example: give a passage that represents the five phrases of the article, the first of which is an algorithm engineer.

NLP has made good progress in recent years, but there are still many problems to be solved, so we are actively trying, but it is such a challenging problem that can make more talented people devote themselves to Daxi to promote its development.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.