Shulou(Shulou.com)06/02 Report--

This article mainly introduces what natural language processing refers to in the computer, which has a certain reference value. Interested friends can refer to it. I hope you will gain a lot after reading this article. Let's take a look at it with you.

Natural language processing is a science that integrates linguistics, computer science and mathematics. Natural language processing is mainly used in machine translation, public opinion monitoring, automatic summarization, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, Chinese OCR and so on.

Natural language processing (Natural Language Processing, NLP) is a science that integrates linguistics, computer science and mathematics.

Natural language processing (NLP) refers to the technology of interactive communication between machines and natural languages used in human communication. Through the artificial processing of natural language, the computer can read and understand it. The related research of natural language processing begins with the human exploration of machine translation. Although natural language processing involves multi-dimensional operations such as phonetics, grammar, semantics, pragmatics and so on, the basic task of natural language processing is to segment the processing corpus based on ontology dictionary, word frequency statistics, context semantic analysis and so on. to form a lexical unit with minimum part of speech and rich in semantics.

Natural language processing is a subject that takes language as the object and uses computer technology to analyze, understand and deal with natural language, that is, the computer is used as a powerful tool for language research, and the quantitative study of language information is carried out with the support of computer. and provide language description that can be used jointly by people and computers. It includes natural language understanding (NaturalLanguage Understanding, NLU) and natural language generation (Natural LanguageGeneration, NLG). It is a typical marginal cross-discipline, which involves language science, computer science, mathematics, cognitive science, logic and so on, and pays attention to the interaction between computer and human (natural) language. The process of using computer to deal with natural language is also called natural language understanding (Natural Language Understanding, NLU), human language technology (Human Language Technology, HLT), computational linguistics Hl (Computational Linguistics), econometric linguistics (QuantitativeLinguistics) and mathematical linguistics (Mathematical Linguistics).

The realization of natural language communication between man and machine means that the computer can not only understand the meaning of the natural language text, but also express the given intention and thought in the natural language text. The former is called natural language understanding and the latter is called natural language generation. Therefore, natural language processing generally includes two parts: natural language understanding and natural language generation. In history, there are many studies on natural language understanding, but less on natural language generation. But that has changed.

Correlation

The realization of natural language understanding and natural language generation is not as simple as people originally thought, but very difficult. From the current situation of existing theory and technology, a general-purpose and high-quality natural language processing system is still a long-term goal, but for certain applications, practical systems with considerable natural language processing capabilities have emerged, and some have been commercialized or even industrialized. Typical examples are: natural language interface of multilingual database and expert system, various machine translation systems, full-text information retrieval system, automatic abstracting system and so on.

Natural language processing, that is, the realization of natural language communication between man and machine, or the realization of natural language understanding and natural language generation is very difficult. The root cause of the difficulty is the various ambiguities or polysemy (ambiguity) that widely exist at all levels of natural language texts and dialogues.

There is a many-to-many relationship between the form (string) and the meaning of a natural language. In fact, this is the charm of natural language. However, from the perspective of computer processing, we must eliminate ambiguity, and some people think that it is the central problem in natural language understanding, that is, to convert potentially ambiguous natural language input into some kind of unambiguous internal computer representation.

The widespread existence of ambiguity requires a lot of knowledge and reasoning to eliminate them, which brings great difficulties to linguistics-based methods and knowledge-based methods. therefore, the research on natural language processing based on these methods has made a lot of achievements in theories and methods over the past few decades, but the achievements in the development of systems that can deal with large-scale real texts are not significant. Most of the systems developed are small-scale and research demonstration systems.

At present, there are two problems: on the one hand, the grammar so far has been limited to the analysis of an isolated sentence, and there is a lack of systematic research on the constraints and effects of context and conversation environment on this sentence. Therefore, there are no clear rules to follow in the analysis of ambiguity, word ellipsis, pronoun reference, different meanings of the same sentence on different occasions or by different people. It needs to strengthen the study of pragmatics in order to solve it step by step. On the other hand, people understand a sentence not only by grammar, but also use a lot of relevant knowledge, including life knowledge and specialized knowledge, which can not all be stored in the computer. Therefore, a written understanding system can only be established within a limited range of words, sentence patterns and specific topics, and only when the storage capacity and running speed of the computer are greatly improved can the scope be expanded properly.

The above problems have become the main problems in the application of natural language understanding in machine translation, which is one of the reasons why the translation quality of today's machine translation system is still far from the ideal goal, and the translation quality is the key to the success or failure of machine translation system. Professor Zhou Haizhong, a Chinese mathematician and linguist, once pointed out in the classic paper "Fifty years of Machine Translation": in order to improve the quality of machine translation, the first thing to solve is the problem of language itself rather than the problem of programming; it is certainly impossible to improve the quality of machine translation by relying solely on some programs to do machine translation system. In addition, under the condition that human beings do not understand how the brain carries out fuzzy recognition and logical judgment of language, it is impossible for machine translation to achieve the degree of "faithfulness, expressiveness and elegance".

Thank you for reading this article carefully. I hope the article "what is Natural language processing in computer" shared by the editor will be helpful to everyone? at the same time, I also hope that you will support and pay attention to the industry information channel. more related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.