Shulou(Shulou.com)06/03 Report--

This article is to share with you about the functions of Nlpir Parser search and mining intelligent platform. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Text mining has become an increasingly popular and important research field in data mining. Different from the general data mining which focuses on relations, transactions and structural data in data warehouse, the text database studied by text mining is composed of a large number of documents from various data sources. These documents may contain structured data such as title, author, publication date and length, or unstructured text components such as abstracts and content, and the content of these documents is a natural language used by human beings. it is difficult for computers to deal with their semantics. Therefore, the traditional information retrieval technology can no longer meet the needs of the increasing amount of text data processing, and then people put forward the method of text mining to compare different documents and arrange the importance and relevance of documents. or find out the patterns or trends of multiple documents.

Nlpir Parser search and mining intelligent platform is the basic tool set of web search, natural language understanding and text mining technology development. The development platform is composed of several middleware, each middleware API can be seamlessly integrated into all kinds of complex application systems of customers, can be compatible with different operating systems such as Windows,Linux,FreeBSD, and can be used by various development languages such as Java,C,C#.

Nlpir Parser search and mining intelligent platform is a set of software specially for the processing and processing of the original text set, which provides a visual display of the processing effect of middleware, and can also be used as a processing tool for small-scale data. Users can use the software to process their own data.

Twelve functions of Nlpir Parser search and mining intelligent platform:

1. Full-text accurate retrieval: support text, number, date, string and other data types, multi-field efficient search, support AND/OR/NOT and NEAR neighbor query syntax, support Uighur, Tibetan, Mongolian, Arabic, Korean and other minority languages. It can be seamlessly integrated with existing text processing systems and database systems.

two。 New word discovery: mining the list of new words with connotation from the file collection can be used for the compilation of users' professional dictionaries; it can also be further edited and labeled and imported into the word segmentation dictionary, so as to improve the accuracy of the word segmentation system and adapt to the new language changes.

3. Word segmentation tagging: word segmentation of the original corpus, automatic recognition of unknown words such as person names and place names, neologism tagging and part of speech tagging. User-defined dictionaries can be imported during the analysis process.

4. Statistical analysis and terminology translation: according to the results of segmentation tagging, the system can automatically carry out unary word frequency statistics and binary word transfer probability statistics (statistics of the frequency of about two words, namely probability). For commonly used terms, the corresponding English explanations will be given automatically.

5. Text clustering and hot spot analysis: can automatically analyze hot events from large-scale data and provide key feature description of event topics. At the same time, it is suitable for the hot spot analysis of long text and short text, such as SMS and Weibo.

6. Classification filtering: according to the pre-specified rules and sample samples, the system automatically filters out samples that meet the requirements from a large number of documents.

7. Positive and negative analysis: for the pre-specified analysis objects and sample samples, the system automatically selects positive and negative scores and sentence samples from a large number of documents.

8. Automatic summary: can automatically extract the essence of the content of single or multiple articles, making it convenient for users to quickly browse the text content.

9. Keyword extraction: for a single article or a collection of articles, several words or phrases that represent the central idea of the article can be extracted, which can be used for refined reading, semantic query and fast matching.

10. Document deduplication: can quickly and accurately determine whether there are records of the same or similar content in the file collection or database, and find out all duplicate records at the same time.

11. HTML text extraction: automatically remove navigation web pages, remove HTML tags and intrusive text such as navigation and advertising, and return valuable text content. It is suitable for the preprocessing and analysis of large-scale Internet information.

twelve。 Automatic coding recognition and conversion: automatically identify the coding of the content, and uniformly convert the coding into GBK coding.

In most cases, the data set of text mining is so large and increasing that it is impossible to store the data on a single machine for operation. Therefore, it is necessary to study a text mining algorithm that can run in parallel to perform text mining tasks in parallel on the computer cluster. Obviously, this combines the needs of cloud computing and data-intensive computing, and this in itself is a growing area.

Thank you for reading! This is the end of this article on "what are the functions of Nlpir Parser search and Mining Intelligent platform". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.