Shulou(Shulou.com)06/02 Report--

Introduction

Over the past few decades, human computing power has been greatly improved; with the continuous accumulation of data and increasingly advanced algorithms, we have entered the era of artificial intelligence. Indeed, the concept of artificial intelligence is difficult to understand, the technology is even more remarkable, and the data and algorithms behind it are very large and complex. Many people are wondering, what practical applications will AI have now or in the future?

In fact, about the practical application of AI and the commercial value it brings is not so "fantasy", very often it is already around us. Next, through the interpretation of relevant AI papers, the column [interpretation of AI papers] will reveal the secrets of how AI technology empowers the field of e-commerce, as well as related landing and practice.

Artificial intelligence technology has a rich application scene in the field of e-commerce. The application scenario is the data entrance, and the data is refined by technology, which in turn acts on the technology, and the two complement each other.

JD.com developed an AI writing service for commodity marketing content based on natural language understanding and knowledge graph technology. And applied this technology to JD.com Mall [find good goods] channel.

JD.com [find good goods] Channel

Hundreds of thousands of merchandise marketing graphics and text materials created through AI not only fill the huge gap between commodity updates and talent writing content updates, but also enhance the content richness of content channels.

At the same time, the content generated by AI is better than that of manual creation and marketing in terms of exposure click rate and conversion rate of details.

Next, let me read the papers selected in AAAI 2020 to see how to use different marketing strategies and styles of marketing copywriting for different groups through AI to improve the marketing conversion rate.

Automatic text summarization ("automatic summarization" for short) is a traditional task in the field of natural language processing, which was proposed in the 1950s. The goal of the automatic summarization task is to obtain a simplified text containing the most important information for a given text. The commonly used automatic summarization methods include abstract automatic summarization (Extractive Summarization) and generative automatic summarization (Abstractive Summarization). Abstract automatic summarization forms a summary by extracting keywords, phrases or sentences that already exist in a given text; generative automatic summarization generates a summary by establishing an abstract semantic representation of a given text and using natural language generation technology.

This paper introduces a method of generative sentence summarization based on keyword guidance, which combines abstract automatic summarization and generative automatic summarization, and achieves better performance than the comparison model on the Gigaword sentence summary data set.

Paper link: http:// 2234BB08E365EEC

The input of the generative sentence summary (Abstractive Sentence Summarization) task is a long sentence, and the output is a simplified short sentence of the input sentence.

We notice that some important words (that is, keywords) in the input sentence provide clues for the generation of the abstract. On the other hand, when people create abstracts for input sentences, they often first find out the keywords in the input sentences, and then organize the language to concatenate these keywords. Eventually, the generated content will not only cover these keywords, but also ensure their fluency and grammatical correctness. We believe that compared with pure abstract automatic summarization and generative automatic summarization, the generative automatic summarization based on keyword guidance is closer to people's habit of creating abstracts.

Figure 1: the overlapping keywords (marked in red) between the input sentence and the reference summary cover the important information of the input sentence, and we can generate the summary based on the keywords extracted from the input sentence

Let's give an example of a simple sentence summary. As shown in figure 1, we can roughly use the overlapping words of the input sentence and the reference summary (except stop words) as keywords, which cover the main points of the input sentence. For example, by using the keywords "world leaders", "closure" and "Chernobyl", we can get the main message of the input sentence, that is, "world leaders call for the closure of Chernobyl". This is consistent with the actual reference summary "World leaders urge support for the Chernobyl nuclear power plant closure plan". This phenomenon is common in sentence summary tasks: in the Gigaword sentence summary dataset, more than half of the words in the reference summary appear in the input sentence.

The input of the sentence summary task is a long sentence and the output is a short text summary. Our motivation is that entering keywords in the text can provide important guidance information for the automatic summarization system. First of all, we take the overlapping words (except stop words) between the input text and the reference abstract as Ground-Truth keywords, encode the input text by sharing the same encoder through multi-task learning, and train the keyword extraction model and summary generation model, in which the keyword extraction model is a sequence tagging model based on the state of the hidden layer of the encoder, and the summary generation model is an end-to-end model based on keyword guidance. After both the keyword extraction model and the summary generation model are trained and converged, we use the trained keyword extraction model to extract keywords from the text in the training set, and use the extracted keywords to fine-tune the summary generation model. In the test, we first use the keyword extraction model to extract keywords from the text in the test set, and finally use the extracted keywords and the original test text to generate a summary.

1. Multi-task learning

In a sense, the text summary task and the keyword extraction task are very similar in order to extract the key information from the input text. The difference lies in the form of its output: the text summary task outputs a complete piece of text, while the keyword extraction task outputs a collection of keywords. We believe that both tasks require the ability of the encoder to recognize important information in the input text. Therefore, we use the multi-task learning framework to share the two task encoders to improve the performance of the encoder.

2. Abstract generation model based on keyword guidance.

Inspired by the work of Zhou et al. [1], we propose a selective coding based on keyword guidance. Specifically, because keywords contain more important information, through the guidance of keywords, we build a selection gate network, which re-encodes the semantic information of the hidden layer of the input text to construct a new hidden layer. The subsequent decoding is carried out based on this new hidden layer.

Our decoder is based on Pointer-Generator network [2], that is, an end-to-end model with replication mechanism. For the Generator module, we propose direct connection, gate fusion and hierarchical fusion to fuse the context information of the original input text and keywords; for the Pointer module, our model can selectively copy the text from the original input and keywords to the output summary.

1. Data set

In this experiment, we choose to experiment on the Gigaword dataset, which contains about 3.8 million training sentence summary pairs. We used 8000 pairs as the verification set and 2000 pairs as the test set.

2. Experimental results

Table 1 shows that our proposed model performs better than the model without keyword guidance. We tested different selective coding mechanisms, namely, self-selection, keyword selection and mutual selection of input text, and the experimental results show that mutual selection is the best; for Generator module, we find that hierarchical fusion is better than the other two fusion methods; our bi-directional Pointer module performs better than the original model that can only be copied from the input text.

Table 1

This paper is devoted to the task of generative sentence summary, that is, how to transform a long sentence into a short summary. Our proposed model can use keywords as guidance to generate more high-quality abstracts and achieve better results than the comparative model.

1) by using a multi-task learning framework to extract keywords and generate abstracts.

2) through the keyword-based selective coding strategy, important information is obtained in the coding process.

3) through the dual attention mechanism, the information of the original input sentences and keywords is dynamically integrated.

4) through the double replication mechanism, the words in the original input sentences and keywords are copied to the output summary.

On the standard sentence summary data set, we verify the effectiveness of keywords to the sentence summary task.

Note:

[1] Zhou, Q.; Yang, N.; Wei, F.; and Zhou, M. 2017. Selective encoding for abstractive sentence summarization. In Proceedings of ACL, 1095-1104.

[2] See, A.; Liu, P. J.; and Manning, C. D. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of ACL, 1073-1083.

In the last column, we introduced in detail how JD.com Mall carries out further technical research and innovation on the existing basis, so as to effectively improve the marketing conversion rate of e-commerce. For more information, please click below.

Revealing the Technical ability of AI behind JD.com Mall-automatically generating Abstracts based on keywords

JD.com AI Research Institute

JD.com AI Research Institute focuses on continuous algorithm innovation, and most of the research will be driven by JD.com 's actual business scenarios. The research institute focuses on computer vision, natural language understanding, dialogue, pronunciation, semantics, machine learning and other laboratories, and has gradually set up workplaces in Beijing, Nanjing, Chengdu, Silicon Valley and other parts of the world.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.