Shulou(Shulou.com)06/01 Report--

This article shows you how to carry out deep neural network model Softmax DNN analysis, the content is concise and easy to understand, can definitely brighten your eyes, through the detailed introduction of this article, I hope you can get something.

Softmax DNN recommendation

One possible DNN model is softmax, which treats the problem as a multi-class prediction problem, where:

The input is a user query.

The output is a probability vector whose size is equal to the number of items in the corpus, indicating the probability of interacting with each item; for example, the possibility of clicking or watching YouTube videos.

Input

Inputs to DNN can include:

Dense features (for example, viewing time and time since the last viewing)

Sparse features (for example, viewing history and country)

Different from the matrix decomposition method, side features such as age or country region can also be added. Here we use x to represent the input vector.

Figure 1. Input layer x

Model architecture

The architecture of the model determines the complexity and expressiveness of the model. By adding hidden layers and nonlinear activation functions (for example, ReLU), the model can capture more complex relationships in the data. However, increasing the number of parameters usually makes the model more difficult to train and more complex to calculate. The output of the last hidden layer is represented by:

Figure 2. Output of hidden layer, ψ (X)

Softmax output: probability distribution of prediction

Figure 4. Loss function

Softmax training

The previous section explained how to incorporate the softmax layer into the deep neural network of the recommendation system. This section describes the training data for this system in detail.

Training data

The softmax training data consists of the query feature X and the item vector (represented as probability distribution p) with which the user interacts, which is marked in blue in the following figure. The variables of the model are weights in different layers, which are marked with orange in the following illustration. Random gradient descent or its variants are usually used to train the model.

Matrix factorization (FM) VS SOFTMAX

DNN model solves many limitations of matrix decomposition, but it is usually more expensive to train and predict. The following table summarizes some important differences between the two models.

Matrix factorization Softmax DNN

Query features are not easy to include and can include internal cold startup is not easy to handle dictionary queries or items. You can use some heuristic methods (for example, for new queries, average embedding of similar queries) to easily handle new query folds by adjusting the weight not observed in the WALS can easily reduce folding and fold easily, requiring techniques such as negative sampling or gravity to train scalability to easily extend to a very large corpus (possibly hundreds of millions or more) However, only limited to the sparse input matrix, it is difficult to expand to a very large corpus, and some techniques can be used, such as hashing, negative sampling and so on. Providing extensibility embedding UMagar V is static, and a set of candidate items can be pre-calculated and stored. Embedded V is static and can be stored. Query embedding usually needs to be calculated at query time, making the service cost of the model higher.

Matrix decomposition is usually a better choice for large corpora. It is easier to expand, the query computation is cheaper, and it is not easy to collapse.

The DNN model can better capture personalized preferences, but it is difficult to train and the query cost is higher. The DNN model is preferable to the matrix decomposition of the score, because the DNN model can use more features to better capture the correlation.

The above content is how to analyze the deep neural network model Softmax DNN. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.