Shulou(Shulou.com)06/03 Report--

This article mainly introduces "After sorting out hundreds of questions, I have summarized 10 concepts that must be mastered in data science interviews..." In daily operation, I believe that after sorting out hundreds of questions, I have summarized 10 concepts that must be mastered in data science interviews... There are doubts on the questions. Xiao Bian consulted all kinds of materials and sorted out simple and easy operation methods. I hope to answer "After sorting out hundreds of questions, I have summarized 10 data science interview must master the concept of…"doubts help! Next, please follow the small series to learn together!

1. Activation Functions

Once you have a basic understanding of neurons or nodes, you will find that activation functions, like light switches, can decide whether to activate neurons or not.

There are several types of activation functions, but the most popular are rectifying linear unit functions, also known as ReLU functions. The ReLU function is superior to the sigmoid and hyperbolic tangent functions because it performs gradient descent faster.

Note that in the image, when x(or z) is large, the slope is very small, which significantly slows down gradient descent, but this does not occur in the ReLU function.

2. Cost Function

The cost function of neural networks is similar to that of other machine learning models. It measures how "good" the neural network's predictions are compared to the actual ones. The cost function is inversely proportional to the quality of the model-the higher the quality of the model, the lower the cost function, and vice versa.

The cost function is optimal. By reducing the cost function of neural network, the optimal weights and parameters of the model can be obtained, so as to maximize the performance of the model.

There are several commonly used cost functions, including quadratic cost, cross-entropy cost, exponential cost, Hellinger distance, Kullback-Leibler divergence, etc.

3. Backpropagation algorithm

Backpropagation algorithm is an algorithm closely related to cost function. Specifically, it is an algorithm for computing the gradient of a cost function. Compared with other algorithms, backpropagation is fast and efficient, so it is popular.

In this algorithm, the gradient calculation starts with the gradient of the last layer of weights and then propagates back to the gradient of the first layer of weights. Therefore, the error of the kth layer depends on the k + 1 layer. That's why it's called backpropagation.

In general, backpropagation works as follows:

Compute the loss function for each input-output pair in the forward phase

Compute the loss function for each pair of reverse phases

Combine the gradient values of the individual weights

Update weights based on learning rate and total gradient

Convolutional Neural Networks (CNN)

A convolutional neural network (CNN) is a neural network that extracts input information (usually an image), classifies different features of the image according to importance, and then outputs predictions. The reason CNN is superior to feedforward neural networks is that it can better capture spatial (pixel) dependence of the entire image, which means it can better understand the composition of the image.

CNN uses a mathematical operation called convolution. Wikipedia defines convolution this way: mathematical operations on two functions produce a third function that represents how the shape of one function is modified by the other. Thus, CNN uses convolution instead of universal matrix multiplication in at least one of its layers.

5. Recurrent Neural Networks

Recurrent neural networks (RNNs) are another type of neural network that can ingest input information of various sizes and therefore work well with sequential data. RNNs take into account both current and previously given inputs, meaning that the same input can technically produce different outputs based on the previously given input.

Technically, RNNs are neural networks in which connections between nodes form a directed graph along a time series, so that they can use their internal memory to process input sequences of variable length.

6. Long Short-Term Memory Networks

A long-short term memory network (LSTM) is a recurrent neural network that compensates for one of the disadvantages of conventional RNNs: short-term memory.

Specifically, if the sequence is too long, i.e. the lag time is greater than 5-10 steps, the RNN tends to ignore the information provided in the previous step. For example, if we enter a paragraph into an RNN, it might ignore the information provided at the beginning of the paragraph. To solve this problem, LSTM was born.

7. Weight initialization

The point of weight initialization is to ensure that the neural network does not converge to an invalid solution. If the weights are all initialized to the same value (e.g., zero), each cell will get exactly the same signal, and the output of each layer will be the same.

So you want to initialize the weights randomly so they are close to zero, but not equal to zero. Stochastic optimization algorithms used to train models are designed to do just that.

Batch vs. Stochastic Gradient Descent

Batch gradient descent and stochastic gradient descent are two different methods for computing gradients.

Batch Gradient Descent calculates gradients using only the entire dataset. It is much slower, especially for larger data sets, but works better for convex or smooth error manifolds.

In the case of stochastic gradient descent, a single training sample is used at a time to compute the gradient. As a result, it is faster and cheaper to compute. However, it tends to bounce when it reaches a global optimum in this case. This produces good solutions, but not optimal solutions.

9. Hyper-parameters

Hyperparameters are not only variables that regulate the network structure, but also variables that control the training mode of the network. Common hyperparameters include:

Model architecture parameters, such as layers, number of hidden cells, etc.

Learning Rate (alpha)

initialization of network weights

Number of periods (defined as one cycle in the entire training dataset)

batch

other

10. Learning Rate

The learning rate is a hyperparameter used in the neural network that controls the number of error-adjusted models that the model responds to estimate each time the model weights are updated.

If the learning rate is too low, model training will proceed very slowly because the weights are updated minimally in each iteration. Therefore, multiple updates are required before the nadir is reached. If the learning rate is set too high, the loss function may exhibit undesirable divergence behavior due to abrupt updates of the weights and may fail to converge.

At this point, the study on "After sorting out hundreds of questions, I have summarized 10 concepts that must be mastered in data science interviews..." is over, hoping to solve everyone's doubts. Theory and practice can better match to help everyone learn, go and try it! If you want to continue learning more relevant knowledge, please continue to pay attention to the website, Xiaobian will continue to strive to bring more practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.