Shulou(Shulou.com)06/01 Report--

How to use PyTorch to distinguish natural language, in view of this problem, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and easy way.

Recently, I have been learning practical examples of pyTorch. Let's give a simple example this time: given a sentence, determine what language it is. This example is, for example, a given sentence:

Give it to me judgment is ENGLISH me gusta comer en la cafeteria judgment is SPANISH

It's such a simple example.

Let's see how to achieve it:

Prepare the data format [(statement, type), …]

Data is the statement used in train, and test_data is the statement used in test.

Data = [("me gusta comer en la cafeteria" .split (), "SPANISH"), ("Give it to me" .split (), "ENGLISH"), ("No creo que sea una buena idea" .split (), "SPANISH"), ("No it is not a good idea to get lost at sea" .split (), "ENGLISH")] test_data = [("Yo creo que si" .split (), "SPANISH") ("it is lost on me" .split (), "ENGLISH")]

Because the text computer room can not recognize, they only know 01 strings, that is, numbers. So we have to map text to numbers.

Word_to_ix = {} for sent, _ in data + test_data: for word in sent: if word not in word_to_ix: word_to_ ix [word] = len (word_to_ix) print (word_to_ix)

The output word_to_ix (meaning word to index) is:

{'me': 0,' gusta': 1, 'comer': 2,' en': 3, 'la': 4,' cafeteria': 5, 'Give': 6,' it': 7, 'to': 8,' No': 9, 'creo': 10,' que': 11, 'sea': 12,' una': 13, 'buena': 14,' idea': 15, 'is': 16,' not': 17 'asides: 18, 'good': 19,' get': 20, 'lost': 21,' at': 22, 'Yo': 23,' si': 24, 'on': 25}

Here, set the parameters to be used in advance.

VOCAB_SIZE = len (word_to_ix) NUM_LABELS = only two types of ENGLISH SPANISH

Fixed template

Def init (self, num_labels, vocab_size): initialization is the size of input and output. Here we are going to input a sentence, the sentence * * is the word with all the dictionaries, and here is vocab_size (how to convert a sentence into a sequence of numbers according to the dictionary), and the output is classification, which is divided into two categories, namely num_labels. Here we use a linear classification, that is, nn.Linear ().

Def forward (self, bow_vec): bow_vec is a digital sequence of sentences. After self.linear (), a linear result (that is, the prediction result) is obtained, and then softmax is performed on this result (log_softmax is used here because the following loss function uses NLLLoss (), that is, negative logarithmic likelihood loss, which needs to be below log).

Class BoWClassifier (nn.Module): # nn.Module this is the neural network template def _ _ init__ (self, num_labels, vocab_size): super (BoWClassifier, self). _ _ init__ () self.linear = nn.Linear (vocab_size, num_labels) def forward (self, bow_vec): return F.log_softmax (self.linear (bow_vec) def make_bow_vector (sentence) Word_to_ix)

I think I can understand what it means. Is to convert a sentence sentence into a digital sequence through word_to_ix. For example, sentence= I am a little bird word_to_id= {you: 0, I: 1, he: 2, not: 3, is: 4, big: 5, small: 6, pig: 7, bird: 8 make_bow_vector,} the result is [0pr 1m 0je 0jol 0je 0j1]. View () is to change the dimension of the downvector.

This is about len (word_to_ix) 1-> 1len (word_to_ix).

Def make_bow_vector (sentence, word_to_ix): vec = torch.zeros (len (word_to_ix)) for word in sentence: vector [word _ to_ ix]] + = 1 return vec.view (1,-1)

You don't have to say that. Same thing. If you want to know what torch.LongTensor means. You can take a look. In Torch, Tensor mainly includes ByteTensor (unsigned char), CharTensor (signed), ShortTensor (shorts), IntTensor (ints), LongTensor (longs), FloatTensor (floats), DoubleTensor (doubles), which are stored as double by default. If you need to point out specifically, set them through torch.setdefaulttensortype () method. For example, torch.setdefaulttensortype ('torch.FloatTensor'). )

Def make_target (label, label_to_ix): return torch.LongTensor ([label_to_ ix [label]])

Let's introduce the function model.parameters () again. His return result is all the parameters in the model. Here we use a linear function, so it is An and b in f (x) = Ax+b (x is the input data), which are needed for feedback and update of these parameters later.

Model = BoWClassifier (NUM_LABELS, VOCAB_SIZE) for param in model.parameters (): print ("param:", param)

You can see that An is 2len (vocab_size) and b is 21

Param: Parameter containing: Columns 0 to 9 0.0786 0.1259 0.0054 0.0558-0.0911-0.1804-0.1526-0.0287-0.1086-0.0651-0.1096-0.1807-0.1907-0.0727-0.0179 0.1530-0.0910 0.1943-0.1148 Columns 10 to 19 0.0452-0.0786 0.1776 0.0425 0.1194-0.1330-0.1877-0.0412-0 .0269-0.1572-0.0361 0.1909 0.1558 0.1309 0.1461-0.0822 0.1078-0.1354-0.1877 0.0184 Columns 20 to 25 0.1818-0.1401 0.1118 0.1002 0.1438 0.0790 0.1812-0.1414 14-0.1876 0.1569 0.0804-0.1897 [torch.FloatTensor of size 2x26] param: Parameter containing: 0.1859 0.1245 [torch.FloatTensor of size 2]

Let's take a look at model's def forward (self, bow_vec): how to use it. Just like the following code, just fill in a parameter in mode () and call the forward function.

Sample = data [0] bow_vector = make_bow_vector (sample [0], word_to_ix) log_probs = model (autograd.Variable (bow_vector)) print ("log_probs", log_probs)

The output is: (that is, the value after log_softmax)

Log_probs Variable containing:-0.6160-0.7768 [torch.FloatTensor of size 1x2]

Let's take a look at the prediction on test.

Label_to_ix = {"SPANISH": 0, "ENGLISH": 1} for instance, label in test_data: bow_vec = autograd.Variable (make_bow_vector (instance, word_to_ix)) log_probs = model (bow_vec) print log_probs print next (model.parameters ()) [:, word_to_ix ["creo"]]

As a result,

Variable containing:-0.5431-0.8698 [torch.FloatTensor of size 1x2] Variable containing:-0.7405-0.6480 [torch.FloatTensor of size 1x2] Variable containing:-0.0467 0.1065 [torch.FloatTensor of size 2]

Now it's time for the important part.

Cyclic training and updating parameters

The loss function we use here is nn.NLLLoss () negative logarithmic likelihood loss. The most common optim.SGD () gradient descent method is still used in optimization, and the final optimization is basically unchanged after 5-30 times of general training.

Each step:

a. First of all, you have to model.zero_grad (), because the next extreme gradient, you have to clear zero in case of problems.

b. Vectorization of data (or digital serialization, into a form that the computer can understand)

c. Get the predicted value

d. Seeking for loss loss_function

e. Find the gradient loss.backward ()

f. Update the parameter optimizer.step ()

Loss_function = nn.NLLLoss () optimizer = optim.SGD (model.parameters (), lr=0.1) for epoch in range (100): for instance, label in data: model.zero_grad () bow_vec = autograd.Variable (make_bow_vector (instance, word_to_ix)) target = autograd.Variable (make_target (label) Label_to_ix)) log_probs = model (bow_vec) loss = loss_function (log_probs, target) loss.backward () optimizer.step ()

Test on a test set

For instance, label in test_data: bow_vec = autograd.Variable (make_bow_vector (instance, word_to_ix)) log_probs = model (bow_vec) print log_probs

It is easy to see in the results that the first example predicts SPANISH and the second is ENGLISH. Succeed.

Variable containing:-0.0842-2.5161 [torch.FloatTensor of size 1x2] Variable containing:-2.4886-0.0867 [torch.FloatTensor of size 1x2] this is the answer to the question on how to identify natural language with PyTorch. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.