Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of the way of training data in keras, the content is detailed and easy to understand, the operation is simple and fast, and it has a certain reference value. I believe you will gain something after reading this keras training data. Let's take a look at it.

Keras training data can be used in many ways, among which the three common ones are fit, fit_generator and train_on_batch. The third kind is quite different from the first two, so this article mainly compares fit and fit_generator.

1. Train_on_batchmodel.train_on_batch (batchX, batchY)

The train_on_batch function accepts a single batch of data, performs back propagation, and then updates the model parameters. The size of the batch data can be arbitrary, that is, it does not need to provide an explicit batch size and belongs to the fine control training model. In most cases, we do not need to be so fine. We can use fit_generator training method in 99% of cases, as described below.

2. Fitmodel.fit (x_train, y_train, batch_size=32, epochs=10)

The way of fit is to load all the training data into memory at once, and then update the model parameters by batch processing batch_size data at a time. Epochs does not need to introduce too much. This training method is only suitable for use when the amount of training data is relatively small.

III. Fit_generator

Using the generator of Python, the batch of data is generated and trained one by one, which does not take up a lot of memory. At the same time, the generator and the model will be executed in parallel to improve efficiency. For example, this function allows us to do real-time data promotion on CPU and model training on GPU

The API is as follows:

Fit_generator (self, generator, steps_per_epoch, epochs=1, verbose=1, callbacks=None, validation_data=None, validation_steps=None, class_weight=None, max_q_size=10, workers=1, pickle_safe=False, initial_epoch=0)

Generator: generator function

Steps_per_epoch: integer. When the generator returns steps_per_epoch secondary data, one epoch is counted and the next epoch is executed. That is, how many times the batch_size is executed under an epoch.

Epochs: an integer that controls the number of rounds of data iteration and ends the training when it arrives.

Callbacks=None. The element in list,list is the keras.callbacks.Callback object, and the callback function in list will be called during training.

For example: def generate_arrays_from_file (path): while True: with open (path) as f: for line in f: # create numpy arrays of input data # and labels, from each line in the file x1, x2 Y = process_line (line) yield ({'input_1': x1,' input_2': x2}, {'output': y}) model.fit_generator (generate_arrays_from_file ('. / my_folder'), steps_per_epoch=10000, epochs=10)

Supplement: keras.fit_generator () attribute and its value

As follows: fit_generator (self, generator, steps_per_epoch=None, epochs=1, verbose=1, callbacks=None, validation_data=None, validation_steps=None, class_weight=None, max_queue_size=10 Workers=1, use_multiprocessing=False, shuffle=True, initial_epoch=0)

Batches of data are generated through Python generator to train the model. Generator can run in parallel with the model, for example, you can use CPU to generate batch data while training the model on GPU.

Parameters:

Generator: an instance of generator or Sequence to avoid copying data directly when using multiprocessing.

Steps_per_epoch: the total number of steps generated from the generator (total number of sample batches). Typically, it should be equal to the number of samples in the dataset divided by the size of the batch.

Epochs: integer, the total number of iterations over the dataset.

Works: the maximum number of processes that need to be started when using process-based threads.

Use_multiprocessing: Boolean value. When True, use process-based threads.

For example: datagen = ImageDataGenator (...) model.fit_generator (datagen.flow (x_train, y_train, batch_size=batch_size), epochs=epochs, validation_data= (x_test, y_test), workers=4) this is the end of the article on "what are the ways to train data in keras?" Thank you for reading! I believe you all have a certain understanding of the knowledge of "what are the ways of training data in keras". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.