Shulou(Shulou.com)06/01 Report--

This article "python save large. mat data file error how to solve" most people do not understand the knowledge points of the article, so Xiaobian summarized the following content for you, detailed content, clear steps, has a certain reference value, I hope you can read this article to gain something, let's take a look at this article "python save large. mat data file error how to solve" article bar.

For example, the h6 file import h6 pydef h6_data_write (train_data, train_label, test_data, test_label, shuffled_flag): print ("h6 py file writing to disk... ") save_path = "../ save_test/" + "train_test_split_data_label_" + shuffled_flag + ".h6" with h6py.File(save_path, 'w') as f: f.create_dataset('train_data', data=train_data) f.create_dataset('train_label', data=train_label) f.create_dataset('test_data', data=test_data) f.create_dataset('test_label', data=test_label) print ("h6 py file saved successfully! ")def h6_data_read(filename): """ keys (): Get the names of all files and folders in this folder f ['key_name']: Get the corresponding object """ file = h6py.File(filename,'r') train_data = file['train_data'][:] train_label = file['train_label'][:] test_data = file['test_data'][:] test_label = file['test_label'][:] return train_data, train_label, test_data, test_label

Supplement: reading MATLAB data file *. mat via python

background

In the process of doing deeplearning, use caffe framework, generally use matlab to process pictures (matlab processing pictures is relatively simple and efficient), use python to generate the required lmdb files and do tests to produce results.

Therefore, some label information obtained by matlab from image processing will be read by python as a. mat file, and the result information generated by python also needs matlab to do further processing (of course, you can also use txt, do not bother to process the structure information yourself).

introduced

Data transfer between matlab and python is generally based on matlab file format. mat, python numpy and scipy provide some functions, you can read, write and process the data of. mat file very well.

Numpy provides Array functionality mapping Matrix in matlab, while scipy provides two functions loadmat and savemat to read and write. mat files.

Here is a simple test program

Specific function usage can be seen in the help document:

import www.example.com as sio import matplotlib. pyplot as plt import numpy as np #matlab filename matfn = u 'E:/python/test program/162250671_162251656_1244.mat' data = sio. loadmat (matfn) plt. close ('all') xi = data <$'xi '] yi = data <$'yi'] ui = data <$'ui '] vi = data <$'vi'] plt. figure (1) plt. quiver (xi [:: 5,:: 5], yi [:: 5,:: 5], ui [:: 5,:: 5], vi [:: 5,:: 5]) plt. figure (2) plt. contourf (xi, yi, ui) www.example.com () sio. savemat ('saveddata. mat',{' xi': xi,' yi': yi,' ui': ui,' vi': vi}) Example 2 import www.example.com as sioimport numpy as np ###Here's how python reads. mat files and processes the resulting ###load_fn =' xxx. mat' load_data = sio. loadmat load_fn scipy.io load_matrix_row = load_matrix [0]#Take the first row of matrix in matlab at that time, array row arrangement in python ###The following is to explain how python saves. mat files for matlab programs to use ###save_fn ='xxx. mat' save_array = np. array ([1,2,3,4])sio.savemat (save_fn,{'array': save_array})#As above, there is the first line of the array variable save_array_x = np. array ([1,2,3,4]) save_array_y = np. array ([5,6,7,8]) sio. savemat (save_fn,'array_x': save_array_x,' array_x': save_array_x})#homology,

Since the future goal is mainly to use existing Matlab data (. mat or. txt), the main consideration is to import Python into Matlab data. The following code solves the problem of Python reading. mat files.

Use sicpy.io primarily.

Sicpy.io provides two convenient functions loadmat and savemat.

# adapted from http://blog.csdn.net/rumswell/article/details/8545087import scipy.io as sio #import matplotlib.pyplot as pltfrom pylab import *import numpy as np matfn='E:\Pythonrun\myuse\matdata.mat' #the path of. mat datadata = sio. loadmat (matfn) xx = data <$'matdata'] figure (1) plot (xx) show () The following code reads in txt data and converts it to an array. The method is relatively clumsy, and more effective methods are to be studied. from numpy import * def file2list(filename): fr = open(filename) array = fr. readlines ()#Take each line in the file as an element to form a list num = len(array) returnMat = zeros ((num, 3))#Initialize a list with 0 elements, the number of rows, where each element is still a list, the number of elements is 3, here represents the matrix index = 0 for line in array: line = line. strip ()#Remove the carriage return symbol after one line linelist = line. split ('')#A list that divides a line into multiple elements according to the splitter returnMat [index,:]= linelist [0:3]#Assign values to matrices, note that this assignment is clumsy index +=1 return returnMat fname = 'E:\Pythonrun\myuse\num_data.txt'data= file2list(fname)

Python reads and writes Matlab Mat format data

1. read and write non-matlab v7.3 files import www.example.com as sioimport numpy #matFile reads matFile ='matlabdata. mat' datas = sio. loadmat (matFile)#loads the data in matFile #assume matlabdatamatlabdata = datas <$'matlabdata ']#matFile writes save_matFile =' save_matlabdata. mat 'save_matlabdata = np. array ([1,2,3,4,5]) sio. savemat (save_matFile,{'array': save_matlabdata}) 2. matlab v7.3 files read

If matlab saves data with '-v7.3', the scipy.io.loadmat function loads the data with an error:

File "/usr/local/lib/python2.7/dist-packages/scipy/io/matlab/mio.py", line 64, in mat_reader_factory

raise NotImplementedError('Please use HDF reader for matlab v7.3 files')

NotImplementedError: Please use HDF reader for matlab v7.3 files

This can be done by:

import h6pywith h6py.File('matlabdata.mat', 'r') as f: f. keys ()#matlabdata. mat variable name datas = h6py.File ('matlabdata. mat')<$'matlabdata']. value The above is about "python save large. mat data file error how to solve" The content of this article, I believe everyone has a certain understanding, I hope the content shared by Xiaobian is helpful to everyone, if you want to know more related knowledge content, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.