Shulou(Shulou.com)06/03 Report--

Editor to share with you how to deal with the problems arising from the introduction of php+ajax into big data, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

Let's talk about the problems we encounter from first to last.

Question 1 according to my original idea, upload the file first and then read the file. Here is the problem, when the file is larger, upload is slower, resulting in the operation seen by the customer has been waiting for the state, impersonal.

How to deal with it: this is what I do. God has a better way to introduce you. I upload the file first, then save the file to a specific folder called import, and then return a file name. This ensures that the file is uploaded successfully. And I can use js to give the client a hint when he returns his name. Then ajax asks php to read the file and insert it into the database. But here's the problem.

Question 2 when I use ajax to ask php to read the file and insert it into the database, I encounter a problem, that is, the ajax request is always broken when 1min. I thought, this should be the reason for the maximum execution time of php max_execution_time, as a result, I changed it to 300 seconds. Still, I think it will be the maximum get time max_input_time of apache. I will add an ini_set result in the code, check max_input_time with ini_get, invalid setting with ini_set, or 60 seconds. I looked up a lot of information on the Internet, but I still don't know why. If there is anything that God knows, please give me a reply. Thank you, rookie. There's nothing I can do about it. I have to go to the server and change the php.ini configuration. The manager said not to change it, but secretly changed it for the sake of testing-- and finally changed it back. After the modification, the test still doesn't work. Or perform a timeout in one minute. I'm really wondering. I don't know why. Ask for advice. I can't help it.

This method will not work, for a 5m file can only be read by branches. Then there is a modification of the code, branch reading is operated like this, first ajax request, and then read 2000 pieces of data each time, and then process the 2000 pieces of data and insert them into the database (the article finally introduces a useful branch reading function). Then each time the ajax is executed, it returns a status character and the number of lines read this time, and then read on next time. Until the final reading is finished. There's also a problem: when I'm checking every row of data, here's the thing: I loop through what I get, and then check if each row exists, and when I determine if $count is greater than 0, when it does, I use continue to execute the next loop. But when I import 10000 entries, I always report an internal error in the server when I import 8000 entries. Very boring, do not understand what to ask, the result can only be replaced by if else. Wonder. A small reminder: when inserting a database, do not insert one entry at a time. It is best to use inset into aaa (`xx`, `xxx`) values ('111'), ('222'). It will be much faster.

Line number reading function, SplFileObject this class library is really easy to use recommended. If you know my problem, ask God for advice.

The copy code is as follows:

Function getFileLines ($filename, $startLine, $endLine, $method = 'rb') {

$content = array ()

$filename = DATA_PATH.DS.'import'. DS. $filename

$count = $endLine-$startLine

$fp = new SplFileObject ($filename, $method)

$fp- > seek ($startLine); / / go to line N, and the seek method parameters are counted from 0

For ($ii = 0; $ii current (); / / current () gets the current line content

$fp- > next (); / / next line

}

Return array_filter ($content); / / array_filter filtering: false,null,''

}

The above is all the contents of this article entitled "how to deal with the problems arising from the introduction of php+ajax into big data". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.