Shulou(Shulou.com)06/01 Report--

After a period of continuous overtime, the performance test project that lasted about 4 months is finally coming to an end. Now think about it, there are all kinds of ups and downs.

As a full-time performance tester for this project, I was lucky that I and other functional testers entered at the same time. After entering, the functional testers were busy familiarizing themselves with the requirements and writing test cases. While I was familiar with the requirements, I wrote performance test plans, test plans and test cases. By the way, I was busy following the functional test team to learn how to use informatica and how to test ETL. After all, this part has not been touched before, by the way, to help the functional test team do some work, so, a month later, the performance test equipment is in place, began to install the environment, a total of four machines, friends who have built the environment all know, the installation manual is mainly to provide ideas, want to install smoothly according to the installation manual it is almost impossible, encountered a lot of problems, through Baidu, help others It took about 8 working days to build the environment, followed by the development of the test script. This process was smooth, and the unsmooth situation occurred during the test execution. When the test execution was multi-user concurrency, it was found that as long as more than 20 users logged in, the database CPU utilization rate would reach about 90% (32C 64G machine was used at that time). Could 32C 64G machine not support the concurrency of 20 virtual users? About 20 functional points have been tested like this. If this problem is not solved, the following mixed scenarios, stability testing, surge testing, fatigue testing and capacity testing cannot be carried out. At this time, there are only about two weeks to go online. The test team is in a hurry. While repairing the defects, the development team also sent someone to support the performance testing, and the development team also checked whether they had been indexed during the whole process. Also for these function points to rebuild the index, the quality manager of the bank is also very anxious, although everyone is very anxious, but the problem has not been solved, the bank invited the data center database maintenance personnel, invited the first female warrior to do a simple analysis, adjust the SGA from the previous 12G to 16G after busy with their own business, let's re-test, say that if there are any questions, type the report After testing, we found that this problem was still there. When we worked overtime the next night, we asked a data center person to make a diagnosis. The result of the diagnosis was that oralce enabled many processes. Compared with the same systems of other concurrent users, other systems did not have so many processes. As for why the utilization rate of CPU is so high, there is still no answer for the time being. I inquired about all kinds of information on the Internet (at that time, I couldn't work at the customer site during the day, so I had to come back at night to query, record it, and check whether there were similar problems in the test environment the next day). I learned whether oracle's shared server mode or proprietary server mode, as well as the viewing of some configuration files mentioned by some people on the Internet, still did not solve the problem. At that time, it was suspected that there might be a problem with the installation or the configuration file, because the idea at that time was that the people who hired the two data centers did not diagnose the problem, and it was not a general problem, nor was it a minor problem (now I think about it, I admired the technology master too much at that time). Finally, I looked at it online with the mentality of trying, and finally solved the problem through such a sql statement.

Select sql_text,spid,v$session.program,process from vsecretsqlarea where v$sqlarea.address=v$session.sql_address and v$sqlarea.hash_value=v$session.sql_hash_value and v$session.paddr=v$process.addr and v$process.spid in vaccounsessionje vandalism process where v$sqlarea.address=v$session.sql_address and v$sqlarea.hash_value=v$session.sql_hash_value and v$session.paddr=v$process.addr and v$process.spid in (PID)

First, use topas to find the process with high CPU consumption, then get the ID of the process, throw the process id at the PID process of the above sql statement, execute the sql statement, and finally catch a sql statement, analyze the sql statement, analyze the execution plan of the sql statement, and find that the sql statement is scanned from four tables at the same time, so what is the purpose of this statement? Why is every function point used in the test? Is it the platform framework? It's not like that, huh?

After logging in to the system, it is found that there are many kinds of prompts / to-do items on the home page (similar to the to-do matters in which the individual loan system prompts the customer's loan to expire or overdue). Now everyone suddenly realized why there is a similar problem at every function point. As long as you log in, the system will automatically query. When multiple users are concurrent, the improper execution plan of the sql sentence leads to the consumption of higher CPU. This problem is finally solved, very happy, this is also the first performance problem to solve on their own, in the past just to assist in positioning, the system went online as scheduled, is it over? No

First: why does the developer say clearly that the index is still the problem after adding the index? personally, I think it is mainly because the project cycle is relatively tight, the development process is not strictly controlled, and according to the development team, the index should be rebuilt later. However, performance testing requires performance testing. They add indexes to most function point tables and ignore the home page reminder / to-do tables.

Second: why didn't people in the data center check the index? I believe in developers too much, because most of these developers have 10 years of work experience, so we believe in what developers say.

Third: why did the people in the data center adjust the SGA directly instead of typing the AWR or statspack report directly? Maybe there are other reasons why they are too busy, we don't know.

Fourth: why did the second big shot in the data center find that the oracle database started so many processes while another system with similar concurrent users did not? our system used a proprietary server, while the other system used a shared server.

Through this question, I would like to say that the first big family should not worship the technology bulls so much that they despise themselves, and second, do not 100% believe the words of the development team or other people, although they are very experienced and you can believe them on the surface, but don't forget that the truth comes out in practice. Third, find problems, be sure to monitor, report and speak, after all, there is a difference between inference and speculation.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.