Shulou(Shulou.com)06/03 Report--

I haven't updated my blog for a long time. Recently, I have done some stress tests (real-time traffic import and offline playback), intranet sources (including all centos and EPEL), homemade RPM packages, and so on. Personally, I have a simple understanding. If you are interested in the above, leave a message and tell me that it will be updated selectively later. I believe it is more useful for all of you.

This article uses a short space to introduce the traffic import of Tcpcopy, and will update another article on the use of offline mode.

A perfect website and system must undergo a complete stress test before going online, which is the basic quality assurance. In the course of testing, we have used AB and Loadrunner are not ideal, unable to accurately restore user requests. Later, we used NetEase's open source Tcpcopy to restore real traffic, as well as offline traffic playback on demand, which played a vital role in the launch of new business.

Brief introduction:

Tcpcopy is a request replication (all tcp-based packets) tool that imports online traffic into the test system.

It has been used in NetEase's advertising system, urs system, nginx hmux protocol and other systems to avoid a lot of problems brought by online.

Now this tool has been widely used in major Internet companies. Wang Bo, the ancestor of tcpcopy (@ wbo65), was the one who made the initial exploration in this area. (designed and implemented in 2009, only more than 300 lines of code supported the initial development of NetEase's advertising delivery system, with zero errors and solving hundreds of problems before launch.) of course, the scope of application of this simplest version is very limited. Wang Bin (@ wangbin579) carried out in-depth transformation of this architecture at the end of 2010, expanding to more than 1000 lines of code. The latest version 1.0.0.

Features:

1) distributed stress testing tool, which uses online data to test the pressure that the system can withstand (far more than ab stress test).

Try the tool is much more real), you can also find some bug in advance.

2) ordinary online test, you can find whether the new system is stable, find many problems that will occur in the process of launching in advance, and get out of the way.

The sender is confident to go online.

3) the function of traffic magnification, which can use a variety of means to construct unlimited online pressure to meet the stress testing requirements of small and medium-sized websites.

Architecture:

The architecture of version 1.0.0 is as follows

The above architecture, that is, the latest architecture, is designed for the purpose of extreme testing, offload the work of intercept from the test server (test server) and put it on another independent secondary server (assistant server, in principle, an idle server on the same network segment must be used as a secondary server) to intercept response packets. Moreover, the original work of capturing response packets from the IP layer to the data link layer has been transferred to capture response packets from the data link layer. These changes have greatly reduced all kinds of interference to the test machine (except for routing settings, but have no impact), and greatly expanded the ability to capture response packets. Of course, this kind of test is more real.

Deployment:

This environment prepares three machines, assuming that we need to import 80 ports of Apache traffic, and the Apache installation process is omitted. If it is difficult to understand the above picture, I have drawn a concise sketch for you to understand.

1. Official node 1.1.1.1 install httpd tcpcopy

Second, test node 1.1.1.2 install httpd

Third, install intercept on the secondary node 1.1.1.3

Install tcpcopycd / tmpwget https://github.com/session-replay-tools/tcpcopy/archive/1.0.0.tar.gztar-zxvf 1.0.0.tar.gz./configure-- prefix=/usr/local/tcpcopymakemake install vi / etc/profile.d/tcpcopy.sh# join the following line export PATH=$PATH:/usr/local/tcpcopy/sbinsource / etc/profile.d/tcpcopy.sh install interceptyum install libpcap-devel cd / tmpwget https://github.com/session-replay-tools/intercept / archive/1.0.0.tar.gztar-zxvf 1.0.0.tar.gz./configure-- prefix=/usr/local/interceptmakemake install vi / etc/profile.d/intercept.sh# join the following line to run in the export PATH=$PATH:/usr/local/intercept/sbinsource / etc/profile.d/intercept.sh# formal environment Convert the IP of external visitors to 192.168.10 gw 24, which makes it easy for the test machine to set up routing tcpcopy-x 80-1.1.1.2 net 80-s 1.1.1.3-c 192.168.10.0 gw 255.255.255.0 gw 1.1.1.run in the test environment, intercept 80 requests and discard intercept-I eth0-F 'tcp and src port 80d

Effect:

When accessing the real node with a browser, check to see if the test node access_log scrolls.

Summary:

Thanks again to Tcpcopy's open source author (@ wangbin579) for his selfless dedication. If you have any questions, please leave a message and reply. I hope I can help you.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.