Shulou(Shulou.com)05/31 Report--

This article will explain in detail about the design and implementation of Web vulnerability scanner, the content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

The graduation project has dug a huge hole for myself. although this idea has had a general model since I was a sophomore, there is still a big difference between the actual realization and the imagination, especially in the accuracy of loophole report. there are some difficult choices in terms of speed and speed, so I opened a new series to record the process of playing with myself.

At present, the mainstream vulnerability scanners are roughly divided into the following three categories.

Active type: directly initiate a scan request. (ps: draw the picture yourself. Don't blame me for being ugly. Hhhh)

Passive: use intermediate agents or other means to discover vulnerabilities.

Cloud scan: a scanner deployed in the cloud that users can scan through a browser.

What I intend to do here is the active type and the cloud type, the main body is the active type, the active type is done, it is not difficult to change to the cloud.

03 work flow of scanner

First of all, as an automated testing tool, we have to figure out the difference between scanners and manual testing.

The process of manual penetration testing is as follows

-Information collection

-vulnerability discovery

-verify vulnerabilities

-leverage methods (EXP or POC)

-write test reports

Flow of the scanner

-Information collection

-Discover vulnerabilities (vulnerability verification)

-generate report

Artificial penetration testing requires signing all kinds of contracts to get as much data and permissions as possible without breaking the functionality of the business. But the scanner is different. You only need to click to verify the vulnerability, and there is no need for subsequent vulnerability exploitation.

Https://shell01.top/2018/11/11/web-scan-01/-Information gathering

Information collection is a crucial step in both artificial penetration testing and scanners. If there is no information collection, it will not pave the way for the next step of finding loopholes. The more information is available for the whole penetration testing or scanning process, the better.

In the WEB scan, the information we probably need is as follows

IP information: the ip here includes port opening information, c segment information, and so on.

Sub-domain name: enterprises generally put all kinds of business under the second-level domain name. For example, the address of Baidu's online disk business is: pan.baidu.com

Fingerprint information: knowing the fingerprint information of the target, you can find the corresponding nday to realize the attack.

Sensitive information: sensitive directories, backup files, unauthorized backend, mailboxes, databases, etc.

Hyperlink: when the crawler crawls the target site, it should be stored in the database to facilitate subsequent processing.

After the information is collected, we need to find the loopholes in the collected content, and after the loopholes are found, we also need to verify the vulnerabilities to avoid a large number of false positives and improve the accuracy.

Here are the main ways to do it

Port: identify port service, carry out corresponding blasting, 0day test

URL: identify parameters and perform tests such as sql injection

Cms: fingerprint recognition. Enter the database to match the corresponding poc

Http protocol: change all kinds of parameters and fuzz

Reporting stage

Generate reports in html or execl format

04 how to design

We have to follow some principles in the design of the scanner, otherwise it is difficult to maintain and add or subtract modules, which is not what we want. 1, enough independence. If the modules influence each other too much, it is obviously not what we need. 2, single responsibility principle if a module has too many functions, then it is likely that one module will fail and the rest will fail, so one module and one function is very necessary. 3, high concurrency design idea scanner if can not do high concurrency, I think no one wants to wait for a long scanning time. So based on the above information, we can determine the functional module of our vulnerability scanner. A crawler module: responsible for crawling the visible url of the target site, processing the corresponding storage, and then hierarchical processing, for example, it can be submitted to the sensitive information module to match the information, or submitted to the vulnerability detection module for detection. As the main eye of missing scan, the crawler module must be very strong, so it will be a problem in the future implementation of the crawler module. Second domain name detection module: query the sub-domain name, including second-level, third-level and multi-level. The main ways to achieve here are blasting, dns, search engine. Three-port scanning module: the real ip blasting server port is obtained by various methods, and the results are submitted to the fingerprint identification module for identification and blasting module for corresponding service blasting. Four fingerprint identification module: built-in a large number of fingerprint information, unexpectedly can expand the nature, carry on the corresponding fingerprint identification, and submit the identification results to the vulnerability detection module for corresponding nday query. Fifth, sensitive information module: has a large number of sensitive information catalogue, carries on the blasting operation, or obtains the corresponding data from the crawler to match. Six blasting module: built-in all kinds of service blasting operation. Seven vulnerability detection module: divided into conventional vulnerability detection, and 0day/nday detection, how to achieve efficient detection, is another problem. Eighth, generate report module: output the results into a scanning report. 9 "main control module":

Carry on the safe and controllable scheduling of each module to operate with high efficiency.

On the design and implementation of Web vulnerability scanner is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.