Shulou(Shulou.com)06/02 Report--

What this article shares with you is about the steps of using python to achieve domain name asset monitoring. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

Application scenario

Domain name inventory monitoring is used to find out the port opening and closing of the server where the ip address of the domain name is located by entering a primary domain name. By doing such monitoring on a regular basis, you can help you know the overall exposure of your assets.

Prefix skills required

Simple use of python

Simple use of linux operating system

The principle of DNS

Tools

Pycharm Professional Edition

Xshell is used to deploy scripts

Step to get the corresponding ip address through the domain name

First of all, we know that there are many types of record values that can be set for a domain name, such as A record, AAAA record, SOA record, TXT record, and so on. The common record values in the security field correspond to their meanings as follows:

Code description AIPv4 address record AAAAIPv6 address record CNAME specification name record, an alias for a host name:

The domain name system will continue to try to find the new name MX email interaction record NS name server record PTR is most commonly used to run reverse DNS to find the starting TXT text record of SOA authoritative record

What we need to use is the A record (IPv6 is not widely used at the moment).

Obtain the IP address of a domain name through A record. The method to implement it with python is as follows:

Def dns_a (domain: str): dns_servers = '114.114.114.114.114.115.115223.5.5.5223.6.6180.76.76.76119.29.29119.29.29.29.29, 1.2.8210.2.4.8117.50.11.11, 52.80.66.66101.226.6218.30.118.6123.81.6140.207.198.68.8.8, 8.4.4, .9.9.9208.67.222.222208.67.220.220 '.split (' ') loop= asyncio.new_event_loop () resolver = aiodns.DNSResolver (loop=loop) resolver.nameservers = dns_servers try: record_a = loop.run_until_complete (resolver.query (domain,' A')) return record_a except: pass return []

In the code, a dns_server, that is, the dns query server, is randomly assigned to each query, and the results of different query servers may be different due to convergence. So you can consider to do better, here only one query, you can also change multiple dns servers to query multiple times, take union (union is all possible, the intersection is accurate but may be missed). Finally, this method can obtain the ip address corresponding to the domain name.

Asyncio and aiodns packages are used in this method, and the installation method is as follows:

Pip install-I https://pypi.tuna.tsinghua.edu.cn/simple/-- trusted-host pypi.tuna.tsinghua.edu.cn asyncio,aiodns

Let's take www.mengwa.store as an example to see what the result is:

You can see that the ip address exists, but you still need to extract it from the result. The methods are as follows:

Record_a = dns_a (domain) ips = [] if type (record_a) = = list:ips = sorted ([answer.host for answer in record_a])

What is stored in this ips is the sorted parsing record ip list.

Port scan for ip addr

The ip address is found through the steps above, and the next step is to scan the port of the ip address. But before scanning, we have one more step to do to determine whether the current ip machine is alive or not.

You may immediately think of the ping method, but in the field of security, many companies require the server to block the ICMP protocol, that is to say, the ping method may return a timeout, but the machine may not be turned on, just take the initiative to block the request for you.

Here, we use the-sPn parameter of nmap to scan, taking the ip address 47.119.137.0 corresponding to www.mengwa.store as an example:

Then we implement this logic in code, using the python-nmap package, and the installation steps are as follows:

Pip install-I https://pypi.tuna.tsinghua.edu.cn/simple/-- trusted-host pypi.tuna.tsinghua.edu.cn python-nmap

The code is as follows:

Def check_ip_reachable (ip: str): nm = nmap.PortScanner () xx = nm.scan (hosts=ip Arguments='-sP') #-sPn can also if 'scan' in xx: I = xx.__getitem__ (' scan') if ip in i: J = i.qualified getitemps _ (ip) if 'status' in j: status = j [' status'] if 'state' in status: State = status ['state'] print (ip State) return state print (ip, 'down') return' down'

The prerequisite for this step is that you have installed nmap on your computer.

Judging from this, if the ip machine is on, then we can scan its port.

The code is as follows:

# implement to scan the port and return the result. Using nmap to implement def port_scan (ip: str): second = check_ip_reachable (ip) if second is not None and second = = 'up': nm = nmap.PortScanner () xx = nm.scan (ip) print (nm.command_line ()) ans = [] if' scan' in xx: I = xx.__getitem__ ('scan') If ip in i: j = i.collect getitemps _ (ip) if 'tcp' in j: tcp = j [' tcp'] for port in tcp: ans.append (port) ans = sorted (ans) return ans else: write_to _ file ('output/' + time_now +' / ips.txt' Ip + 'unreachable\ n') return []

The nm.command_line () method prints out the currently executed command line. (in essence, this package is a command line call package, and then parse.)

Taking 47.119.137.0 as an example, the results are as follows:

The method here uses the common ports that nmap scans by default. If necessary, you can also add Ports parameters to the Scan method to customize the ports to be scanned, or even full ports.

Connect in series Implement domain name to port monitoring def handle_ip (ip: str): if not task_ip_set.__contains__ (ip): try: print ('ip:' + ip,' start port scanning') port_result = port_scan (ip) val =''print (' ip:', ip, 'discover port' Port_result) for i in port_result: val = val +'\ n' + ip + "-" + i.diagnostic stringing _ () write_to_file ('output/' + time_now +' / ip_ports.txt' Val) task_ip_set.add (ip) except Exception as e: print (e) def handle_domain (domain: str): if not task_domain_set.__contains__ (domain): print ('domain:' + domain) record_a = dns_a (domain) ips = [] if type (record_a) = list : ips = sorted ([answer.host for answer in record_a]) val =''for i in ips: val = val +'\ n' + domain + "- -" + i.roomstringing _ () write_to_file ('output/' + time_now +' / domain_ips.txt' Val) print (ips) for i in ips: handle_ip (I)

Task_domain_set and task_ip_set are used to remove the weight of the target.

So, are we done here?

No

Subdomain name discovery

If my main domain name asset has a lot of secondary domain names, and I don't know which subdomain names I have, then I need to do subdomain name discovery at this time. Then call the previous method on each subdomain to find the port open to its corresponding ip.

Generally speaking, there are three ways to discover subdomain names:

The first method is to resolve the subdomain name from the content by crawling the website where the main domain name is located.

The second is to get the results through external query sites, such as crt, virustotal, fofa, etc.

The third method obtains the result through dictionary explosion, and constantly polls the dns server to check whether it has the value of A record. If so, it indicates that the sub-domain name exists. If not, it is temporarily considered that the sub-domain name does not exist.

In the first method, a crawler needs to be implemented. After getting the content of the website, the sub-domain name is extracted by regular expression. The extraction method is as follows:

Def get_subdomains (value: str, domain: str): X = f'(? P [a-zA-Z0-9.] + {domain}) 'pattern = re.compile (x) y = pattern.findall (value) ans = set () for i in y: ans.add (I) print (ans) return ans

The second method, using interface calls, can be found in the relevant API interface.

The third method is the explosion of subdomain names. The core code is as follows

Answers = await self.explorvers.query (cur_domain,'A') # find A record # if A record has a value and no error is reported, the subdomain name exists

When you have completed the above steps and deduplicated them, you can use these subdomains as input for the previous steps. In this way, the scope of monitoring is greatly improved for a primary domain name that users need to monitor.

Please do not use it for the monitoring of non-existing domain names. The losses caused have nothing to do with me. The above test data are all my existing domain name and ip.

These are the steps to implement domain name asset monitoring with python. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.