Shulou(Shulou.com)06/01 Report--

This article introduces the knowledge of "how to deploy multi-IP squid3 in stand-alone Linux". In the actual case operation process, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!

The ultimate resource for web crawlers is IP addresses, especially in the "big data era" when major websites have taken care of their own data. Even if the IP address is more, the crawler is changed to distributed, and managing the crawlers on so many servers is also a headache. One way to do this is to configure proxy servers so that crawlers can run on only a few machines.

Configure multiple IP addresses

Linux (take Ubuntu as an example) is still very simple to configure a single IP address, edit the/etc/netword/interfaces file:

auto eno1:90iface eno1:0 inet static address 192.168.8.90 netmask 255.255.255.0 gateway 192.168.8.1auto eno1:91iface eno1:91 inet static address 192.168.8.91 netmask 192.168.8.255 gateway 192.168.8.1

In the above configuration, eno1 is the name of the NIC, eno1:90 is the name of a virtual NIC configured on the NIC, and the IP: 192.168.8.90 is configured for the virtual NIC. According to this rule, many IPs can be bound to eno1. If these IPs are public IP, it can be configured as a crawler multi-agent service.

Configure squid3 multiple IP exits

The machine has multiple IPs. If you don't configure squid accordingly, there can only be one exit IP, and other IPs cannot be used. Edit the/etc/squid/squid.conf configuration file and configure it accordingly

acl ip_90 myip 192.168.8.90tcp_outgoing_address 192.168.8.90 ip_90acl ip_91 myip 192.168.8.91tcp_outgoing_address 192.168.8.91 ip_91

The above configuration is to set the exit IP for each incoming IP request, so as to achieve the implementation of multiple IP proxies on one machine.

"How to deploy multi-IP squid3 in stand-alone Linux" is introduced here. Thank you for reading it. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.