Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to use the GE language to achieve a HTTP agent". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to use the GE language to achieve a HTTP agent".

Here we mainly talk about the tunnel connection established by the CONNECT method in the HTTP/1.1 protocol and the realization of HTTP Proxy. The advantage of this kind of proxy is that it does not need to know the data requested by the client, but only needs to forward it intact. It is very convenient to deal with the request of HTTPS, and the proxy can be implemented without parsing his content.

Start agent snooping

To do a HTTP Proxy, we need to start a server and listen on a port to receive requests from the client. Golang provides us with a powerful net package for us to use, and it is very convenient for us to start a proxy server to listen.

L, err: = net.Listen ("tcp", ": 8080") if err! = nil {

Log.Panic (err)

}

As for the above agents, we have implemented a server that listens on port 8080. We do not write an ip address here, so we listen on all ip addresses by default. If you only want it to work locally, you can use 127.0.0.1Drex8080 so that the machine cannot access your proxy server.

Listen to receive agent requests

After starting the proxy server, we can begin to be unable to accept the proxy request, and only with the request can we do further processing.

For {

Client, err: = l.Accept () if err! = nil {

Log.Panic (err)

} go handleClientRequest (client)

}

The Accept method of the Listener interface will accept the connection data sent by the client. This is a blocking method. If the client does not send the connection data, it just blocks and waits. The received connection data will be immediately handed over to the handleClientRequest method for processing. Here, the purpose of opening a goroutine with a go keyword is not to block the reception of the client, and the proxy server can immediately receive the next connection request.

Parse the request to get the IP and port to be accessed

With the client's proxy request, we also have to extract the IP and port of the remote host that the client wants to access from the request, so that our proxy server can establish a connection with the remote host and proxy forwarding.

The header information of the HTTP protocol contains the host name (IP) and port information we need, and it is in clear text. The protocol is very standard, similar to:

CONNECT www.google.com:443 HTTP/1.1

Host: www.google.com:443

Proxy-Connection: keep-alive

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10 / 12 / 0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36

You can see that what we need on the first line, the first line of information is separated by spaces, the first part of CONNECT is the request method, here is CONNECT, in addition to GET,POST, etc., are the standard methods of the HTTP protocol.

The second part is the request of URL,https. Only the request of host and port,http is a completed url. We will understand it by looking at an example later.

The third part is the protocol and version of HTTP, which we don't need to pay too much attention to.

The above is a request from https. Let's take a look at http's:

GET http://www.flysnow.org/ HTTP/1.1

Host: www.flysnow.org

Proxy-Connection: keep-alive

Upgrade-Insecure-Requests: 1

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10 / 12 / 0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36

As you can see for htt, there is no port number (default is 80); there is more schame-http:// than https.

With the analysis, we can get the requested url and method information from the HTTP header information.

Var b [1024] byte

N, err: = client.Read (b [:]) if err! = nil {

Log.Println (err) return

} var method, host, address string

Fmt.Sscanf (string (b [: bytes.IndexByte (b [:],'\ n')]), "% s% s", & method, & host)

HostPortURL, err: = url.Parse (host) if err! = nil {

Log.Println (err) return

}

Then we need to parse the url further to get the remote server information we need.

If hostPortURL.Opaque = "443" {/ / https access

Address = hostPortURL.Scheme + ": 443"

} else {/ / http access

If strings.Index (hostPortURL.Host, ":") =-1 {/ / host without port, default is 80

Address = hostPortURL.Host + ": 80"

} else {

Address = hostPortURL.Host

}

}

This completes the acquisition of the information to be requested from the server, which may be in the following formats

Ip:port

Hostname:port

Domainname:port

It could be ip (v4orv6), hostname (intranet), or domain name (dns resolution).

The proxy server establishes a connection with the remote server

With the information of the remote server, you can dial up to establish a connection, and only with the connection can you communicate.

/ / after obtaining the requested host and port, start dialing

Server, err: = net.Dial ("tcp", address) if err! = nil {

Log.Println (err) return

}

Data forwarding

After the dialing is successful, the data proxy transmission can be performed.

If method = = "CONNECT" {

Fmt.Fprint (client, "HTTP/1.1 200 Connection established\ r\ n")

} else {

Server.Write (b [: n])

} / / forward

Go io.Copy (server, client)

Io.Copy (client, server)

There is a separate response to the CONNECT method, where the client says to establish a connection, and the proxy server responds to it before it can request access like HTTP.

Complete code

At this point, all of our proxy servers have been developed. Here is the complete source code:

Package mainimport ("bytes"

"fmt"

"io"

"log"

"net"

"net/url"

"strings") func main () {

Log.SetFlags (log.LstdFlags | log.Lshortfile)

L, err: = net.Listen ("tcp", ": 8081") if err! = nil {

Log.Panic (err)

} for {

Client, err: = l.Accept () if err! = nil {

Log.Panic (err)

} go handleClientRequest (client)

}

} func handleClientRequest (client net.Conn) {if client = = nil {return

} defer client.Close () var b [1024] byte

N, err: = client.Read (b [:]) if err! = nil {

Log.Println (err) return

} var method, host, address string

Fmt.Sscanf (string (b [: bytes.IndexByte (b [:],'\ n')]), "% s% s", & method, & host)

HostPortURL, err: = url.Parse (host) if err! = nil {

Log.Println (err) return

} if hostPortURL.Opaque = = "443" {/ / https access

Address = hostPortURL.Scheme + ": 443"

} else {/ / http access

If strings.Index (hostPortURL.Host, ":") =-1 {/ / host without port, default is 80

Address = hostPortURL.Host + ": 80"

} else {

Address = hostPortURL.Host

}

} / / after getting the requested host and port, start dialing

Server, err: = net.Dial ("tcp", address) if err! = nil {

Log.Println (err) return

} if method = = "CONNECT" {

Fmt.Fprint (client, "HTTP/1.1 200 Connection established\ r\ n")

} else {

Server.Write (b [: n])

} / / forward

Go io.Copy (server, client)

Io.Copy (client, server)

}

Thank you for your reading, the above is the content of "how to use GE language to achieve a HTTP agent". After the study of this article, I believe you have a deeper understanding of how to achieve a HTTP agent in GE language, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.