Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "analyzing Web application security HTTP". Many people will encounter such a dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Overview

As mentioned earlier, HTTP follows the request / response model, where the client connected to the server makes a request and the server responds to it.

The HTTP message (request or response) consists of several parts:

Request line

Request header

Request body

Part: the request line, which describes the type of request, the resource to be accessed, and the version of HTTP used.

GET / players/lebron-james HTTP/1.1

GET indicates that the request type is GET,/players/lebron-james, which is the resource to be accessed, and the latter part of the line indicates that the HTTP1.1 version is used.

The second part: the request header, immediately after the request line, is used to describe the additional letter to be used by the server.

GET / players/lebron-james HTTP/1.1 Host: nba.com Accept: * / * Coolness: 9000

For example, in this request, the client has attached three additional headers to the request: Host,Accept and Coolness.

Wait a minute. What is Coolness?

Headers do not have to use specific reserved names, but it is generally recommended that you rely on standardized names from the HTTP specification: the more you deviate from the standard, the less you will be understood by the other party in the exchange.

For example, Cache-Control is a header file that defines whether the response is cacheable: most proxies and reverse proxies understand it in full accordance with the HTTP specification. If you rename the Cache-Control header to Awesome-Cache-Control, proxies will no longer know how to cache responses because they are not built according to the specification you just proposed.

Sometimes, however, it may make sense to include a "custom" title in the message, because you may want to add metadata that is not actually part of the HTTP specification: the server can decide to include technical information in its response so that the client can simultaneously execute the request and obtain important information about the server status of the reply:

... X-Cpu-Usage: 40% X-Memory-Available: 1%.

When using custom headers, always add a key to them so that they do not conflict with other headers that may become standards in the future: historically, this worked until everyone started using the "non-standard" X prefix in reverse. This has become the norm. X-Forwarded-For and X-Forwarded-Proto headers are examples of custom headers that are widely used and understood by load balancers and agents, even if they are not part of the HTTP standard.

If you need to add your own custom headers, you usually use an automatically generated prefix, such as Acme-Custom-Header headers or A-Custom-Header headers.

After the title, a request may contain a body separated from the title by a blank line:

POST / players/lebron-james/comments HTTP/1.1 Host: nba.com Accept: * / * Coolness: 9000 Best Player Ever

Our request is completed: the first line (location and protocol information), the request header, and the request body. Note that the request body is completely optional, and in most cases it is only used when we want to send data to the server-- which is why the above example uses POST.

The response is not much different:

HTTP/1.1 200 OK Content-Type: application/json Cache-Control: private, max-age=3600 {"name": "Lebron James", "birthplace": "Akron, Ohio",...}

One piece of information published by the response is the version of the protocol it uses and the status of the response. The same goes for the request header, adding a newline character after the body, if necessary.

As mentioned earlier, the protocol has been revised several times and some features have been added over time (new header files, status codes, and so on), but the underlying structure has not changed much (request line, request header, and body). What really changes is how the client and server exchange these messages-- let's take a closer look.

HTTP vs HTTPS vs H2

HTTP has undergone two significant semantic changes: HTTP/1.0 and HTTP/1.1.

So, "where are HTTPS and HTTP2?"

HTTPS and HTTP2 (H2 for short) are more technical changes because they introduce new ways to deliver messages over the Internet without seriously affecting the semantics of the protocol.

HTTPS is a "secure" extension of HTTP that involves establishing a public secret between the client and the server, ensuring that we communicate with the right party, and encrypting messages exchanged with the public secret (more on this later). The goal of HTTPS is to improve the security of the protocol HTTP, while the goal of H2 is to bring it faster.

H2 uses binary instead of plain text messages, supports multiplexing, and uses HPACK algorithms to compress headers. To make a long story short, H2 is a performance improvement for HTTP/1.1.

Site owners are reluctant to switch to HTTPS because it involves extra round trips between the client and the server (as mentioned above, a common secret needs to be established between the two parties), thus slowing down the user experience: by default, they have no excuse to use H2 encryption because features such as multiplexing and server push make its performance better than normal HTTP/1.1.

HTTPS

The goal of HTTPS is to allow clients and servers to communicate securely over TLS (Transport layer Security), and TLS is the successor to SSL (secure Sockets layer).

The problem TLS aims at is quite simple. You can use a simple analogy: your partner calls you at noon when you are in a meeting and asks you to tell them the password of your online bank account because they need to perform a bank transfer to ensure that your son's education expenses are paid on time. The important thing is that you tell them now, or your child may be shut out of school the next morning.

You now face two challenges:

Authentication: make sure you are really talking to your partner, because it is possible that others will pretend to be them

Encryption: communicate when colleagues are unable to understand and record passwords

This is exactly what HTTPS is trying to solve.

To verify who you are talking to, HTTPS uses a public key certificate, which is just a certificate that declares the identity behind a particular server: when you connect to an IP address through HTTPS, the server behind that address will provide you with its certificate to verify its identity. Going back to our analogy, it's probably just that you ask your partner to spell their social security number. Once you have verified the correctness of the numbers, you get an extra level of trust.

But that doesn't stop hackers from learning the victim's social security number, stealing your partner's smartphone and calling you. How do we verify the identity of the caller?

Instead of directly asking your partner to spell their social security number, you call your mother (who lives right next door) and ask her to go to your apartment and make sure your partner spells their social security number. This adds an extra level of trust because you don't think your mother is a threat and rely on her to verify the identity of the caller.

In HTTPS terminology, your mother is called CA, short for Certificate Authority (Certificate Authority): CA's job is to verify the identity behind a particular server and issue a certificate with its own digital signature: this means that when I connect to a particular domain, I do not produce a certificate generated by the domain owner (called a self-signed certificate). Instead, it is issued by CA.

The authority's job is to ensure that they verify the identity behind the domain name and issue certificates accordingly: when you "order" a certificate (often referred to as a SSL certificate, even if you now use TLS instead), the authorities may call someone or ask you to change the DNS setting to verify that you can control the related domain. After the verification process is complete, it will issue a certificate, which you can then install on the Web server.

Clients such as browsers will connect to your server and get this certificate so that they can verify that it looks real: browsers have some "relationship" with CA because they track a list of trusted CA. To verify whether the certificate is really trustworthy. If the certificate is not signed by a trusted authority, the browser displays an informative warning to the user:

Ensuring that the communication between you and your partner is secure is half done: now that we have solved authentication (verifying the identity of the caller), we need to make sure that we can communicate securely without being eavesdropped by others in the process. As I mentioned, you are in a meeting and need to spell your online banking password. You need to find a way to encrypt your communication so that only you and your partner can understand your conversation.

You can do this by establishing a shared key between the two parties and encrypting the message with that key: for example, you can decide to use a variant of Caesar cipher based on the wedding date.

If both parties have a stable relationship, like you and your soul mate, this will be effective because they can create a key based on a common memory that no one else knows. However, browsers and servers cannot use the same mechanism because they do not know each other in advance.

Instead, there is a variant of the Diffie-Hellman key exchange protocol, which ensures that no pre-known parties establish a shared key and that others cannot "sniff" it. This requires a little bit of mathematical knowledge, which is an exercise for the reader.

Once the key is established, the client and server can communicate without fear that someone will intercept their messages. Even if hackers do this, they do not have the public key needed to decrypt the message.

HTTPS is everywhere

Still debating whether you should support HTTPS on your website? I have no good news: browsers have begun to push users away from sites that do not support HTTPS to "force" web developers to provide a fully encrypted browsing experience.

Behind the slogan "HTTPS is everywhere", browsers began to object to unencrypted connections-Google announced that it would mark HTTP sites as "insecure" from Chrome 68.

More worryingly for sites that don't use HTTPS, the "unsafe" tag turns red once users type anything into the page-a move that should encourage users to think twice before exchanging data with sites that don't support HTTPS.

Compare this with the appearance of a website running on HTTPS with a valid certificate:

In theory, the site is not necessarily secure, but in practice, it scares users away-as a matter of course. It makes sense to stick to unencrypted HTTP communication when H2 is not yet common, and there is little reason to do so today.

GET and POST

As we saw earlier, the HTTP request starts with a special request line:

First, the client tells the server what verbs it is using to execute the request: common HTTP verbs include GET,POST,PUT and DELETE, but the list can continue to use unusual (but still standard) verbs such as TRACE, OPTIONS, or HEAD.

In theory, no method is safer than others; in fact, it's not that simple.

GET requests usually have no body, so parameters are contained in URL (such as www.example.com/articles?article_id=1), while POST requests are usually used to send ("post") the included data.

Another difference is the side effect of these verbs: GET is an idempotent verb, meaning that no matter how many requests you send, you will not change the state of the web server. On the contrary, POST is not idempotent: for every request you send, you may be changing the status of the server (for example, consider issuing a new payment-now you may understand why the site requires you not to refresh the page transaction during execution).

Idempotency: it means that one and multiple requests for a resource should have the same side effect, that is, one visit and multiple visits will bring the same change to the resource.

To illustrate an important difference between these methods, we need to take a look at the web server logs, which you may already be familiar with:

192.168.99.1-[192.168.99.1]-- [29/Jul/2018:00:39:47 + 0000] "GET /? token=1234 HTTP/1.1" 200525 "-" Mozilla/5.0 (X11) Linux x86x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36 "404 0.002 [example-local] 172.17.0.8 KHTML 9090 525 0.002 192.168.99.1-[192.168.99.1]-[29/Jul/2018:00:40:47 + 0000]" GET / HTTP/1.1 "200525"Mozilla/5.0 (X11) Linux x86x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36 "393 0.004 [example-local] 172.17.0.8 AppleWebKit/537.36 9090 525 0.004 192.168.99.1-[192.168.99.1]-[29/Jul/2018:00:41:34 + 0000]" PUT / users HTTP/1.1 "20123" http://example.local/"Mozilla/5.0 (X11) Linux x86 / 64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36 "4878 0.016 [example-local] 172.17.0.8 Chrome/65.0.3325.181 Safari/537.36 9090 23 0.016 201

As you can see, the web server records the request path: this means that if you include sensitive data in the URL, it will be leaked by the web server and saved somewhere in your log-your key will appear in clear text, which we absolutely need to avoid. Suppose a hacker can access one of your old log files, which may contain credit card information, access tokens for private services, and so on: this would be a complete disaster.

The Web server does not record the HTTP header or body because the data to be saved is too large-which is why it is usually safer to send information through the request body rather than the URL. From this we can conclude that POST (and similarly, non-idempotent methods) is safer than GET, even if it is more about the way data is sent when using a particular verb than a particular verb itself is safer than other verbs: if you include sensitive information in the body of the GET request, then you will not encounter any more problems than when using POST, even if this method is considered unusual.

This is the end of the content of "analyzing Web Application Security HTTP". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.