Shulou(Shulou.com)06/03 Report--

This article is mainly about "how to understand interface idempotency". Interested friends may wish to take a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "how to understand interface idempotency".

What is interface idempotency?

First, let's look at the concept of idempotency:

Idempotency is originally a mathematical concept, which can be understood when it is used on an interface: if the same interface sends out the same request many times, the operation must be performed only once. When an exception occurs in the calling interface and repeated attempts, it will always cause losses that the system cannot bear, so this phenomenon must be prevented.

For example, if the idempotence of the interface is not implemented, there will be serious consequences: payment interface, repeated payment will lead to multiple deductions; order interface, the same order may be created multiple times.

Why does the problem of interface idempotency arise?

So, under what circumstances will the problem of interface idempotency arise?

Network fluctuations, which may cause duplicate requests

If the user repeats the operation, the user may unintentionally trigger multiple order transactions during the operation, or even deliberately trigger multiple transaction applications without a response.

Failure or timeout retry mechanisms are used (Nginx retry, RPC retry or business layer retry, etc.)

Page refresh repeatedly

Use the browser back button to repeat the previous action, causing the form to be submitted repeatedly

Repeatedly submit the form using the browser history

Duplicate HTTP request from browser

Scheduled tasks are executed repeatedly

User double-click the submit button

How to ensure the idempotency of the interface?

So the most important thing is, how to ensure the idempotency of the interface?

The solution is divided into two directions, one is that the client prevents repeated calls, and the other is that the server verifies. Of course, it is not absolutely reliable for the client to prevent repeated commits, and the advantage is that it is relatively simple to implement.

The button can only be operated once

Generally, the button is grayed out or loding status after submission, to eliminate the duplicate records caused by repeated clicks, such as adding operations, resulting in two records due to two clicks.

Token mechanism

Repeat submission is allowed functionally, but it does not cause side effects. For example, click n times to generate only one record. The specific implementation is to apply for a token when you enter the page, and then all subsequent requests are accompanied by this token, and the backend avoids duplicate requests according to token.

Use Post/Redirect/Get mode

Perform page redirection after submission, which is the so-called Post-Redirect-Get (PRG) mode, which simply means that when the user submits the form, he jumps to a redirected information page, which avoids the repeated submission caused by the user pressing F5 refresh, and there is no warning of repeated submission of the browser form, and it can also eliminate the problem that pressing the browser forward and backward leads to the same repeated submission.

Store special signs in session

On the server side, generate a unique identifier and store it in session. At the same time, the front end obtains the value of this identifier and writes it into the concealment of the form, which is used for users to enter information and click to submit. On the server side, get the value of hidden fields in the form. Compared with the unique identifier in session, equality means that the request is submitted for the first time, and then the unique identifier in session is removed. If it is not equal, it means that it is a duplicate submission and is no longer processed.

Use unique indexes to prevent new dirty data

Using the unique index mechanism of the database, when the data is duplicated, inserting the database will throw an exception to ensure that there will be no dirty data.

Optimistic lock

If you update the existing data, you can update it with locks, or you can use optimistic locks when designing the table structure, and use version to do optimistic locks, which can ensure both execution efficiency and idempotency. The version version of optimistic locks should be self-increasing in updating business data.

Update table set version = version + 1 where id = # {id} and version = # {version}

Example: when there is a duplicate request, the first request will get the version version number of the current product, and the version will be 1. Immediately after the first request has not updated the version of the product, the version obtained by the second request is still 1. In this case, the first request operation update will be updated with version as a condition and the product version will become 2. When the second request was made to manipulate the update, it was obvious that the version inconsistency caused the update to fail.

Select + insert or update or delete

The scheme is to query before operation, meet the requirements and then insert. This scheme can solve the idempotent problem in a system without concurrency. When a single JVM has concurrency, it can use JVM to add locks to ensure idempotency. In a distributed environment, it can not guarantee idempotency, and can be guaranteed by using distribution.

Distributed lock

If the distribution is a system, it is difficult to build a globally unique index, for example, the unique field cannot be determined. At this time, a distributed lock can be introduced. Through a third-party system (redis or zookeeper), the data is inserted or updated in the business system, the distributed lock is acquired, and then the operation is done, and then the lock is released. In fact, the idea of multithreaded concurrent locks is introduced into multiple systems. That is, it has to be solved in the distributed system. Key points: a long process requires that it cannot be executed concurrently. Distributed locks can be acquired according to a certain flag (user ID+ suffix, etc.) before the process is executed. Other processes will fail when they are executed, that is, only one of the processes can be executed successfully at the same time. After the execution is completed, release the distributed locks (distributed locks are provided by third-party systems).

Idempotent state machine

In the business related to the design document, or the business related to the task, it will definitely involve the state machine (state change diagram), that is, there is a state on the business document, and the state will change under different circumstances. in general, there is a finite state machine, at this time, if the state machine is already in the next state, there is a change of the previous state, which can not be changed in theory. The idempotency of the finite state machine is guaranteed. Note: orders and other documents business, there is a long state flow, we must have a deep understanding of the state machine, which is of great help to improve the design ability of the business system.

Anti-weight meter

Take payment as an example: use a unique primary key to do the unique index of the anti-repetition table, such as using the order number as the unique index of the anti-repetition table, and each request inserts a piece of data into the anti-repetition table according to the order number, indicating that the later business can be processed. After processing the business logic, delete the order number data in the anti-repetition table, and later if there is a duplicate request. The insertion will fail due to the unique index of the anti-repetition table, and the operation will fail directly until the first request returns the result. It can be seen that the function of the anti-repetition table is the locking function.

Note: it is best to judge the idempotence of the state machine first.

Buffer queue

All requests are quickly received and put into a buffer queue, and then asynchronous tasks are used to process the data in the queue to filter out repeated requests. the advantage of this solution is that synchronous processing is changed to asynchronous processing and high throughput. The disadvantage is that the request results can not be returned in time, and subsequent polling is needed to get the processing results.

Global unique number

For example, the source source + unique sequence number is passed to the backend, and the backend determines whether the request is duplicated. Only one request can be processed when the request is concurrent. Other concurrent requests either return duplicate requests or wait for the previous request to be executed.

At this point, I believe you have a deeper understanding of "how to understand interface idempotency". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.