Shulou(Shulou.com)06/02 Report--

This article mainly introduces how to use Redis to optimize interface performance in high concurrency scenarios, which has a certain reference value. Interested friends can refer to it. I hope you will learn a lot after reading this article. Let's take a look at it.

The content of this article

Using Redis to optimize Interface performance in High concurrency scenarios

Database optimistic lock

With the approach of double 12, all kinds of promotional activities are becoming more and more popular, such as second killing, coupon grabbing, group buying and so on.

The main scenarios involving high concurrency competing for the same resource are second kill and coupon grabbing.

Premise

Activity rules

The number of prizes is limited, such as 100

There is no limit to the number of participating users

Each user can only participate in one second kill.

Activity requirements

No more, no less, all 100 prizes should be given out.

One user can grab at most one prize.

On a first-come-first-served basis, first-come-first-served users will receive prizes.

Database implementation

Pessimistic lock performance is so poor that this article will not discuss it, but discuss the advantages and disadvantages of using optimistic locks to solve high concurrency problems.

Database structure

If you don't win the lottery, UserId is 0.RewardAt is NULL.

When winning the lottery, UserId is the winning user. ID,RewardAt is the winning time.

Optimistic lock implementation

Optimistic locks don't actually exist. Optimistic locks are done using a field of data. For example, the example in this article is implemented in UserId.

The implementation process is as follows:

1. Query the prize with a UserId of 0. If no prize is found, no prize will be found.

SELECT * FROM envelope WHERE user_id=0 LIMIT 1`

2. Update the user ID and winning time of the prize (assuming that the prize ID is 1, the winning user ID is 100, and the current time is 2019-10-29 12:00:00), the user_id=0 here is our optimistic lock.

UPDATE envelope SET user_id=100, reward_at='2019-10-29 12 WHERE user_id=0 AND id= 001`

3. Check the return value of the execution of the UPDATE statement. If 1 is returned, it proves that the prize is successful, otherwise it proves that the prize has been robbed by someone else.

Why add optimistic locks?

Under normal circumstances, it is no problem to get the prize and update the prize to the designated user. If you do not add user_id=0, the following problems will occur in high concurrency scenarios:

Two users queried a prize that did not win at the same time (concurrency problem occurred)

Update the winning user of the prize to user 1, only ID= prize ID

The above SQL execution is successful, and the number of affected lines is 1. In this case, the API returns user 1 winning the prize.

Next, the winning user will be updated to user 2, and the update condition is only ID= prize ID.

Because it is the same prize, the prize that has been given to user 1 will be redistributed to user 2, and the number of rows affected will be 1. The interface returns user 2 and wins the prize.

So the final result of the prize is distributed to user 2

User 1 will come to complain to the activist, because the lottery interface returns to user 1 to win the prize, but his prize is robbed, so the activist can only lose money.

Lottery process after adding optimistic lock

1. The condition for updating user 1 is id= red packet ID AND user_id=0. Since the red packet is not assigned to anyone at this time, user 1 updates successfully, and the API returns that user 1 wins the prize.

two。 When updating user 2, the update condition is id= red packet ID AND user_id=0. Since the red packet has been assigned to user 1, this condition will not update any records. The API returns user 2 winning the prize.

Advantages and disadvantages of optimistic lock

Advantages

Performance is OK because there is no lock

It won't oversend.

Shortcoming

Usually do not meet the "first-come-first-served" rule of activity, once concurrency occurs, there will be a non-winning situation, at this time there are prizes in the prize bank

Pressure test

The performance of the pressure test on MacBook Pro 2018 is as follows (HTTP server implemented by Golang, MySQL connection pool size 100jmeter pressure test):

500 concurrent 500 Total requests average response time 331ms delivery success 31 Throughput 458.7 Universe

Redis implementation

You can see that the contention ratio under the implementation of optimistic lock is too high, which is not the recommended implementation method. Let's optimize this second kill business through Redis.

Reasons for the High performance of Redis

Single thread saves thread switching overhead

Memory-based operations although persistence operations involve hard disk access, they are asynchronous and will not affect the business of Redis

IO multiplexing is used

Realization process

Write the code of the prizes in the database to the Redis queue before the event starts

Use lpop to pop up elements in the queue when the activity is in progress

If successful, use the update syntax to hand out the prize

UPDATE reward SET user_id= user ID,reward_at= current time WHERE code=' prize code'

If you fail to get the prize, there is no prize available at present, and you can be reminded that you have not won the prize.

In the case of Redis, concurrent access is guaranteed through Redis's lpop (), which is an atomic method that ensures that concurrency pops up one by one.

Pressure test

The performance of the stress test on MacBook Pro 2018 is as follows (HTTP server implemented by Golang, MySQL connection pool size 100Magi Jmeter pressure test sold on behalf of Redis connection pool)

500 concurrent 500 total requests average response time 48ms release success 100Throughput 497.0xps

Conclusion

You can see that the performance of Redis is stable, there will be no oversend, and the access latency is about 8 times less, and the throughput has not yet reached the bottleneck, you can see that Redis has a great performance improvement for high concurrency systems! The access cost is not high, so it is worth learning!

Experiment code / / main.gopackage mainimport ("fmt"github.com/go-redis/redis" _ "github.com/go-sql-driver/mysql"github.com/jinzhu/gorm"log"net/http"strconv"time") type Envelope struct {Id int `gorm: "primary_key" `Code string UserId int CreatedAt time.Time RewardAt * time.Time } func (Envelope) TableName () string {return "envelope"} func (p * Envelope) BeforeCreate () error {p.CreatedAt = time.Now () return nil} const (QueueEnvelope = "envelope" QueueUser = "user") var (db * gorm.DB redisClient * redis.Client) func init () {var err error db Err = gorm.Open ("mysql", "root:root@tcp (localhost:3306) / test?charset=utf8&parseTime=True&loc=Local") if err! = nil {log.Fatal (err)} if err = db.DB () .Ping () Err! = nil {log.Fatal (err)} db.DB (). SetMaxOpenConns (100) fmt.Println ("database connected. Pool size 10 ")} func init () {redisClient = redis.NewClient (& redis.Options {Addr:" localhost:6379 ", DB: 0, PoolSize: 100,}) if _, err: = redisClient.Ping (). Result (); err! = nil {log.Fatal (err)} fmt.Println (" redis connected. Pool size 100 ")} / / read Code write Queuefunc init () {envelopes: = make ([] Envelope, 0,100) if err: = db.Debug () .Where (" user_id=0 ") .Limit (100) .Find (& envelopes) .Error Err! = nil {log.Fatal (err)} if len (envelopes)! = 100 {log.Fatal ("less than 100 prizes")} for I: = range envelopes {if err: = redisClient.LPush (QueueEnvelope, redisClient.LPush [I] .Code). Err () Err! = nil {log.Fatal (err)}} fmt.Println ("load 100 envelopes")} func main () {http.HandleFunc ("/ envelope", func (w http.ResponseWriter, r * http.Request) {uid: = r.Header.Get ("x-user-id") if uid = "" {w.WriteHeader (401) _, _ = fmt.Fprint (w) "UnAuthorized") return} uidValue, err: = strconv.Atoi (uid) if err! = nil {w.WriteHeader (400) _, _ = fmt.Fprint (w, "Bad Request") return} / / detect whether the user has robbed if result, err: = redisClient.HIncrBy (QueueUser, uid, 1). Result () Err! = nil | | result! = 1 {w.WriteHeader (429) _, _ = fmt.Fprint (w, "Too Many Request") return} / / check whether code is in the queue, err: = redisClient.LPop (QueueEnvelope). Result () if err! = nil {w.WriteHeader _, _ = fmt.Fprint (w) "No Envelope") return} / / issue red packets envelope: = & Envelope {} err = db.Where ("code=?", code). Take (& envelope). Error if err = = gorm.ErrRecordNotFound {w.WriteHeader (200) _, _ = fmt.Fprint (w) "No Envelope") return} if err! = nil {w.WriteHeader _, _ = fmt.Fprint (w Err) return} now: = time.Now () envelope.UserId = uidValue envelope.RewardAt = & now rowsAffected: = db.Where ("user_id=0"). Save (& envelope) .RowsAffected / / add user_id=0 to verify whether Redis really solves the contention problem if rowsAffected = = 0 {fmt.Printf ("contention occurs. Id=%d\ n ", envelope.Id) w.WriteHeader (500) _, _ = fmt.Fprintf (w,") Id=%d\ n ", envelope.Id) return} _, _ = fmt.Fprint (w, envelope.Code)}) fmt.Println (" listen on 8080 ") fmt.Println (http.ListenAndServe (": 8080 ", nil))} Thank you for reading this article carefully. I hope the article" how to use Redis to optimize Interface performance in High concurrency scenarios "shared by the editor will be helpful to you. At the same time, I also hope that you will support and pay attention to the industry information channel, and more related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.