Shulou(Shulou.com)06/03 Report--

The current design scheme

1.1. Control count:

In the current project, there are many interfaces that require permission access control to the accessor. The current design scheme is to use redis cluster to store the access count information of each access control point. The key value is = PlatformId (access platform) + InterfaceId (system interface) + dayTime (date and time), and the value value is the statistical list of visits for each period of the day.

1.2. Control rules:

Through the page configuration and the formulation of control rules, the business system loads the control rules at startup, and accesses redis to obtain the control times, and then completes the logical judgment in the business system, and the ACL control does the request interception processing.

Current pain points:

2.1. In the case of large amount of data and high concurrency, because the business system is integrated and concurrent processing, there will be an instant qps peak of a single point of redis to 13w/s. The reason for this is that the design of Platform+InterfaceId+Day as key will cause a single point of request of the access side to be directly mapped to the backend redis when the QPS is too large, which does not take advantage of the clustering effect of redis.

2.2. The current access control function is closely coupled with the business code, so it is impossible to expand the access control function horizontally, which makes it inconvenient for management and optimization.

Structure diagram:

Preliminary solution:

4.1. Reduce the number of visits to redis and subdivide the key value of redis so that it can be evenly distributed to the redis cluster.

4.2. The ACL access control function is serviced to support stateless horizontal expansion.

4.3. Asynchronous processing logic is introduced to separate the ACL service from the business side. Subsequent changes in ACL services are not aware of the business system.

4.4. Realize the pre-calculation function, do the grouping pre-calculation in the current ACL system to do the redis transfer time, and reduce the frequency of updating redis.

4.5. Delay detection control, ACL access control is mainly to intercept some excessive requests, it is not necessary to ensure real-time and consistency. High throughput processing power is achieved through delay statistics strategy.

4.6. In the notification interception mechanism, the business side does not need to do any interception control statistics and analysis, but only needs to accept the notification from the ACL system and perform the notification action.

Specific implementation details:

5.1. All configuration information is managed by the configuration center, and ACL and Service (service system) register and subscribe to the service on zk.

5.2. Service sends statistical messages to MQ, and ACL subscribes to the TOPIC to consume

5.3. ACL does preliminary preprocessing calculation in its own memory, and regularly refreshes to redis cluster to do aggregation operation.

5.4. Start the timing service to scan all control rules, detect the rules that need to be intercepted, create a broadcast message and send it to the MQ.

5.5. The Service subscribes to the broadcast message, discovers the message, parses it and intercepts the access that meets the rules in memory.

5.6. When the configuration center information changes, it is pushed to all subscribers through the broadcast, and the subscribers get the message and go to the service provider to pull the information and update their memory information.

The structure diagram is as follows:

At present, the reconstruction scheme of the project is being implemented, and the follow-up implementation effect needs to be updated in the second edition.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.