Shulou(Shulou.com)05/31 Report--

This article introduces the knowledge of "how to plan the database operation and maintenance system from the perspective of life cycle". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations! I hope you can read it carefully and be able to achieve something!

Recently, when we were planning the OKR goals with the team, we discussed a lot of issues. I first listed some existing problems, and intended to follow the derivation:

1) list current problems

2) Classification and summary of problems

3) sort out the existing experience and existing schemes

4) the expected goal of a certain period of time can be obtained by combining time / performance-price ratio.

On the whole, the workload is still quite large, coupled with people's different understanding of the problem, so it is easy to discuss too much in many details, it is difficult to focus.

So I thought about it and prepared to consider it according to the dimension of the life cycle, so I sorted out a version of the design, which was divided into four levels, that is, from applying for resources and permissions, to service launch, service optimization, and finally, related service data migration and transfer.

In the overall design, we will find many inadequacies and omissions in the consideration. After many refinements, I adjusted the design to the following mode:

Let me explain one by one:

1) Specification / selection / planning: this stage puts more emphasis on the whole. If we start with the basic operation and maintenance directly, it will be too late. Some of the service quality is poor, the delivery time is long, and in essence, the previous infrastructure is not solid enough. So this is a mutually beneficial relationship, such as the design and implementation of development specifications, architecture design (such as distributed architecture design). Technology selection (such as MySQL 8.0 middleware technology research, ClickHouse technology research, TiDB technology selection, MyRocks storage engine test analysis, etc.), SQL audit (upgrade and improvement of existing audit services, etc.), high availability (top priority, involving health check scripts, rapid switching of Consul services, database high availability pre-research and testing, etc.), basic services (such as monitoring, alarm and task scheduling related services) Basic settings (such as abandoning lower versions such as CentOS_6, unifying disk configuration to SATA-SSD, etc.)

2) basic OPS: involving resource delivery (including uplink and downloading, resource expansion, etc.), permission delivery (application account, account permission change, account recovery, etc.), installation and deployment (such as database software installation and deployment, initialization), basic configuration (basic configuration, such as ntp,crontab, etc.), backup recovery (basic backup set based on the basic dimension of data backup and data recovery, data recovery based on point-in-time)

3) Operation and maintenance optimization: object changes (need to evolve to automated online mode), and integration of online change tools for large table changes. In addition, the focus is to do some related optimization, such as parameter optimization (such as database optimization parameters, basic configuration adaptation), object optimization (data table optimization, index optimization), SQL optimization (execution plan optimization, index recommendations, etc.), configuration optimization (system configuration, etc.) Service configuration optimization, etc.)

After these three dimensions are done, you will find that some of them will still hate hard work, which involves data migration and data flow. The data itself flows between different types of environments. How to ensure the stability of the data? accurate flow is also an important goal.

4) data migration and data flow, data migration mainly implements one-click data migration, which mainly includes two aspects:

(1) one-click database migration, migrating from one server to another service, and one-click implementation

(2) Database version upgrade, such as upgrading from MySQL 5.5to 5.7and from 5.7to 8.0can be achieved with one click.

In addition, data flow to the data warehouse, big data, how to efficiently and stably support, how to achieve real-time data flow mechanism and rapid migration / synchronization between multiple environments is also the key goal.

For the technical base, the primary goal is the document, the document can be divided into a variety of documents from the above four dimensions, such as standard design document, pre-research document, scheme design document, operation document, case document and so on.

The next service delivery should be unified into the API model, and the evolution can be evolved from script to tool, from tool to API path.

The two major branches of the base are cloud platform construction and service construction. Cloud platform construction covers a larger area, providing service delivery of product thinking, and higher requirements for technical architecture and development efficiency. This part should not aim too high, but still have to provide strong power according to their own situation. Among them, metadata construction is the core goal. At this level, metadata should be integrated to achieve process management.

The service construction on the right is closer to the back-end service, managing periodically at the instance, database, table, field and index level from the perspective of life cycle, while the auxiliary services provided are closer to the actual operation and maintenance. for example, slow log optimization, patrol service and fault self-recovery, and the business side is a translucent open form.

This is the end of the content of "how to plan the database operation and maintenance system from the perspective of life cycle". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.