Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Direct attack on DTCC2019 site: exploration and practice of Intelligent Operation and maintenance of Database

2025-03-01 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Database >

Share

Shulou(Shulou.com)06/01 Report--

On May 10, the 10th China Database Technology Conference (DTCC2019) is in full swing. As one of the focuses of discussion in this conference, the popularity of "Database Intelligent Operation and Maintenance" was explosive. Industry experts from Jingdong Logistics, Tencent, ByteDance, Jingdong Mall and Convenience Peak attended the site and shared their experiences on hot topics such as database automation operation and maintenance, remote disaster recovery system, etc.

Jingdong logistics super large-scale storage system database cluster to promote security tips

In the fast shopping experience of Jingdong Logistics, warehousing and delivery timeliness are the most critical links. In the e-commerce industry, Jingdong Logistics has a super warehouse management system (referred to as WMS system), covering the links from warehousing, warehousing and delivery, especially the ultra-large-scale warehouse system cluster unique in the e-commerce industry, which plays a decisive role.

▲ Jingdong Logistics Senior DBA Gao Wenjia

Gao Wenjia gave a detailed introduction to WMS system, and put forward six suggestions for database operation and maintenance: awe production environment, fear, can stop; Standardize process operation, reject human feelings "accident"; Regular fault drill, prepare emergency plan; High-risk operation + double confirmation, reduce misoperation; Active operation and maintenance + automatic operation and maintenance to avoid emergency fire fighting; Training + active communication to retain risks in the development stage.

TDSQL Intelligent Operation and Maintenance Platform-Bian Que Architecture and Practice

As a finance-grade database,TDSQL has six core features, including strong data consistency, finance-grade high availability, high performance and low cost, enterprise-grade security, linear horizontal scaling, and intelligent O & M. TDSQL completely avoids the security risks caused by human error by providing "Red Rabbit" self-service operation and "Bian Que" intelligent DBA.

▲ Tencent Financial Cloud T4 Expert Lei Hailin

"Red Rabbit" self-service operation service, from the administrator's perspective, in the availability, security, efficiency, cost dimensions of comprehensive control, 90% of daily operations can be completed through the Web page, reduce human error while helping financial users save management and economic costs, reduce risks.

Lei Hailin introduced that "Bian Que" intelligent DBA has the characteristics of fault early warning, automatic fault diagnosis, historical event analysis, optimization suggestion, self-service operation through management console, reduction of DBA work intensity, etc., helping financial users to prevent system abnormalities.

Canal automated operation and maintenance and remote disaster recovery system practice

Traditional database operation and maintenance methods are difficult to meet the stability and efficiency of big data scenarios. Canal as middleware solves the MySQL Binlog acquisition, saves the Binlog to the message queue, and then receives the stream computing framework, or offline computing framework.

For this reason, ByteDance (ByteDance) has intelligent operation and maintenance of Canal, automatically perceives database changes and active adaptation, and proposes multi-computer room deployment and remote disaster recovery solutions, realizing unified intelligent management of multi-computer room Canal instances. Li Chang said,"We adopt Manager Mode architecture, which requires ensuring data consistency, accuracy, service stability, offline warehouse construction and online real-time synchronization. "

▲ ByteDance (ByteDance) Senior Big Data Platform Engineer Li Chang

For future development planning, Li Chang revealed,"We hope to support automatic instance rebalancing in terms of stability to avoid excessive load on a single machine; in terms of operation and maintenance, support instance configuration operation and maintenance center, intelligent monitoring and early warning of instance traffic." "

Application of machine learning in database operation and maintenance

Traditional database operation and maintenance methods have many limitations, such as passive optimization (monitoring/alarm/slow SQL/application error reporting, etc.); time-consuming and inefficient, it is difficult to form a closed loop; limited by human limitations, complex scenarios lack scalability; lack of data value in the decision-making process, and greater subjectivity in decision-making standards; and the contradiction between DBA expert shortage and database service requirements is more prominent.

How to allocate resources accurately and prevent them in advance? The SmartDBS system of Jingdong intelligent operation and maintenance platform based on machine learning is gradually solving these problems. The system consists of five modules: classification, prediction, diagnosis, decision-making and scheduling. The data of classification, prediction and diagnosis finally enter the decision module to participate in the decision of container resource allocation, and are pushed into the scheduling to realize the rational redistribution of resources.

▲ Peng An, Development Engineer of Jingdong Mall

"The value of SmartDBS lies in expert systems, intelligent diagnostics and predictive analytics," Penang said. Expert decision-making, reduce costs and improve efficiency, can avoid the subjectivity of personnel maintenance; multidimensional data modeling analysis comprehensive diagnosis database; single index feature prediction multi-model data verification analysis. "

Convenience Bee Database Operation and Maintenance Automation Evolution from 0 to 1

Convenience peak database management platform includes backup system, slow query system, online change system, online query system, MySQL high availability system, etc. It includes requirements application, SQL operation, cluster management, capacity management, log query, slow query management, service governance, database management, backup management, Redis management platform, cluster monitoring, OPS management platform, Beta management, platform weekly report and other functions.

▲ Convenience Bee DBA Chen Haifeng

As for the development process of Convenience Peak SQL Change System, Chen Haifeng revealed that there are mainly three stages: "embryonic period, breakthrough period and iteration period.""First of all, our work is mainly process standardization, backup monitoring and slow query system. Then, SQL change, SQL query, database high availability research and development; from October 18 to now, we began to deploy expansion clusters for service governance and capacity management. "

Database operations automation comes from pain points at work, and this pressure drives continuous technological change. Through the sharing of the above five industry experts, we can think about the intelligent operation and maintenance of databases from more angles. The future will be an era of automated and intelligent database operations.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Database

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report