Shulou(Shulou.com)06/02 Report--

From October 17th to 19th, the 10th China system Architects Conference (SACC 2018), hosted by IT168's ITPUB Enterprise Community platform, was held in Beijing. The theme of this conference is "Ten-year Architecture, the Road to growth", which gathered thousands of technicians such as CTO, R & D director, senior system architect, development engineer and IT manager to share and discuss problems in the field of IT technology.

At the sub-forum of "main Line 1: distributed Storage and Middleware Design" on the afternoon of the 17th, a number of IT leaders and technical experts from well-known manufacturers and Internet companies expressed their technical experiences in distributed storage, and answered questions and discussed technical questions for everyone on the spot.

DiDi is providing a variety of travel and transportation services, including express cars, chauffeured cars and taxis, for hundreds of millions of users, with daily orders reaching 30 million. In such a large scale of business, storage has naturally become a major problem. Didi object storage system currently stores tens of billions of files and PB-level data.

Wang Li, senior technical expert of DiDi

At the forum, Wang Li, a senior technical expert of DiDi and head of Didi Cloud Storage Research and Development, gave a keynote speech on "the practice of Architecture Evolution of Didi object Storage system", introducing the evolution process and design considerations of Didi object storage system with the increase of business scale and the change of business requirements.

The second person to share is Ding Jun, head of the online storage department of JD.com Mall, who introduced JD.com Kmuri V storage products from memory storage to persistent storage, from persistent storage to distributed locks and configuration distribution products developed based on KMeiV persistent storage. According to the introduction, KMel V storage system includes three parts: memory storage-jimdb, persistent storage-sharkstore and hybrid storage.

Ding Jun, head of online Storage Department of JD.com Mall

The three parts have their own different characteristics, memory storage mainly shows high throughput and low latency, with online scaling, automatic fault recovery and wide area replication and other functions; persistent storage has distributed strong consistency, support online splitting, automatic fault recovery, support schema, massive data, support range query, single table operation and other core features Hybrid storage can achieve hot and cold data storage separation, providing memory-level performance experience and disk-level storage capacity.

Driven by the need to reduce the time it takes to convert large amounts of raw data into useful information and actions to zero, streaming media technology has become popular: when the data arrives, it processes the data instantly, continuously, and infinitely. From the industrial Internet of things to connected cars, to real-time fraud detection and so on, there are more and more applications that need to respond quickly to user behavior. However, the components we currently use are not designed for these applications, and their mechanism will not expand from small to large, and the mismatch of the basic system design brings higher complexity.

Guo Sijie, co-founder of Streamlio

Flow is difficult to scale because it assumes many destructive system functions, such as treating data as continuous and infinite rather than finite and static, by dynamically extending service, storage, and computing power to coordinate the amount of data arrived. the ability to deliver consistent and fast results. So, what kind of distributed storage do we need for streaming media? Guo Sijie, co-founder of Streamlio, brought a speech on the topic "from file storage, object storage to streaming storage-rethinking distributed storage in the era of streaming computing", discussing the evolution of distributed storage systems driven by use cases and the needs of distributed storage in the streaming media world. In addition, Mr. Guo Sijie also used Apache Pulsar/BookKeeper as an example to discuss how to design such a system to meet the challenges.

The last person to share the technology is Xu Li, a senior technical expert from Aliyun, who is the head of product and research and development of Aliyun file storage. He has nearly a decade of experience in architecture and research and development of Aliyun Feitian distributed storage system. The theme of Xu Li's speech is "the Evolution of distributed Architecture for the New Generation of Cloud Native File Storage". He introduces the evolution and transformation of the current new generation of file storage technology architecture from six dimensions: protocol diversity, security and reliability, high performance and sharing, flexibility and ease of use, enterprise characteristics, and ecological innovation.

Xu Li, a senior technical expert in Aliyun

According to Xu Li, the main features of the new generation of Cloud Native file storage include: easy to use-fully managed, that is, ready to use, only need to mount standard file protocols; flexible use-on-demand, seamless expansion; high scalability-capacity, performance level expansion, Internet-oriented distributed architecture; high performance-linear correlation between performance and capacity, low latency, high IOPS, high throughput. Aliyun file storage provides simple, fast and efficient data migration capabilities, multiple data access methods and best mounting methods. Cloud monitoring features with visual access behavior and easy-to-use, fully hosted, reliable and secure backup features help users to complete end-to-end best practices.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.