Shulou(Shulou.com)11/24 Report--

The annual OSCAR Open Source Industry Conference 2023 is about to be held. We intend to find the peak figures of the past 5 years, have a dialogue with them, review the gains and losses of the past, and look forward to the future.

Pan Juan, the peak figure of OSCAR open source, introduces

Co-founder of SphereEx & CTO, Apache Member & Incubator Mentor, Apache ShardingSphere PMC, AWS Data Hero, China Mulan Open Source Community mentor, Tencent Cloud TVP. Used to be responsible for the design and development of JD.com Mathematical Database Intelligent platform, and now focuses on distributed database & middleware ecology and open source field. Was named "2020 China Open Source Pioneer", 2021 OSCAR Peak Open Source figure. CSDN IT leader in 2021, published his paper "Apache ShardingSphere,A Holistic and Pluggable Platform for Data Sharding" in ICDE in 2022. In 2022, he was selected as the Globee Awards "Chief Technology Officier of the Year" honor. In 2022, he published the overseas book A Definitive Guide to Apache ShardingSphere as the first author.

2023 interview

On the afternoon of August 23rd, Pan Juan was interviewed by "Open Source way" on behalf of the OSCAR Conference Organizing Committee (hereinafter referred to as OSCAR) in the office of SphereEx Company. The following is based on the live recording:

OSCAR: first of all, let's talk about the meaning or feeling of OSCAR pinnacle character setting.

Pan Juan: the setting of pinnacle open source characters is of great significance and value to the open source community and developers. This recognizes individuals who have made outstanding achievements in the field of open source, and is also a window to promote the culture and value of open source around the world. On the other hand, the establishment of the award can also encourage more developers to participate in open source contribution, so that open source and technological innovation can be expanded to a larger scope, in terms of culture and fashion, to promote the development of open source and technology. Stimulate the spirit of innovation and collaboration and promote the development of global open source technology and culture. In addition, I have seen the list of OSCAR pinnacle figures over the years, and the selected pinnacle figures are still of "quality". They are relatively objective and fair, are more gold-rich, and start well, and then they will get everyone's attention. These pinnacle figures are indeed recognized by everyone, so that more people will actively declare and run in the election every year. As long as we ensure that the quality of the pinnacle characters is high, the rules are made, and the fairness of judgment is in place, more open source contributors will be encouraged.

OSCAR: the mechanism of open source pinnacle is recommendation. Now that the conference is still a month away, would you recommend some open source contributors?

Pan Juan: I think I will recommend it. In my mind, there are some teachers who are willing to recommend it for two reasons: first, I recognize the efforts of open source contributors, and I hope they can get a certain return. Only in this way can we form a positive circle, otherwise, hiding developers will form a vicious circle, which we should try our best to avoid. Second, the reason why I recommend them, um. I still hope they can get a positive feedback.

OSCAR: in the past few years, especially recently, we have experienced changes, including the past of COVID-19 's epidemic and technological changes. I would like to ask you to talk about your views and views.

Pan Juan: let's talk about two aspects. First of all, when we talk about open source, not only the open source culture and the open source trend, but also the participants, technologies, companies and organizations. When the trend is over, these will be more or less affected. For example, the topic we discussed slowly changed from open source to AI and ChatGPT. Affected by these, those who developed in the most exciting part, whether individuals, organizations, communities, companies. Have been affected. But from another point of view, this is also a return to normal, because nothing is always high, and the thing itself is a rising and falling tide. We must accept such a law of history that there is no right or wrong. It is the law of the development of history. To accept this law, we can only adjust based on such a reality.

OSCAR: as you personally, is SphereEx's CTO, Apache ShardingSphere PMC, AWS Data Hero,OSCAR open source peak figures, multiple roles, that is, leaders in the circle, standing in front of the arms shouting people, with such a role, how do you view the current situation?

Pan Juan: from the manager's point of view, what I can do is to control the things around me as much as possible and push them for the better, such as my community, some Apache projects and my company. I will take some measures, such as adjusting the R & D cycle and adopting a small step iterative approach to make the community and products sustainable. I will also provide guidance to other projects to ensure that these projects continue to develop. At this time, we need to settle down, immerse ourselves in research and development, and do a good job of products. Being well prepared for the next challenge is something I can control.

From a personal point of view, as a KOL, what I can do is to express my own personal experience, at least so that those who hear it can use my point of view as a reference. So I think it's very good.

OSCAR: when it comes to what you can control, tell us about Apache's project, SphereEx's products. What is the current status?

Pan Juan: let's start with open source. At present, the project I am in mentor is a map database project donated by Baidu to Apache Software Association: HugeGraph, which is carried out in a small iterative way. Because the investment is limited, we can only do some small functions that can be realized in a short time, and the release cycle is also shorter.

The same is true of the Apache ShardingSphere project, as it is a community, it is now trying to attract as many new people as possible to keep the community alive. Some community members have become accustomed to this community, affected by the current heat drop, or other reasons, reduced some investment, at this time, if there is some fresh blood, think open source is good, can join in, I think this is another way to activate the development of this community.

From the company's point of view, the current stage is definitely customer-centered, so, uh, the feature of our current product release may no longer be planned for three years, or even one year, which will take a lot of time to do basic construction. What we are doing now is three months, because customers need certain types of characteristics, so we have to devote time and energy to this part to quickly bring the product to market. Push it to the customer, and then in order to get the order and get the market verification quickly, these are some measures that I can take now. Our commercial version has already released 1.6 version, and we will release 1.7 version immediately, that is, what the current customer needs and what the market needs. We will quickly meet.

OSCAR: can you give me an example?

Pan Juan: we have a customer who has an urgent need for encryption and decryption on the Oracle database. For security considerations, we have to deal with the data. In the past, we would want to consider all the mainstream databases such as Oracle, MySQL, PG and so on, and give more consideration to the underlying support. We hope to support the encryption and decryption of all databases once and for all. Now, we have changed our strategy, and we find that more users want to support Oracle, accounting for 50%. Then we will first run this scenario and then introduce it to the market. If there are other databases that need to be supported, then we will do it again. For example, PG is such an iterative process.

At present, many enterprises are also reducing costs and increasing efficiency, and they also hope to save costs, such as moving cold data from the production environment to cheap storage, and putting online data on cold backup as much as possible, but cold backup still needs to be queried. At this time, they will wonder if there is a software that can automatically do these things according to certain rules. At this time, SphereEx will provide users with such a function. Instantly migrate online data to cold backup safely and reliably to achieve the purpose of cost reduction. Under the special needs, the design of our products also changes.

OSCAR: is it in the original plan of SphereEx R & D to provide solutions for users?

Pan Juan: yes, what we need to do is Data as Service, the whole platform for database management, governance and data access, especially in heterogeneous situations. In theory, our original core is to solve the problem of distribution and the problem of federated query between different databases. This is our original main line and secondary task, which is the hot and cold data separation we just mentioned. But now, with the passage of time, we find that the demand for the secondary line is gradually increasing, so we need to allocate some energy and turn the secondary line into a new major task. This is an adjustment we made. But these are still a piece of our "big plate".

OSCAR: that means changing your mind and focusing on what you need more urgently. Oh, by the way, can I talk to you about the recent development of Apache ShardingSphere?

Pan Juan: ShardingSphere is a community-driven project that cannot be understood in terms of business aspirations. Of course, SphereEx employees are one of the most important sources of strength in the community, and SphereEx will submit more basic and general-purpose code for business features to the upstream. In addition, the ongoing feature of the ShardingSphere project is to improve the project SQL support, it is not a specific function, such as CDC, it is the infrastructure function, more general, more for the bottom. In this way, we can get more people to participate, for example, some companies are using Oracle, some companies use MySQL, if A users only care about Oracle, then they may support Oracle SQL, and other companies will provide support in other dialects.

OSCAR: you just mentioned that the open source community you participate in, such as the ShardingSphere community, needs to bring in more fresh blood. Is there any specific method or strategy to share with you?

Pan Juan: first, open source projects have to be done, such as ShardingSphere to support more SQL, we now begin to dock MongoDB, there are new things we can do, this is the fundamental first step, if a project has been done, it means that there is no way to do it. Second, for example, today's interview, ApacheCon Asia attended last week, to do a conference sharing, will naturally bring some traffic; third, ShardingSphere itself has been open source for five or six years, relatively speaking, processes and other things have been quite mature, it is time to raise the demand. Fourth: we will also participate in many activities overseas, such as Open Source India, India's local open source conference, as long as it appears at such a conference, people will definitely participate, there are many ways for people to try, if they are interested, they will further develop, and new people will join.

Not all of the methods mentioned above are 100% effective, but as long as you do it, someone will come.

What are the advantages of the current version of OSCAR:SphereEx from a solution perspective, or from an industry perspective?

Pan Juan: advantage point one: no transformation. For example, Oracle, MySQL, PG and other databases are encrypted and decrypted, because what we do is an intermediate layer, which has little impact on users' business. In particular, an enterprise uses a mixture of multiple databases, and it is very troublesome to encrypt and decrypt different databases in different databases. We provide the middle tier, and we can support different types of databases in one step, and then the business does not need to be reformed. All we need to do is introduce SphereEx. The configuration that used to point to the database now points to SphereEx, which is one of the advantages of attracting many customers.

Advantage point 2: similar to the lightweight database solution, for example, we recently signed a customer of a central enterprise, they did not choose a new database, but based on the existing MySQL group, chose SphereEx, so that it is more secure and stable, the original database does not move, but just introduce SphereEX, this risk is controllable, if completely and thoroughly change to a new set of database It means replacing the original infrastructure, which will be a big move. At this time, who will bear the risk, especially the core data, as well as the ecological construction around the distributed database, such as the original MySQL cluster, PG cluster, and the introduction of SphereEx, the original architecture has not changed, the operation and maintenance staff, ecology and tools have not changed, and if it is switched to new, then it will rely entirely on the new database manufacturer. There is a risk of being locked in.

Advantage point three: cost advantage, now mention the database, that is the huge cost of figures, without a few million simply can not come down, the goal of SphereEx is to solve the problem, but also several times the cost of the new database to solve practical problems, the market is price-sensitive, but also can solve problems: if the query is slow, you can consider more SphereEx.

OSCAR: from the perspective of CTO, what do you think of data in current privacy computing, big models, etc.?

Pan Juan: we look back 20 years ago, at that time, it can be called data desert. Compared with today, there are many types of databases, the emergence of corresponding database tools, and the emergence of data-as-a-service such as SphereEx. It is precisely because the explosive development of the Internet and mobile Internet has produced many so-called data assets. Now everything is digital, and human beings have produced an unlimited amount of data. What we are thinking about now is how to use the data and how to tap the value. we see all kinds of databases, tools handled by big data, as well as the explosion of open source projects: Hadoop, Spark, Hive, etc., everyone is trying to dig the treasure and develop all kinds of applications.

The emergence of ChatGPT is another way to play. We provide a certain infrastructure, have data assets, and have a lot of software tools that can help us mine and use data, so that the data can flow. Now we can train large models and use these infrastructure software to mine the underlying data value really efficiently, in line with the scene. This is a very powerful thing. For example, the past method: half a day to write some scripts, spent a long time to explore the mining, I do not know if it is useful, now it is different, ChatGPT and other direct change of life, originally thought that big data training is far away from us, now casually visit a website, intelligent customer service behind is a large model, quickly integrated into the application, and the deployment speed is extremely fast It is a very interesting thing to make a new round of "rolls" on top of the infrastructure we have done in the past. Slowly, we will find that we can use the data, save the data, and constantly mine the data, and finally use it in the scene to maximize its value. This is a thing that is very difficult to generate commercial value. I think that the companies behind AI may start wave after wave and eventually settle down.

Will OSCAR:SphereEx embrace such a process?

Pan Juan: I don't think it will now. I know it's a trend in the future, but I have to consider the cost first. I can't throw away the existing one and quickly embrace the new one. I have to make sure that people of my current size can survive. First, I have to consider the cost. Second, I have to consider whether the scene and the customer will pay the bill. The final completion of this thing is not to meet my own needs, but to really meet the needs of the business.

It is necessary to further explore how to combine the company's products with AI to meet the needs of customers in order to make customers pay the bill. And also need corresponding talents, these need to be considered, can not be said because it is hot, so we have to embrace it, obviously will not work, will eventually fall to the ground. If I am an individual, I do not affect the company and do what I like to do, but to lead the whole company to run, what I consider at this time is how to integrate AI into the company's products, whether it is investment in production and research, or investment in talent, to polish the delivery of products, there are still a lot of things to consider.

OSCAR: have our developers started using coding assistants such as GitHub Copilot?

Pan Juan: our developers have not used it yet. I have tested it myself. The reason why it is not useful is not that these coding assistants are not good, but that our special scenario and the business logic we have to write cannot help us to do this. Copilot can only do some more basic functions. Our R & D team is ultimately responsible for the features, and the functions are more complex, not a process from 0 to 1. It's a process from 80 to 100, and it's a process that depends on developers.

OSCAR: can you give some suggestions or experience references for newcomers involved in open source? For example, what does open source mean for students who are still in college?

Pan Juan: in view of the current situation in the recruitment market, it is an opportunity to tell students to try open source. At this stage, students must find what they want to do, or a direction of technology, or quickly become a leader. Or quickly complete the transformation from students to professionals, these things can be "pre-positioned", so how can they be "pre-positioned"? Then open source can be one of the ways after two years of undergraduate or graduate students, and you can also choose other ways. Open source is like an interest group that can practice open source technology in the open source technology community. It's like connecting with football technology in a football community.

For example, we have a classmate here, from Sri Lanka, is a junior, from Google Summer of Code to Apache ShardingSphere, I am his mentor, one summer vacation, this student mastered such a set of skills, dealing with SQL and so on, very happy, but also wrote a certain blog, the so-called pre-position is: did not study the school database-related courses, nor signed up for extracurricular training courses, acquired such skills And the code written by the company is also applied to the actual environment.

To take another example, there is a classmate who is in a common state where young people don't know what they want. He just graduated and wanted to find a job. Then he worked in open source for a while and got to know a lot of people he couldn't get in touch with. Then he found a direction. Do you want to go out and have a look? this is why open source brings him some opportunities. If he is sitting still, then there will be nothing. Probably still confused. Open source opens a window of opportunity, which is what makes it interesting.

Coming back to the question, it is to encourage everyone to participate in open source by participating in the community, and only if they participate in it will there be opportunities.

OSCAR: then it's time for you to play freely. Relative to the OSCAR audience, open source, Apache community, and so on, what you want them to hear.

Pan Juan: I hope that today is my sharing, there are other open source peak people to share, no matter who shares, the best way is to take action, even if it is only a point, must do, if just listen, after a while forget. As long as you start, there is bound to be a change, and when there is a change, you will know that there is at least a good chance of knowing what you want.

What I have always believed in is to do it from a small place, and then to explore, and finally to stick to it, because only quantitative change can cause qualitative change, these three steps are very important, for myself, the first step: if you want to change, start small. Step 2: keep exploring in this process, and the third step is to stick to it and practice.

OSCAR: so what is the biggest feeling that you have chosen to start a business for more than two years?

Pan Juan: starting a business is a bit like operating a ship. When you get on board, you don't know what you will encounter. In the course of sailing, you will encounter rocks and storms, so what I can do is to control the ship to move in the right direction as smoothly as possible. This is a very big feeling for me. In order to be able to do this, my knowledge system and cognitive and behavioral patterns are all iterating and changing.

For example, the original time, I do open source or Community, you are happy, I am happy, we are happy together, after starting a business, the company is not like this, there must be unhappy people in the company, for collective action, always have to sacrifice some people's intuition and perception, as a decision maker, must make a tradeoff, must make a decision, must promote progress, and bear all the consequences.

OSCAR: at the moment, we are at a critical juncture. The environment is changing and there is a general lack of confidence. Do you think your current state will be affected?

Pan Juan: the development of things is spiraling up, rising and falling. In the low ebb, what we need to do is to examine and analyze ourselves, adjust our goals, and stick to it with confidence! Take SphereEx, for example. First of all, we have established a relatively perfect open source community ecology, products and services have also been recognized by users. Therefore, the current market volatility will not have a significant impact on our core business. Secondly, our company's current financial and capital management strategies can cope with market changes and uncertainties. In addition, our investors and board of directors also understand the particularity of basic software open source commercial companies and have full confidence in our long-term strategy and future development.

Last but not least, think about it, is this what you want to do from the bottom of your heart? I believe in the power of open source technologies and business models, and recognize the path I have decided to take, which is enough, leaving the rest to time. Because I know that no matter how many times I repeat my choice, I will stick to the direction I want, and no one can predict the future and right or wrong.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.