Shulou(Shulou.com)06/02 Report--

When it comes to artificial intelligence and deep learning, people always think of algorithms and models at the first time, and then the most fundamental data that provide power sources. Due to the rapid progress and wide application of artificial intelligence technology, our way of dealing with data has changed from collecting to obtaining information.

If you don't convert the stored data into available information, then the data-- narrowly speaking-- is just a pile of bytes. Before completing this conversion process, it sometimes takes years to collect sufficient data, such as trials of new medical processes, drugs or equipment; group behavior based on infrequent external factors; and climate change.

First of all, the importance of data preservation cannot be denied.

There is a mouthful about data, you don't know what you don't know. There is a good example: "junk DNA". The term was coined by a geneticist in the 1970s to refer to 95% of the 98% DNA in the genome that does not compile any proteins or enzymes. Biologists at that time believed that since almost all specific physiological functions were accomplished by proteins, DNA that did not encode proteins should be useless and could be called "junk DNA". By the beginning of this century, it was found that some junk DNA actually regulated the way and time of chromosome replication.

For people at that time, the cost of storing data was very high. Of course, DNA sequencing is more expensive, which is one of the reasons people wanted to keep junk DNA data in the first place. The cost of collecting data is high, and the cost of storing data is also high, so we want to be more grateful to those who did the right thing before us. They store these old data under the pressure of cost, giving us the opportunity to find more information in it.

We know that some weather forecasting centers keep all the data collected every day, including the output of their forecast models. When these sites have a new prediction model, they run the old data through the new model, look at the output and observation of the model, and see if the new model is better than the old model, and how good it is. For a city, this job seems easy, but for the whole planet, it is a lot of data and information comparison.

As a result, the challenge for storage and data architects is often how to preserve this data by developing architectures that meet performance, scalability, and governance requirements.

The change from data collection to information mining

From the beginning of data collection, its only purpose is to make all the data collected meaningful. Manual data collection and analysis is very time-consuming, and converting data into information is time-consuming and costly.

The information age began with the use of Hollene punch cards during the 1890 US census, and although they are blank, they are different from the formatted cards you have seen. The key problem here is that although there was a lot of data before 1890, there were no tools to analyze it, and it was expensive to convert it into information.

It is clear that the information generated in the 1890 census is very basic by today's standards. But by the standards of the 1890s, it was revolutionary. In this way, people can look at the results of the census very quickly and make decisions (for example, actionable information based on data).

Today, we no longer refer to the tabularization of the 1890 census data as information. The definition of information-- compared to data-- should be based on contemporary standards, just as some definitions in many other areas are changing.

The size and scope of the information analysis market is expanding, from self-driving cars to safety camera analysis to medical development. In every industry, in every corner of our life, there are rapid changes, and the speed of change is also increasing. All of this is data-driven, and all collected new and old data is used to develop new types of available information. There are many problems around the needs of data collection and information development.

In addition to keeping the data active, compliance is also important

Many requirements are based on the information and data types you have. For example, some may involve the use of a so-called DAR (Data Encryption at Rest), which encrypts storage devices so that data is almost impossible to access if removed from the system. The degree of difficulty depends on the encryption algorithm and its size, complexity, etc. We can sum up this type of requirements as "operational requirements", that is, the rigid requirements for architecture, equipment, etc. in the whole process of data value, in order to meet the performance, availability, and data integrity required for business operation, all of these problems need to be addressed in order to maintain the viability of data and information.

In addition, your data or information should also be based on your industry's best practices or regional regulations, such as the GDPR (General data Protection regulations) recently introduced by the European Union. In other words, your use of data needs to be compliant at all times. The resulting architectural or process changes are also important tasks that need to be addressed by the architect.

The last thought

Compliance is not easy, and it is not cheap. There are many factors that determine its cost, but trying to enforce compliance after planning and building an architecture is always more expensive than doing it beforehand.

The author believes that when defining compliance requirements, you should focus on the future, not just the present, because it will be more costly and challenging to force things after the event. This means that we need to constantly study compliance requirements in the industry, as well as best practices. Data will only become more important in the future, we will always face challenges, why not make a plan to deal with it first.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.