In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-28 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >
Share
Shulou(Shulou.com)06/01 Report--
Editor to share with you how to use Hadoop to enter the era of large database, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!
How popular is Hadoop? It can be seen from a series of actions in the industry. Mainstream database manufacturers, including Oracle, Microsoft and Sybase, have released Hadoop connector products to make it easier for users to transfer information between traditional relational databases and open source distributed processing systems.
These manufacturers regard Hadoop connector software as an important part of the "big data management" strategy, but it is not only mainstream database manufacturers that are doing this. Like data warehouse provider Teradata and Hewlett-Packard's Vertica have launched similar Hadoop products, there is no lack of Informatica, Talend and other data integration software manufacturers. Startups such as Hortonworks, Cloudera and MapR also play an important role in this ecosystem.
Rod Cope, the technical director of OpenLogic, has a lot of experience in using Hadoop, and he warns users to consider the application scenarios and data requirements before using Hadoop connectors. Cope introduces that his company uses a combination of Hadoop, Hbase and a listed NoSQL database that, as part of OpenLogic's main business, can help its customers audit software applications to verify that the embedded open source code used conforms to the relevant license. OpenLogic has not yet deployed any connector software, but Cope has shown some curiosity about this technology, thinking that such software can be used to transfer frequently accessed data from a relational database to Hbase for archiving.
But Cope believes that Hadoop connector software does not solve all the problems, and interested users need to pay attention to the speed of loading data. When dealing with big data, people tend to pay less attention to performance standards than before, and if it takes too long to load data to Hadoop users, then there is little point in using connectors. The problem is not with Hadoop, but with the data source you load.
David Menninger, an analyst at Ventana Research Institute, said that Hadoop distributed File system (HDFS) and the database products built on it can provide users with very good data management and analysis solutions, compared with traditional relational databases and data warehouses. The data may be machine-generated big data, such as Web search logs, social media messages, mobile phone calls and other unstructured data.
Menninger pointed out that a typical scenario used by Hadoop connector software is that enterprises use Hadoop system to extract a small amount of structured analysis information from a large number of unstructured data sources, and then transfer it to a relational database for further analysis using BI tools.
"at present, users put information into relational databases, mainly because it is not easy to make reports with Hadoop data sources," Menninger said. "there is a mature reporting and analysis system in the industry, of course, for relational data."
This kind of data transmission is not necessarily an one-shot deal, maybe you are counting the number of times an event has happened, and then you want to calculate the number of times two things happen together. You can go back to the data source and process the information again, which is why people don't delete unstructured data, they can be stored in Hadoop.
In addition, compared with SQL database, Hadoop provides a better environment for advanced analysis and data mining applications. For example, analyze customer service phone logs and social media information to find out customers' points of interest and word-of-mouth about a product. This is a very difficult thing for SQL, but it can transfer information to a relational database or data warehouse through Hadoop connectors.
Cameron Befus, vice president of Tynt Multimedia, said they used Hadoop to provide analytics services to more than 500000 users. In addition, Tynt uses the open source MySQL database as back-end support. So far, Befus doesn't see the need to deploy Hadoop connectors. "We do transfer data, but it's usually straightforward," he said. "We import files directly from Hadoop into MySQL, which might be easier if we use connectors, but it's not a problem for us."
But IT analysts believe that with the popularity of Hadoop, the use of such connector software will gradually increase. Analysts like Menninger believe that the company hopes to introduce the results of Hadoop-based analysis into a larger business environment, which is also the driving force behind the development of connector technology. What is the most important when we look at big data? That's what these data can tell me about the key problem. Users want to be able to build a bridge between unstructured data, streaming data, meaningful data and highly structured data, so that the root of the problem can be found through analysis.
The above is all the content of this article "how to use Hadoop to enter the era of big database". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.