Shulou(Shulou.com)06/01 Report--

This article shows you what the solution of Redash+kettle is, which is concise and easy to understand. It will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Redash has its own simple result set reoperation capability, which provides one-stop data analysis application capabilities from data extraction, data modeling, data analysis to data visualization and early warning distribution; even if you try professional ETL tools, you can also achieve joint dynamic analysis of multiple data sources (interactive analysis functions such as linkage, drilling, dynamic parameters, etc.). Allow users to quickly access all kinds of data sources, and realize the process of data import-processing-analysis without the help of data warehouse, so Redash can realize the application scenario of real-time intelligent data analysis mainly, supplemented by simple data processing without data warehouse.

However, with the continuous improvement of enterprise user data volume and analysis complexity, the lightweight data storage and data flow processing module on the data analysis platform is difficult to withstand the huge computing pressure. if a large number of index operations and historical data year-on-year / ring comparison operations are needed, it is still necessary to build a data warehouse or data platform in a planned way. Redash can perfectly integrate the open source ETL tool Kettle to build enterprise data warehouse and BI system.

Data does not flow directly from the source into the BI system like water. Usually we need ETL (the process of data extraction, transformation and loading) to grab the data from the source to the data warehouse of BI. ETL tools can integrate the scattered, disorderly and non-uniform data in the enterprise.

Kettle is an efficient and easy-to-use ETL tool that enables data to be loaded from multiple heterogeneous data sources to destination addresses. Kettle can run on Windows, Linux, Unix, supports graphical GUI design interface, flows in the form of workflow, and is efficient and stable in data extraction, quality inspection, data cleaning, data conversion, data filtering and so on.

Kettle introduction

Kettle is a component-based integrated system, which includes the following main parts:

1.Spoon: graphical interface tool (GUI). Spoon allows you to design Job and Transformation through a graphical interface, which can be saved as a file or in a database. You can also run Job and Transformation directly in the Spoon graphical interface

2.Pan:Transformation executor (command line mode), Pan is used to execute Transformation in the terminal, there is no graphical interface.

3.Kitchen:Job executor (command line mode), Kitchen is used to execute Job in the terminal, there is no graphical interface.

4.Carte: an embedded Web service for remotely executing Job or Transformation,Kettle to set up a cluster through Carte.

5.Encr:Kettle command-line tools for string encryption, such as encrypting database connection parameters defined in Job or Transformation.

Application case

A supermarket uses redash to make real-time data analysis reports, which mainly obtains the latest real-time data in the early stage, which requires high timeliness, so redash is used to dynamically obtain the real-time display of database data.

But in the later stage, with the development of business, data fetching is becoming more and more complex, and it is necessary to analyze a large number of data indicators and year-on-year comparison, and when the requirement of data immediacy is not high, redash multiple result set query operations are used to retrieve data repeatedly, and the efficiency of data analysis is relatively low. In the case of a very large amount of data, kettle can be used to work and convert directly, and multiple conversions can be performed in one job, so the efficiency of data analysis will be greatly improved. finally, we will cooperate with Redash for visual analysis, making reports for display.

Kettle can simplify the creation, update and maintenance of data warehouse, help enterprises quickly build data pipelines between heterogeneous data sources, so that complex data projects no longer become a bottleneck, Redash+kettle= complete BI project solution, to achieve one-stop to build enterprise-level data visualization analysis cloud platform.

The above content is what is the solution of Redash+kettle? have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.