Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how ClickHouse compiles and debugs under windows. Many people may not know much about it. In order to let everyone know more, Xiaobian summarizes the following contents for everyone. I hope everyone can gain something according to this article.

What is ClickHouse?

In the era of big data, every piece of data carries a kind of information. All kinds of excellent sensors, gateways and IOT devices are outputting their own operation information with logs all the time. After this information is stored, it is calculated in multiple dimensions to form our current big data environment. In order to facilitate calculation, there are many excellent databases and components, they are good at solving problems in various scenarios in their own fields, one of which is in the OLAP scene, with amazing performance indicators born database, this is ClickHouse.

It was open-sourced in 2016 by Yandex, a conservative Russian business company (similar to Baidu in China). I'm not going to talk about ClickHouse today, because when you first get interested in ClickHouse, your first thought is,"Talk is cheap, Show me the code." "But for large-scale projects, code download is easy, some frameworks are complex, dynamic calls can clearly know the process of the project, compiled and debugged once, will have a deeper understanding of logic, but also reduce the difficulty of reading, today is to talk about how to compile debugging, especially in windows using powerful Visual Studio to debug ClickHouse.

compile ready

I will assume that everyone's level will know some basic compilation information, here only do the requirements:

WSL or CentOS 7, I am used to reducing virtual machine consumption under windows, the two environments are not much different;

GCC 7.4.0, ClickHouse heavily uses C++17 syntax, requiring compiler version 7.4;

cmake 3.14.5, This download package is directly decompressed, and the soft chain can be established.

ninja 1.9.0, download the corresponding package directly, unzip it

Visual Studio, I use VS2019

Compilation process Download source code

Try to use git download in linux environment (including WSL), ClickHouse should reference a lot of external open source projects, some source files will establish soft links in the pulling process, if in windows, these soft links will fail to establish, resulting in late compilation errors. There is no need to download the source code package directly on git, so that the external reference of open source projects will not be pulled, resulting in compilation.

//Pull git clone --recursive https://github.com/yandex/ClickHouse.gitcd ClickHouse//Switch to branch 19.7, or just pull that branch code at first git tag -lgit checkout v19.7.3.9-stable Start compiling mkdir buildcd buildcmake.. ninja clickhouse VERIFICATION Compilation Results dbms/programs/clickhouse server -VClickHouse server version 19.7.3.1

If the above results appear, then you have succeeded.

However, we haven't achieved our goal yet, we need to debug it under windows.

Debugging Compilation Preparation Why recompile

By default ClickHouse is compiled into static modules, packaged into one large program. In order to reduce link time when debugging code changes later, we modified it to compile into a dynamic link library, which can also be explored module by module. Also, ClickHouse is non-DEBUG mode by default, since different algorithms are enabled in DEBUG and non-DEBUG modules in the memory management section. We intend to use ninja to call gcc this set of non-debugging modules compiled into so library, in the need to modify the code compiled with Visual Studio, and Visual Studio debugging, generally DEBUG module, so other modules (ninja) in compilation, we changed to DEBUG mode in advance.

Modify the CMake profile

Change CMakeLists.txt in root directory, USE_STATIC_LIBRARIES to FALSE.

option (USE_STATIC_LIBRARIES "Set to FALSE to use shared libraries" FALSE) recompile mkdir buildcd buildcmake.. -DCMAKE_BUILD_TYPE=Debug ninja -j 4 Check compilation results

If the libdbmsd.so library appears under build/dbms, the compilation is successful!

Creating VS Engineering

At present, we should have the full amount of code, many modules have been compiled into so library, what we have to do now is to add a main.cpp file, call the interface to run up the part we care about successfully.

create drawing

Create ClickHouse.sln, ClickHouse.vcxproj two projects in the root directory, the project type is Linux project, and then open VS project. As shown below:

Note: I used the Show All Files view and added a main.cpp file with the following code, also a test case under Parser.

#include #include #include #include int main(int, char **)try{ using namespace DB; std::string input = " SELECT 18446744073709551615, f(1), '\\\\', [a, b, c], (a, b, c), 1 + 2 * -3, a = b OR c > d.1 + 2 * -g[0] AND NOT e

100" " ORDER BY Visits, PageViews" " LIMIT LENGTH('STRING OF 20 SYMBOLS') - 20 + 1000, 10.05 / 5.025 * 5" " INTO OUTFILE 'test.out'" " FORMAT TabSeparated"; ParserQueryWithOutput parser; ASTPtr ast = parseQuery(parser, input.data(), input.data() + input.size(), "", 0); std::cout

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.