Shulou(Shulou.com)06/01 Report--

This article mainly uses a bug to record how to analyze the crash information of a MySQL bug.

Version: Percona 5.7.17-11

Database restart log analysis terminate called after throwing an instance of 'std::out_of_range' what (): vector::_M_range_check04:10:09 UTC-mysqld got signal 6; mysqld got signal 6; .Thread pointer: 0x0Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this Something wentterribly wrong...stack_bottom = 0 thread_stack 0x40000/dbdata/mysql3306/bin/mysqld (my_print_stacktrace+0x35) [0xf3e175] / dbdata/mysql3306/bin/mysqld (handle_fatal_signal+0x4b4) [0x7c3b94] / lib64/libpthread.so.0 (+ 0xf7e0) [0x7f79f28e87e0] / lib64/libc.so.6 (gsignal+0x35) [0x7f79f137d495] / lib64/libc.so.6 (abort+0x175) [0x7f79f137ec75] / usr/lib64/libstdc++.so.6 (_ ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d) [ 0x7f79f1c37a8d] / usr/lib64/libstdc++.so.6 (+ 0xbcbe6) [0x7f79f1c35be6] / usr/lib64/libstdc++.so.6 (+ 0xbcc13) [0x7f79f1c35c13] / usr/lib64/libstdc++.so.6 (+ 0xbcd32) [0x7f79f1c35d32] / usr/lib64/libstdc++.so.6 (_ ZSt20__throw_out_of_rangePKc+0x67) [0x7f79f1bdadb7] / dbdata/mysql3306/bin/ mysqld [0x11d8f15] / dbdata/mysql3306/bin/ mysqld [0x11d99d5] / dbdata/mysql3306/bin/mysqld (_ Z17 examples statsupdateP12 Upd_option_t+0x9dc) [0x11de0cc] / dbdata/mysql3306/bin/mysqld (dict_stats_thread+0x4f2) [0x11e0512] / lib64/libpthread.so.0 (+ 0x7aa1) [0x7f79f28e0aa1] / lib64/libc.so.6 (clone+0x6d) [0x7f79f1433bcd] You may download the Percona Server operations manual by visiting http://www.percona.com/software/percona-server/. You may find information in the manual which will help you identify the cause of the crash.

This part is the stack frame when the database crashes, because the signal received is 6 SIGABRT, as long as the signal is captured and its behavior is changed. This section is called Stack Trace in the official MySQL documentation. Refer to:

28.5.1.5 Using a Stack Trace

In fact, we can already see the problem of statistics collection here, because we see dict_stats_thread, which is the callback function of the statistics collection thread.

Second, generate a more visual Stack Trace1 and obtain the memory address through Stack Trace

Get the following:

[0xf3e175] [0x7c3b94] [0x7f79f28e87e0] [0x7f79f137d495] [0x7f79f137ec75] [0x7f79f1c37a8d] [0x7f79f1c35c13] [0x7f79f1c35d32] [0x7f79f1bdadb7] [0x11d8f15] [0x11d99d5] [0x11de0cc] [0x11e0512] [0x7f79f28e0aa1] [0x7f79f1433bcd] 2, put these addresses in a file

For example, vi / tmp/err0222.log can be put in.

3. Get the library file link file through the nm command

For example, nm-D-n. / mysqld > / tmp/mysqld.sym

4. Use the mysql tool resolve_stack_dump to get the output

As follows:

[root@dyzsdb2 bin] # / resolve_stack_dump-s / tmp/mysqld.sym-n / tmp/err0222.log | c++filt 0xf3e175 my_print_stacktrace + 53 0x7c3b94 handle_fatal_signal + 1204 0x7f79f28e87e0 _ end +-258115144 0x7f79f137d495 _ end +-280574355 0x7f79f137ec75 _ end +-280568243 0x7f79f1c37a8d _ end +-271422363 0x7f79f1c35be6 _ end +-271430210 0x7f79f1c35c13 _ end +-271430165 0x7f79f1c35d32 _ end +-271429878 0x7f79f1bdadb7 _ end +-271802481 0x11d8f15 dict_stats_analyze_index_for_n_prefix (dict_index_t*, unsigned long, std::vector const*) Mtr_t*) + 4949 0x11d99d5 dict_stats_analyze_index (dict_index_t*) + 2693 0x11de0cc dict_stats_update (dict_table_t*, dict_stats_upd_option_t) + 2524 0x11e0512 dict_stats_thread + 1266 0x7f79f28e0aa1 _ end +-258147207 0x7f79f1433bcd _ end +-279827035

In fact, through the function call here, we can find that there is a problem with the statistical data collection.

Third, query Bug https://bugs.launchpad.net/percona-serverhttps://bugs.mysql.com/ through the official website.

Bring up the representative information in the error message and search in the official website. By looking at percona, it is found that this bug is caused by the upstream MySQL code BUG number: Bug # 84940

And found that it was fixed in 5.7.18 and the internal BUG number is given as follows:

[10 Feb 2017 8:12] Shane BesterOli, Umesh, this would be same as internal:Bug 24585978-INNODB: ASSERTION TOTAL_RECS > 0 FAILURE IN FILE DICT0STATS.CC 4. Query where Bug has been modified.

I consulted brother Yinfeng of Ali here before I knew how to check it. First of all, I got the internal bug number: 24585978

Then search in the commit log of git and get

Git-- no-pager log > / root/commitlog

Vi / root/commitlog found the commit number:

29acdaaaeef9afe32b42785f1da3d79d56ed7e59

Here is the place to fix this bug:

Commit 29acdaaaeef9afe32b42785f1da3d79d56ed7e59Author: Thirunarayanan Balathandayuthapani Date: Wed Feb 8 12:00:52 2017 + 0530 Bug # 24585978 INNODB: ASSERTION TOTAL_RECS > 0 FAILURE IN FILE DICT0STATS.CC Analysis: = There was missing bracket for IF conditon in dict_stats_analyze_index_level () and it leads to wrong result. Fix: = Fix the IF condition in dict_stats_analyze_index_level () so that it satisfied the if condtion only if level is zero. Reviewed-by: Jimmy Yang diff-- git a/storage/innobase/dict/dict0stats.cc b/storage/innobase/dict/dict0stats.ccindex 3494070..55a2626 100644-a racer storageracket innobaseCharpy raceme raceme 0stats.ccstats.ccrack @-1099Lie10 @ dict_stats_analyze_index_level (leaf-level delete marks because delete marks on non-leaf level do not make sense. * /-if (level = = 0 & & srv_stats_include_delete_marked? 0level + if (level = = 0 & (srv_stats_include_delete_marked? 0: rec_get_deleted_flag (rec) -page_is_comp (btr_pcur_get_page (& pcur) {+ page_is_comp (btr_pcur_get_page (& pcur) {if (rec_is_last_on_page & & prev_rec_is_copied) 5. Why is it so modified?

Here is my simple analysis. Please forgive me for what is wrong.

We find that the modification here is actually an extra parenthesis, but the meaning is very important.

If (level = = 0 & & srv_stats_include_delete_marked? 0: rec_get_deleted_flag (rec, page_is_comp (btr_pcur_get_page (& pcur) is modified to if (level = = 0 & (srv_stats_include_delete_marked? 0: rec_get_deleted_flag (rec)) Page_is_comp (& btr_pcur_get_page)

Before modification:

If level! = 0, no matter how the innodb_stats_include_delete_marked parameter is set, it must trigger to determine whether the del_flag exists, and then skip the line by setting the offset, but the subsequent (* total_recs) + +; will not be triggered and may be 0 in extreme cases.

The found_level: in the upper code dict_stats_analyze_index actually requires that the number of rows of non-leaf nodes not equal to 0 is as follows:

/ * if any of these is 0 then there is exactly one page in the B-tree and it is empty and we should have done full scan and should not be here * / ut_ad (total_recs > 0); ut_ad (n _ prefix-1] > 0); VI. How to avoid

There are the following ways to avoid this Bug when viewing the official website

Upgrade to 5.7.18 set parameter innodb-stats-persistent = 0 innodb-stats-transient-sample-pages = 20 innodb-stats-auto-recalc = 0

Setting these parameters is actually using the old non-solidified way of collecting statistics, rather than collecting through thread dict_stats_thread. The following is a logical fragment located in row_update_statistics_if_needed as follows:

If (dict_stats_is_persistent_enabled (table)) {/ / Parameter innodb-stats-persistent function if (counter > n_rows / 10 / * 10% * / & & dict_stats_auto_recalc_is_enabled (table)) {/ / Parameter innodb-stats-auto-recalc Action dict_stats_recalc_pool_add (table); table- > stat_modified_counter = 0;} return } / * Calculate new statistics if 1 / 16 of table has been modified since the last time a statistics batch was run. We calculate statistics at most every 16th round, since we may have a counter table which is very small and updated very often. * / if (counter > 16 + n_rows / 16 / * 6.25% * /) {ut_ad (! mutex_own (& dict_sys- > mutex)); / * this will reset table- > stat_modified_counter to 0 * / dict_stats_update (table, DICT_STATS_RECALC_TRANSIENT);}

If you do this, you will certainly not call the function that triggers bug. If you are interested, you can look at the logic of dict_stats_update (table, DICT_STATS_RECALC_TRANSIENT);. It's actually the old way that breakpoints can be typed on the btr_estimate_number_of_different_key_vals function.

Author Wechat:

Wechat .jpg

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.