Shulou(Shulou.com)05/31 Report--

Greenplum tool GPCC and GP log time mismatch example analysis, I believe that many inexperienced people do not know what to do, so this paper summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

Today, my colleague gave feedback on a problem. I didn't care too much before. Although it didn't hurt, I thought that if I didn't pay attention to it, the problems I would encounter in the later stage would emerge one after another, so let's take a look at it as one of my tasks today. Whether we can locate and solve it, of course, if we look at it later, we can also see that we have found a general way to deal with the problem.

The problem is obvious: the GPCC tool can display the contents of the GP log, but it obviously doesn't match the time in the GP log.

A screenshot of GPCC is as follows, which is simply like Oracle's OEM tool. Be able to view the status of the cluster, do some basic information collection and visualization. The part of the red block diagram is to display the error message in the log.

I enlarged the contents of the log to make it easy to see.

The following is a snippet from GPCC.

Intercept a section of GPCC for reference.

03 Apr14:18:07

ERROR

MPP detected 1 segment failures, system is reconnected (cdbfts.c:228)

U:datax_userdb:TESTDB host:10.xxxx

The log of GP is shown below:

2018-04-03 00 CST 1815 07.055801

"datax_user", "TESTDB", p173295 Department 972601120, "10.xxxx", "64523", 2018-04-03 00:17:40 CST,1811909,con659620,cmd1,seg-1,dx572994,x1811909,sx1, "ERROR", "XX000", "MPP detected 1 segment failures, system is reconnected (cdbfts.c:228)", "COPY test_map, line 11705805:" 20150826 | 38377 | 5364390 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 00:1.

... "", "COPY test.test_map (xxxx) FROM STDIN delimiter as'| 'NULL' null'", 0Magna, "cdbfts.c", 228, "Stack trace:

1 0xb0aefe postgres errstart (elog.c:502)

2 0xc29d9f postgres FtsHandleNetFailure (cdbfts.c:227)

3 0xbd4ca5 postgres cdbCopyEndAndFetchRejectNum (cdbcopy.c:804)

4 0x6b757a postgres CopyFromDispatch (copy.c:3823)

5 0x6c6c9c postgres DoCopyInternal (copy.c:1767)

6 0x6c8388 postgres DoCopy (copy.c:1883)

7 0x9a5f9d postgres ProcessUtility (utility.c:1100)

8 0x9a364b postgres PortalRun (pquery.c:1505)

9 0x99a5bc postgres (postgres.c:1811)

10 0x99e9b9 postgres PostgresMain (postgres.c:4760)

11 0x8f8dfe postgres (postmaster.c:6672)

12 0x8fba90 postgres PostmasterMain (postmaster.c:7603)

13 0x7fbeff postgres main (main.c:206)

14 0x37f901ed1d libc.so.6 _ _ libc_start_main + 0xfd

15 0x4be869 postgres + 0x4be869

"

According to the time situation, the time shown in the gpcc is obviously faster than that in the GP log. After careful comparison, it is 14 hours faster according to the accuracy.

Another question is what the segment failure mentioned in the error log means and whether an explanation can be given.

So we still have to go back to the GP log, and we need to make an understanding of the context and play back the operation at that point in time. Looking ahead, we quickly located the relevant log, and it turned out that we were doing a batch of copy operations, and one of the copy operations was likely to be blocked due to network jitter.

So the basic conclusion of the error message is as follows:

Through the log, we can make it clear that in the process of GP doing copy, there may be a network problem that hinders the operation, and GP tries to reconnect to segment.

After basically explaining the problem, let's take a look at the nature of the problem, why the timestamp in the system is different from that in the log by 14 hours.

So naturally, we will ask a question: how is the data transferred from the log to the front end?

Another question is how the data is transferred from the back end to the front end, the initial direction is above the time zone, but I looked at the deployed software configuration, and there is no configuration for the time zone.

After consulting some friends, I decided to see if there was any official explanation.

It took some effort, but fortunately I found one, which was updated at the end of March. It is a good time to encounter this problem at this time.

Voiceover is to search or pay attention to some skills, or search can not come out is really embarrassing. In fact, the official suggestion is that it is because of the specific setting of the time zone, it can also be understood to be a bug. In the implementation, this problem is caused by Chinese support. If you want to make a WA, you can reset the GPCC archives and users' timezone. Of course, you also need to restart the GP cluster to take effect. The modified date and time stamp shows that it is not CST, but HKT, and you may need to evaluate whether there are other areas of influence.

So the basic conclusions about the inconsistency of time are as follows:

This is a problem with GPCC, which appears in the 3.x version and is the same in the lower version.

To fix this problem, you need to reconfigure the time zone, re-GP the cluster, and consider whether there is an opportunity to fix the problem later. The premise is to fully test and verify in the test environment. For now, keep the status quo.

After reading the above, have you mastered the method of analyzing cases with time mismatch in Greenplum tools GPCC and GP logs? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.