Shulou(Shulou.com)06/03 Report--

The 2015-01-19 partition of the phenomenon query fct_traffic_navpage_path_detl displays the results as follows:

Hive 1.1.0 returns the following result. You can see that the last field is NULL.

87FQEZT1UEDXJHJQPFFX7G7ET8S2DVPM 2357378283356 91501191044440048 7326356 NULL

87FQEZT1UEDXJHJQPFFX7G7ET8S2DVPM 2357378283356 121501191035580028 7326356 NULL

UBDTK8D9XUZ9GRZU8NZNXDEG73D4PCZG 2362223711289 161501191549050061 14837289 NULL

Y49EY895ACABHS95DRQEE8DVFEB8JSE1 2360853052224 111501191426280023 115883224 NULL

The result returned by Hive 0.10 is as follows:

87FQEZT1UEDXJHJQPFFX7G7ET8S2DVPM 2357378283356 915011910440048 7326356 2015-01-1910: 44:44

87FQEZT1UEDXJHJQPFFX7G7ET8S2DVPM 2357378283356 121501191035580028 7326356 2015-01-1910: 35:58

UBDTK8D9XUZ9GRZU8NZNXDEG73D4PCZG 2362223711289 161501191549050061 14837289 2015-01-1915: 49:05

Y49EY895ACABHS95DRQEE8DVFEB8JSE1 2360853052224 111501191426280023 115883224 2015-01-1914: 26:28

It was later found that the meta of this table was different from the meta of the partition 2015-01-19.

Hive > desc dw.fct_traffic_navpage_path_detl

OK

Date_id string

Chanl_id bigint

Sessn_id string

. .

Detl_event_id string

Nav_refer_intrn_reslt_sum string

Nav_intrn_reslt_sum string

Nav_refer_intrn_kw string

Nav_intrn_kw string

Detl_track_time string

Cart_track_time string

Ds string

# Partition Information

# col_name data_type comment

Ds string

Hive > desc dw.fct_traffic_navpage_path_detl partition (ds='2015-01-09')

OK

Date_id string

Chanl_id bigint

Sessn_id string

. .

Detl_event_id string

Ds string

# Partition Information

# col_name data_type comment

Ds string

As shown in the table above, the meta has the following fields more than the meta of partition (ds='2015-01-09')

Nav_refer_intrn_reslt_sum string

Nav_intrn_reslt_sum string

Nav_refer_intrn_kw string

Nav_intrn_kw string

Detl_track_time string

Cart_track_time string

Later, I checked that hive allows table meta to be different from partition meta, see

Http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3CCA+v58Ocy=uoHsLNQGrYYpRhawFse-p-6U4sjmzNL6VERYrSKTg@mail.gmail.com%3E

You can specify a partition when you add or change fields:

ALTER TABLE table_name [PARTITION partition_spec]

ADD | REPLACE COLUMNS (col_name data_type [COMMENT col_comment],...)

[CASCADE | RESTRICT]

So why does hive 0.10 show "correct" and hive 1.1.0 display an error?

Later, when looking at the jira, we found that the hive0.10 Sedre is based on the meta of Table, while the version of hive after 0.11 is based on the meta of parition.

Https://issues.apache.org/jira/browse/HIVE-3833 can say that hive 0.11 is an improvement. Although hive0.10 says "correct", it is actually a mistake to read tabel's meta, so those fields are displayed, in fact, the result is wrong, it is a defect of hive0.10.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.