How to implement Null value Verification in COGROUP in PIG
This article will explain in detail how to achieve null verification in COGROUP in PIG. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.
Environment: 0.10.0
COGROUP: grouping among 2 + relationships
Recently do a PIG class project, re-look at the PIG, by the way to make a summary. I felt something wrong when I saw COGROUP.
Official: http://pig.apache.org/docs/r0.8.1/piglatin_ref2.html
A = load 'student' as (name:chararray, age:int, gpa:float); B = load' student' as (name:chararray, age:int, gpa:float); dump B; (joe,18,2.5) (sam,3.0) (bob,3.5) X = cogroup A by age, B by age;dump X (18, {(joe,18,2.5)}, {(joe,18,2.5)}) (, {(sam,3.0), (bob,3.5)}, {}) (, {}, {(sam,3.0), (bob,3.5)})
It's hard to understand why there are two pieces of data when age is empty. Refer to other blogs to take a look at the usage of COGROUP and find that this is not the case.
Coding no regrets blog: http://www.codelast.com/?p=3621
Later, I created a batch of data to try COGROUP with no null value and null value.
CA: (jack,32,run) (liza,22,eat) (tom,20,mouse) CB: (jack,32,run) (jerry,10,steal) (tom,20,mouse) COGROUP: (tom, {(tom,20,mouse)}, {(tom,20,mouse)}) (jack, {(jack,32,run)}, {(jack,32,run)}) (liza, {(liza,22) Eat)}, {}) (jerry, {}, {(jerry,10,steal)})
Add a field with a null value.
CA: (jack,32,run) (liza,22,eat) CB: (jack,32,run) (jerry,10,steal) COGROUP: (jack, {(jack,32,run)}, {(jack,32,run)}) (liza, {(liza,22,eat)}, {}) (jerry, {}, {(jerry,10) Steal)}) (, {(, 20m mouse)}, {}) (, {}, {(, 20m mouse)})
If you add M and N null value fields respectively, the amount of data will not have M null values, will it? It's either two or a big hole, which magnifies the amount of data.
CA: (jack,32,run) (liza,22,eat) (, 200 liza,22,eat) (, 202) (, 203) (, 204) CB: (jack,32,run) (jerry,10,steal) (, 301) (, 302) COGROUP: (jack, {(jack,32,run)}, {(jack,32,run)}) (liza, {(liza,22,eat)}, {}) (jerry, {}, {(jerry,10,steal)}) {}) (, {}, (, 301), (, 302), (, 303), (, 304), (, 305), (, 306), (, 307)}), (, 307), (, 302), (, 303), (, 303), (, 305), (, 305), (, 305), (, 306)), (, 307)), (, 307)), (, 302), (, 302), (, 303), (, 303), (, 304), (, 305), (, 305), (, 306), (, 307)), (, 307)), (, 302), (, 303), (, 303), (, 303), (, 30 I hope the above content can be of some help to you and learn more knowledge. If you think the article is good, you can share it for more people to see.
