Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

How to add permanent custom functions in hive

2025-01-30 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Share

Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to add permanent custom functions in hive. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

Permanently customize the hive function 1: the reason for doing this:

Some functions are relatively basic and common, and it is troublesome to create temporary function every time. Such basic functions need to be integrated directly into hive to avoid having to create them every time.

2: step

I have an account at zb_test.

Custom functions are ready

Log in to your Linux account and modify the .bashrc file in the home directory of the account:

Change the CLASSPATH to the following:

Export CLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$SQOOP_HOME/lib:/opt/boh-2.0.0/hadoop/share/hadoop/

Tools/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/*:/opt/boh-2.0.0/hive/li

B/*:$CLASSPATH (designed to be used when compiling by yourself)

Get the source code of hive-0.12.0-cdh6.0.0-src.tar.gz

Extract it, find the hive-exec-0.12.0-cdh6.0.0.jar package (the hive function is related to this), and then extract it.

Jar-xvf XXX.jar

Compile the java file of the custom function, such as under utf

Javac. / utf/*

Generate .class files and put them in the abc directory.

Type the .class file into a jar package

Jar-cvf. / hello.jar. / abc/*

Generate a jar package called hello.jar

Put the jar package in the / data/zb_test/ directory

Then modify the / data/zb_test/.bashrc file's:

Export CLASSPATH=/data/zb_test/hello.jar:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$SQOOP_HOME/lib:/opt/boh-2.0.0/hadoop/share/hadoop/tools/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/*:/opt/boh-2.0.0/hive/lib/*:$CLASSPATH (for testing)

Register the function test in the list of functions in hive

Modify the $HIVE_HOME/src/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java file

Import org.apache.hadoop.hive.ql.udf.UDFTest

RegisterUDF ("test", UDFTest.class,false)

Important steps:

1: put all the newly generated .class files under the modified $HIVE_HOME/src/ql/src/java/org/apache/hadoop/hive/ql/udf

2: then compile the FunctionRegistry.java file: after compiling, we find the $HIVE_HOME/src/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.class file in hive-exec-0.12.0-cdh6.0.0.jar, and we replace it.

These paths are all under the path of the decompressed org of hive-exec-0.12.0-cdh6.0.0.jar.

In fact, after these two steps have been done, re-pack the newly decompressed hive-exec-0.12.0-cdh6.0.0.jar into a jar package and replace the one in hive.

This is how to add permanent custom functions to the hive shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Servers

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report