Shulou(Shulou.com)05/31 Report--

This article mainly shows you "how to permanently customize hive functions", the content is simple and easy to understand, organized clearly, I hope to help you solve doubts, let Xiaobian lead you to study and learn "how to permanently customize hive functions" this article bar.

Permanently custom hive function 1: Reasons for doing this:

There are some functions that are more basic, common, and each time you have to create temporary functions. Trouble, such basic functions need to be directly integrated into hive to avoid creating them every time.

2: Steps

I have an account zb_test

Custom functions are ready

Log in to your linux account and modify the.bashrc file in your home directory:

Replace CLASSPATH with the following:

export CLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$SQOOP_HOME/lib:/opt/boh-2.0.0/hadoop/share/hadoop/

tools/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/*:/opt/boh-2.0.0/hive/li

b/*:$CLASSPATH (intended to be used when compiling)

Get the source code for hive-0.12.0-cdh6.0.0-src.tar.gz

Unzip, find hive-exec-0.12.0-cdh6.0.0.jar package (hive function related to this), and then unzip.

jar -xvf XXX.jar

Compile java files for custom functions, such as under utf

Javac ./ utf/*

Create a.class file and place it in the abc directory.

Type the.class file into a jar package

Jar -cvf ./ hello.jar ./ abc/*

Create a jar package called hello.jar

The requested URL/data/zb_test/was not found on this server.

Then modify the/data/zb_test/.bashrc file:

export CLASSPATH=/data/zb_test/hello.jar:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$SQOOP_HOME/lib:/opt/boh-2.0.0/hadoop/share/hadoop/tools/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/lib/*:/opt/boh-2.0.0/hadoop/share/hadoop/common/*:/opt/boh-2.0.0/hive/lib/*:$CLASSPATH

Register the function test in hive's function list

Modify the file $HIVE_HOME/src/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java

import org.apache.hadoop.hive.ql.udf.UDFTest;

registerUDF("test", UDFTest.class,false);

Important steps:

1: Put all the.class files you just generated under $HIVE_HOME/src/ql/src/java/org/apache/hadoop/hive/ql/udf

2： Then compile the FunctionRegistry.java file: After compiling, we find the $HIVE_HOME/src/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.class file in hive-exec-0.12.0-cdh6.0.jar, and we replace it.

These paths are all under the path hive-exec-0.12.0-cdh6.0.0.jar unzipped org.

In fact, after these two steps are done, re-package the just-decompressed hive-exec-0.12.0-cdh6.0.0.jar into a jar package and replace the one in hive.

That's all for "How to customize hive functions permanently". Thanks for reading! I believe that everyone has a certain understanding, hope to share the content to help everyone, if you still want to learn more knowledge, welcome to pay attention to the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.