Shulou(Shulou.com)06/03 Report--

This article mainly explains "how to connect Spark SQL to remote Hive Metastore". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's learn how to connect Spark SQL to a remote Hive Metastore!

For Spark 1.x, you can set with :

System.setProperty("hive.metastore.uris", "thrift://METASTORE:9083");final SparkConf conf = new SparkConf();SparkContext sc = new SparkContext(conf);HiveContext hiveContext = new HiveContext(sc);

Or

final SparkConf conf = new SparkConf();SparkContext sc = new SparkContext(conf);HiveContext hiveContext = new HiveContext(sc);hiveContext.setConf("hive.metastore.uris", "thrift://METASTORE:9083");

Update If your Hive is Kerberized :

Try setting these before creating the HiveContext :

System.setProperty("hive.metastore.sasl.enabled", "true");System.setProperty("hive.security.authorization.enabled", "false");System.setProperty("hive.metastore. keranthus.principal", hivePrincipal);System.setProperty("hive.metastore.execute.setugi", "true"); At this point, I believe that you have a deeper understanding of "how to connect Spark SQL to remote Hive Metastore", you may wish to actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.