Shulou(Shulou.com)06/03 Report--

IDEA development WordCount (Spark) error report

WordCount {} WordCount {(args: Array []) {hadoopAddr = conf = SparkConf (). SetAppName () sc = SparkContext (conf) mapRdd = sc.textFile (hadoopAddr) result = mapRdd.flatMap (_. Split (). Map ((_)). ReduceByKey (_ + _). Collect ()}}

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties

15-12-08 22:07:16 INFO SparkContext: Running Spark version 1.4.0

15-12-08 22:07:17 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... Using builtin-java classes where applicable

15-12-08 22:07:18 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

Java.io.IOException: Could not locate executable null\ bin\ winutils.exe in the Hadoop binaries.

At org.apache.hadoop.util.Shell.getQualifiedBinPath (Shell.java:355)

At org.apache.hadoop.util.Shell.getWinUtilsPath (Shell.java:370)

At org.apache.hadoop.util.Shell. (Shell.java:363)

At org.apache.hadoop.util.StringUtils. (StringUtils.java:79)

At org.apache.hadoop.security.Groups.parseStaticMapping (Groups.java:104)

At org.apache.hadoop.security.Groups. (Groups.java:86)

At org.apache.hadoop.security.Groups. (Groups.java:66)

At org.apache.hadoop.security.Groups.getUserToGroupsMappingService (Groups.java:280)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.