What if there is a java.io.IOException: Mkdirs failed to create error in HDFS

This article mainly introduces how to do java.io.IOException: Mkdirs failed to create errors in HDFS. It is very detailed and has certain reference value. Friends who are interested must finish reading it.

When writing spark code, when saving the results of rdd to hdfs

Rdd.saveAsTextFile ("/ data/spark")

There it is.

Java.io.IOException: Mkdirs failed to create

Personally, I always thought that this mistake should be written on hdfs, but as soon as I executed it, I reported an error. Finally, I found that this method was written locally. If I want to write to hdfs, I should write the full path.

Rdd.saveAsTextFile ("hdfs://192.168.23.21:9000/data/spark")

That's it. Sum it up.

SaveTextFile saves RDD to a file, and local mode saves it in local file. Cluster mode means that if it is based on Hadoop, it is saved on HDFS. Because I am using local mode, I will report the above error. If I use a cluster, I can write that directly.

The above is all the contents of the article "java.io.IOException: what to do with Mkdirs failed to create errors in HDFS". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!