Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to solve the problems encountered by map reduce". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to solve the problems encountered by map reduce".

Question: mkdir: Permission denied: user=root, access=WRITE

We encounter this kind of problem when doing mapreduce, just use the user specified by him to execute mapreduce. For example, if he requests hdfs, we will use the hdfs user to execute it. The code is as follows:

System.setProperty ("HADOOP_USER_NAME", "hdfs")

Step one:

Solution:

Go to the server to modify the configuration file of hadoop: conf/hdfs-core.xml, find the configuration item of dfs.permissions, and change the value of value to false

Dfs.permissions

False

Note that the experimental platform can be set up like this, but in work, it can't be like this, it's not safe.

If it still doesn't work, dfs doesn't have permission.

Step 2:

The cause of the problem:

Environment hadoop2,cdh6 creation

Permission issues occur when creating file paths using hadoop fs-mkdir / usr/xxx

=

1.hadoop fs-ls / user

Check the user rights learning under this path

Drwxrwxrwt-yarn hadoop 0 2014-09-03 11:58 / user/history

Drwxr-xr-x-hdfs hadoop 0 2014-09-03 12:00 / user/spark

Where: yarn-- creator, hadoop-group, and root users belong to guest type

two。 Log in using the user hdfs

Sudo-uhdfs hadoop fs-mkdir / usr/

3. Change the user's permissions through the hdfs user

Sudo-uhdfs hadoop fs-chown user: user group / usr

For example, if we are root users, we can use the command groups root to check the root user's group, and get the group, which is also root, now change the user's permissions.

Sudo-uhdfs hadoop fs-chown root:root / usr

Then use the command hadoop fs-ls / to see the directory / usr, the user name is root, and the user group is root

It's done.

Thank you for your reading, the above is the content of "how to solve the problems encountered by map reduce". After the study of this article, I believe you have a deeper understanding of how to solve the problems encountered by map reduce, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.