Shulou(Shulou.com)06/01 Report--

This article is to share with you the content of an example analysis of a large footprint of var directories in a CDH cluster. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Problem description

Using the df-h command on the CM node, we found that the space of the / var directory was 50G, but the utilization rate reached 93%, so we went to the / var directory to check it layer by layer, and finally found that the / var/lib/cloudera-scm-server/commands directory occupied an amazing 41G.

Df-h

Du-sh *

Analysis of problems

First of all, let's talk about the purpose of the / var/lib/cloudera-scm-server/commands directory, which is used to store some of the commands executed in the history of CM, as well as the resulting result sets. The default storage period of this directory is 2 years, that is, if enough historical commands and result sets are executed within 2 years, especially if the cluster is large enough, the entire / var directory may be filled, so how to modify the default address and duration of this directory?

If the browser you use is Chinese by default, go to CM > manage > Settings > search the local data storage directory to modify the default directory, or select Advanced in the settings to see

If the browser you are using is English by default, search for Local Data Storage Directory to modify the default directory in CM > adminstration > Settings > or select Advanced in the settings

In order to prevent the log directories of each node in the cluster from taking up too much space, in addition to the parameters described above, it is recommended to configure various types of log directories and the size or duration of retained logs through CM according to the situation of the cluster.

Thank you for reading! On the "CDH cluster var directory occupies a large space example analysis" this article is shared here, I hope the above content can be of some help to you, so that you can learn more knowledge, if you think the article is good, you can share it out for more people to see it!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.