Shulou(Shulou.com)06/01 Report--

This article introduces how to use Netdata to set up real-time performance monitoring in Ubuntu. The content is very detailed. Interested friends can use it for reference. I hope it will be helpful to you.

Netdata provides accurate performance monitoring through an extensible Web dashboard that displays processes and services on Linux systems. It monitors metrics such as CPU, memory, disk, network, process, etc.

Once installed, Netdata does not require additional configuration, but provides important customizations. The application is designed to be as efficient and fast as native console management tools such as vmstat,iostat and htop.

The steps in this tutorial cover everything you need to successfully set up a Ubuntu 16.04 server running Netdata using its built-in Web server or optionally using Nginx.

Prepare for

An Ubuntu 16.04 server, including non-root users with sudo privileges.

Install Git on the server.

(optional) install Nginx on the server. This is not mandatory, but allows more functionality and security than Netdata's built-in Web server.

Step 1-install Netdata

Before we install anything, make sure that the system package index is up to date.

$sudo apt-get update

Next, install Netdata's dependencies, including gcc (a C compiler), GNU Autoconf tools, GUID management, and the compression library for Netdata's internal Web server.

$sudo apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make autoconf autoconf-archive autogen automake pkg-config curl

The next set of packages is optional, but Netdata recommends it, including Python, some Python packages, and Node.JS. The stable version of Node.js bundled with the system package Manager is suitable for the requirements of Netdata. Next, install them.

$sudo apt-get install python python-yaml python-mysqldb python-psycopg2 nodejs lm-sensors netcat

To install Netdata itself, we must use the project's GitHub repository. Clone the Netdata repository to your home directory.

$git clone https://github.com/firehol/netdata.git-- depth=1 ~ / netdata

Move to the directory of the new clone.

$cd ~ / netdata

Now use the netdata-installer.sh shell script in this directory to build and install the application. Be sure to attach the sudo here, otherwise the way Netdata collects system data (through the data collector) will not work properly.

$sudo. / netdata-installer.sh

The output you will initially see contains information about where Netdata will store all its components. You can read through these so that you can be more familiar with how the program is deployed on the file system after installation.

Installer Output. . . It will be installed at these locations:-the daemon at / usr/sbin/netdata-config files at / etc/netdata-web files at / usr/share/netdata-plugins at / usr/libexec/netdata-cache files at / var/cache/netdata-db files at / var/lib/netdata-log files at / var/log/netdata-pid file at / var/run. . .

Press ENTER to continue the installation. After a moment, at the end of the output, you will see the following message:

Installer Output. . . ^ |. Netdata. -..-|'-'is installed and running now! -'+-> enjoy real-time performance and health monitoring...

Netdata is now successfully installed and running, and will run automatically when the server is restarted. If you enable UFW, you need to open port 19999 for Netdata's internal Web server.

$sudo ufw allow 19999/tcp

At this point, you can view the default dashboard by visiting http:// your_server_ip:19999/ in your favorite browser. You will see an overview of the real-time metrics of the system.

Step 2-configure memory usage for Netdata

You can view the current configuration of Netdata by visiting http://your_server_ip:19999/netdata.conf in your browser.

You will see here that all (or most) of the configurable options have been commented out (that is, starting with #). This is because the configuration of Netdata uses a set of assumed default values. Any disabled setting uses the default value of Netdata; if you uncomment a setting, the specified value overrides the default value. This makes the configuration file contain only what you modify.

Back on your server, the configuration file itself is / etc/netdata/netdata.conf. The option we will customize in this tutorial is the history parameter. It controls the size of the in-memory database used by Netdata.

Here, you need to determine how much RAM to provide for Netdata, or how long to keep the recorded chart data before losing it:

3600 seconds (chart data is retained for 1 hour) using 15 MB RAM

7200 seconds (chart data is retained for 2 hours) using 30 MB RAM

14400 seconds (chart data is retained for 4 hours) using 60 MB RAM

28800 seconds (chart data is retained for 8 hours) using 28800 MB RAM

43200 seconds (chart data is retained for 12 hours) using 180 MB RAM

86400 seconds (24-hour chart data retention) using 86400 MB RAM

Keep in mind that the above estimate is based on the number of charts used by the inventory dashboard. Deleting or adding custom charts in the future will affect these estimates.

Open the main configuration file for Netdata using nano or your favorite text editor.

$sudo nano / etc/netdata/netdata.conf

Locate the history option in the [global] section.

/ etc/netdata/netdata.conf

The copy code is as follows:

. . . [global] # glibc malloc arena max for plugins = 1.. . # hostname = test-netdata # history = 3600 # update every = 1.. .

Replace the 3600 value with the value you previously determined. Here, we are using 14400, which provides us with 4 hours of data retention. Be sure to uncomment the line by dividing the # symbol so that Netdata no longer ignores this option.

/ etc/netdata/netdata.conf

. . . [global] # glibc malloc arena max for plugins = 1.. . # hostname = test-netdata # history = 3600 # update every = 1.. .

Save and close the file after making this change. While this change may increase the amount of RAM that Netdata will use, the next change we will implement in the next step should significantly reduce it.

Step 3-enable kernel same page merging

Even if Netdata runs directly and is used initially, we can make more changes to the way Netdata uses system resources, which will speed up and optimize its performance. We will start by enabling kernel same page merging (KSM for short). Netdata developers estimate that this will reduce Netdata memory usage by 40-60%.

When enabled, the KSM daemon periodically scans memory for pages of the same content that can be replaced by a single write-protected page. In this context, it allows the same memory pages to be shared between different processes or programs running on the system. This reduces the creation of unnecessary duplicate memory content.

To permanently enable this aspect of the Linux system kernel, open the / etc/rc.local file using a text editor.

The copy code is as follows:

$sudo nano / etc/rc.local

After all other normal Linux system services and processes are started, the / etc/rc.local file or run control file is executed. It is useful for specifying custom services-or, in our example, to enable KSM while the server is running.

Add the two commands shown below to the end of the file, before the last line exit 0, as follows:

/ etc/rc.local

#! / bin/sh-eBay # rc.local## This script is executed at the end of each multiuser runlevel.# Make sure that the script will "exit 0" on success or any other# value on error.## In order to enable or disable this script just change the execution# bits.## By default this script does nothing.echo 1 > / sys/kernel/mm/ksm/runecho 1000 > / sys/kernel/mm/ksm/sleep_millisecsexit 0

The first command echo 1 > / sys/kernel/mm/ksm/run adds 1 to the runKSM kernel file, which enables this feature. The second command, echo 1000 > / sys/kernel/mm/ksm/sleep_millisecs, tells the KSM daemon to run once per second and evaluate 100 pages for deduplication.

After you save and close the file, KSM will now be enabled the next time the server is restarted and will be retained for future restarts. To enable this server during its current uptime, you need to manually run the two commands you added to rc.local on the command line.

First, start and enter a new shell session as the root user of the server by using the-s flag in sudo. This is necessary because two early commands use output redirection (through the > operator) and need to be executed by root shell to succeed.

$sudo-s

Now enter the previous two commands:

Echo 1 > / sys/kernel/mm/ksm/runecho 1000 > / sys/kernel/mm/ksm/sleep_millisecs

Then, keep the root shell and return to the normal shell session.

$exit

Finally, to apply all the changes we made in this and previous steps, we must restart Netdata.

$sudo systemctl restart netdata

You can verify that KSM is now enabled by looking for the newly added active KSM chart in the Netdata dashboard. This can be found under Memory > Memory Deduper in the menu tree on the right.

KSM Chart Image

Now that we know that the dashboard and KSM are running, it's worth accessing the dashboard through a Web server like Nginx rather than the built-in Web server. This is not entirely necessary, but it is recommended, so follow the next step to set it up.

Step 4-hosting the dashboard via Nginx (optional)

Hosting Netdata through Nginx makes it easier to secure access to dashboards and, if necessary, allows you to point to Netdata dashboards on other servers in the future.

You also need to install the apache2-utils package. This package contains the htpasswd program, which we will need later to secure access to the dashboard page.

$sudo apt-get install apache2-utils

Next, create a new Nginx host profile. Note that here we use a file in the / conf.d directory, but if you use Nginx for something other than Netdata, you can use / sites-available and / sites-enabled.

$sudo nano / etc/nginx/conf.d/default.conf

This profile will tell the Nginx agent the incoming request for the dashboard page. We will also add a simple authentication prompt so that only people with the correct user name and password can access it.

Here is the entire configuration file that you want to copy and paste. Change the two red highlights of the above listen and server_name directives to the server's IP address and the assigned domain name, respectively. Note that you include; at the end of each. If you don't have a domain name, you can leave the server_name directive as example.com.

/ etc/nginx/conf.d/default.conf

Upstream netdata-backend {server 127.0.0.1 keepalive 1999; keepalive 64;} server {listen your_server_ip:80; server_name example.com; auth_basic "Authentication Required"; auth_basic_user_file netdata-access; location / {proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://netdata-backend; proxy_http_version 1.1 Proxy_pass_request_headers on; proxy_set_header Connection "keep-alive"; proxy_store off;}}

Once you have determined that your file matches this block of code, you can save and exit to return to the command prompt.

Let's explain this file so that you can understand what it is doing. This is the first part:

/ etc/nginx/conf.d/default.conf

Upstream netdata-backend {server 127.0.0.1 keepalive 1999; keepalive 64;}

This upstream module called netdata-backend uses the server's loopback address 127.0.0.1 and Netdata port 19999 to locate the built-in Netdata Web server. The keepalive directive sets the maximum number of upstream idle connections that can be kept open at any given time (for each Nginx worker process). The proxy_pass directive requires this upstream module definition later.

Directly after that is the main server block.

/ etc/nginx/conf.d/default.conf

Upstream netdata-backend {server 127.0.0.1 keepalive 1999; keepalive 64;}

The first two lines in the block define the external IP address that Nginx should listen when the client tries to connect. The server_name directive tells Nginx to run this server block when the client uses the specified domain name.

The last two lines of this code snippet set up simple HTTP username and password authentication. It uses the auth_basic module to display user name and password prompts. You can customize the prompt message

Here, it is set to Authentication Required. Auth_basic_user_file specifies the file name and location of the login credentials used to store authentication prompts, which you will create later in this step.

The last location block is nested within the server block, processing the agent and passing the incoming request to the Nginx.

/ etc/nginx/conf.d/default.conf

Server {listen your_server_ip:80; server_name example.com; auth_basic "Authentication Required"; auth_basic_user_file netdata-access

All in all, this code is the code that passes the client request to the Netdata Web server, and then the dashboard. The upstream module that the proxy_pass instruction line references back to the beginning of the file must point to it by the name netdata-backend.

If it is not already installed, you can save and close this profile. This is where the htpasswd program in the apache2-utils package helps us create user login credentials for dashboard login prompts.

First, we will create a user name and password. Run the following command to replace sammy with the user name you want to use in Netdata. When prompted, enter the password you want to use.

$sudo htpasswd-c / etc/nginx/netdata-access sammy

This will create a file in / etc / nginx / netdata-access that contains the user name and password you provided.

The generated password is encrypted in the file, so it cannot be treated as plain text.

Restart Nginx to make the new configuration we added active.

$sudo systemctl restart nginx

To test the Nginx configuration for the first time, visit http://your_server_ip in your favorite Web browser. You will see an authentication prompt. Enter the user name and password you provided earlier in this step to access the dashboard, which we will discuss in step 5.

Step 5-explore the dashboard

If you do not follow the optional step 4, please visit http://your_server_ip:19999/ in your favorite browser immediately to access the dashboard.

You will see an overview of dashboards and system real-time metrics. The HUD style indicator at the top of the page changes when you hover over each period of time in the subsequent drawing.

HUD Indicators Image

The graphs and charts in this overview section provide a detailed description of each system, ranging from CPU to memory to network traffic.

CPU Chart Example

The chart is interactive and you can use the mouse button to drag to the left or right to pan back and forth at different intervals.

RAM Chart Example

Hold down SHIFT and use the mouse wheel to scroll or scroll out of the drawing to zoom out or widen the time stamp. Double-click the chart to reset it to the default appearance and view.

Network Chart Example

The fastest way to navigate through the dashboard is to use the menu tree on the right side of the page. This changes the focus and color based on the part of the page you are currently viewing.

Menu Tree

Scroll down the page or use the menu tree to access a more in-depth chart. These are very detailed and controlled in the same way as the charts in the initial overview section.

Menu Tree Image

Netdata provides many of these additional statistics-specific charts.

An important part of GUI is updating the page. Netdata receives updates periodically and keeps your installation up to date. The top of the dashboard contains an Update button that can be used to check whether the new version is available for download.

Update Button Image

Clicking this button opens a new menu with more details and opens the check now button to manually check for updates.

Update Check Details Image

If updates are available, you only need to run the provided update script from the Netdata Git repository, which we cloned to the home directory of the Linux user in the first step. That is, when updates are available, just run sudo ~ / netdata/netdata-updater.sh from the command line.

On how to use Netdata in Ubuntu to set up real-time performance monitoring is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.