How to Add a CentOS 7 Sensu Client

In the guide "How to Install and Configure Sensu Monitoring on CentOS 7", you learned how to setup a Sensu server. This tutorial is the second part, where you will learn how to add a Sensu client. A Sensu client is a server that Sensu monitors.

I assume that the Sensu client is a CentOS 7 virtual machine. Log into SSH and follow the steps below. You won't have to alter anything on the Sensu server. As long as the Sensu client successfully authenticates with the Sensu server, checks will automatically be executed.

Before we get started, make sure that your firewall is configured correctly. With a default setup, open outgoing TCP ports 5671 and 5672.

Step 1: Add the Sensu repository

The package sensu-client is not in the default CentOS repositories. Add the repository by pasting the following content into the file /etc/yum.repos.d/sensu.repo:

[sensu]
name=sensu
baseurl=https://sensu.global.ssl.fastly.net/yum/$releasever/$basearch/
gpgcheck=0
enabled=1

Next, update yum and install sensu-client:

yum update
yum install sensu

All services (sensu-server, sensu-client and sensu-api) will be installed by installing the sensu package. You can safely ignore all services except for the sensu-client that we will be using. Do not start the service yet.

Step 2: Install necessary plugins

Sensu's model is to distribute load over clients that results from executing checks and pushing check results. As a result, you need to install Sensu plugins on the clients themselves rather than on the Sensu server. Some plugins are compiled locally, which means we'll need build tools on the clients:

yum groupinstall "Development tools"

The following plugins are very popular and used for many checks. Execute each of these commands to install the plugins once the development tools packages have been installed. Depending on the size of your server, this might take a while:

sensu-install -p uptime-checks
sensu-install -p process-checks
sensu-install -p http
sensu-install -p filesystem-checks
sensu-install -p cpu-checks
sensu-install -p memory-checks
sensu-install -p disk-checks
sensu-install -p load-checks
sensu-install -p network-checks

Step 3: Add configuration files

We will be adding four configuration files to the Sensu client.

• /etc/sensu/conf.d/client.json:

  This file contains subscriptions, the IP address for the Sensu client and the client name. Replace the values ipaddress and hostname with values that describe the client you are adding. Also, specify subscriptions defined on the Sensu server or, if you do not have any, remove the entire subscriptions section. Don't forget to remove the comma after name if you do:

   {
   	"client": {
   		"address": "-----ipaddress-----",
   		"environment": "production",
   		"name": "-----hostname-----",
   		"subscriptions": [
   			"linux"
   		]
   	}
   }

• /etc/sensu/conf.d/transport.json:

  In this file, we define the transport used. By default, the transport used is RabbitMQ. Paste the following contents into the file. No values have to be changed:

   {
   	"transport": {
   		"name": "rabbitmq",
   		"reconnect_on_error": true
   	}
   }

• /etc/sensu/conf.d/rabbitmq.json:

  The RabbitMQ server and credentials are specified in this file. Replace hostname with the DNS name. By default, the port is 5672 for unencrypted connections. The RabbitMQ user and password should be replaced as well. These details can differ for each Sensu client or be the same for all clients. This is a practical decision that is made by the user.

  The vhost is / by default but it is often changed to /sensu. If you're not sure which to use, try both:

   {
   	"rabbitmq": {
   		"host": "-----hostname-----",
   		"password": "-----rabbitmq_password-----",
   		"port": 5672,
   		"user": "-----username-----",
   		"vhost": "/sensu"
   	}
   }

• /etc/sensu/conf.d/checks.json:

  Sensu supports two types of checks: standalone checks and subscription checks. Subscription checks are defined on the Sensu server and the subscriptions are defined in the client.json file on a Sensu client. Standalone checks are checks that are defined on the Sensu clients themselves. These are usually checks that only apply to one specific server. For example, a check that would be in a subscription is a check to see if the web server is still running. A standalone check could be a check to see if server A can reach server B. No other server needs that check.

  Standalone checks are defined in this file. Below you will find an example of a checks.json file with some example standalone checks:

   {
   	"checks": {
   		"httpd": {
   			"command": "sudo check-process.rb -p httpd",
   			"handlers": [
   				"default",
   				"pushover"
   			],
   			"interval": 180,
   			"occurences": 3,
   			"refresh": "1800",
   			"standalone": true,
   			"ttl": 1200
   		},
   		"mysqld": {
   			"command": "sudo check-process.rb -p mysqld",
   			"handlers": [
   				"default",
   				"pushover"
   			],
   			"interval": 180,
   			"occurences": 3,
   			"refresh": "1800",
   			"standalone": true,
   			"ttl": 1200
   		}
   	}
   }

After you have finished configuration of the Sensu client, change the rights:

chown -R sensu:sensu /etc/sensu/conf.d

Step 4: Give permissions to the sensu user

Checks are executed as the user sensu. Naturally, this user has limited rights. However, some checks need root permissions to execute. We will allow the user sensu to execute any check as root but limit its other permissions.

Paste the following contents into the file /etc/sudoers.d/sensu:

Defaults:sensu
secure_path=/opt/sensu/embedded/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
 
sensu ALL=(ALL) NOPASSWD: /opt/sensu/embedded/bin/check*.rb *
sensu ALL=(ALL) NOPASSWD: /opt/sensu/embedded/bin/metrics*.rb *

The first two lines define the default path which allows you to specify only the script name in the check command rather than the entire path. For example, simply using the command sudo check-process.rb -p mysqld will function without having to specify the entire path (/opt/sensu/embedded/bin/check-process.rb -p mysqld).

The other lines allow the sensu user to execute any check script as root.

Finally, start the sensu-client service and automatically start it at boot:

systemctl start sensu-client
systemctl enable sensu-client

If you have Uchiwa installed, the new client should appear in the list. If not, debug the sensu-client by inspecting the log file /var/log/sensu/sensu-client.log