Grafana: monitor metrics collected by collectd into InfluxDB (English)

2014-10-05

  english    monitoring 

Grafana

Actually, the modern tool to render beautiful charts is Grafana.

I will explain in this article how to install a stack using collectd + InfluxDB + Grafana.

The idea is to have collectd that will collect us metrics on the system and tools installed on. These metrics will be added into InfluxDB database and Grafana will render them with amazing real-time charts.

To do that, we also will need a third tool: a proxy that wil listen collectd events and insert metrics into InfluxDB.

Collectd installation

On Debian, simply run the following command (elsewhere, you can go on https://collectd.org/download.shtml web page to obtain the package):


$ sudo apt-get install collectd

Edit configuration file (/etc/collectd/collectd.conf) in order to load plugins you want to obtain metrics on and enable particularly the network plugin that will allow us to connect with the proxy tool later:


# Uncomment this:
LoadPlugin network

# Uncomment/update this:
<Plugin network>
  Server "127.0.0.1" "8096"
</Plugin>

InfluxDB installation

On Debian (64 bits), execute the following commands (elsewhere, go on http://influxdb.com/download/ to obtain package):


$ wget http://s3.amazonaws.com/influxdb/influxdb_latest_amd64.deb
$ sudo dpkg -i influxdb_latest_amd64.deb

If you want to update configuration, you can edit the configuration file at the following location: /opt/influxdb/current/config.toml.

InfluxDB comes with an HTTP API listening on the port 8086 and a web administration interface on port 8083.

collectd / InfluxDB proxy installation

This proxy is written in Go and is available here: https://github.com/hoonmin/influxdb-collectd-proxy

Ensure that you have an up-to-date Go library before typing the following commands:


$ git clone https://github.com/hoonmin/influxdb-collectd-proxy.git
$ cd influxdb-collectd-proxy
$ make

Once installation is finished, we can start the tools: collectd and InfluxDB:


$ /etc/init.d/collectd start
$ /etc/init.d/influxdb start

Now, we will create the "collectd" database into InfluxDB in which all metrics collected by collectd will be inserted:


$ curl "http://localhost:8086/db?u=root&p=root" -d "{\\"name\\": \\"collectd\\"}"

Please note that root password can be updated using the web interface (port 8083).

We will launch the proxy into a screen named "collectd-influxdb".


$ screen -s collectd-influxdb
$ bin/proxy --influxdb="localhost:8086" --typesdb="/usr/share/collectd/types.db" --database="collectd" --username="root" --password="root"

For your information, the following options are also available with this proxy:


$ bin/proxy --help
Usage of bin/proxy:
  -database="": database for influxdb
  -influxdb="localhost:8086": host:port for influxdb
  -logfile="proxy.log": path to log file
  -normalize=true: true if you need to normalize data for COUNTER types (over time)
  -storerates=true: true if you need to derive rates from DERIVE types
  -password="root": password for influxdb
  -proxyhost="0.0.0.0": host for proxy
  -proxyport="8096": port for proxy
  -typesdb="types.db": path to Collectd's types.db
  -username="root": username for influxdb
  -verbose=false: true if you need to trace the requests

In order to verify that the collectd metrics are correctly inserted into the InfluxDB database, you can run the following cURL command:


$ curl -G 'http://localhost:8086/db/collectd/series?u=root&p=root&q=list+series&pretty=true'

If you want to know more about the InfluxDB query language, you can go here: http://influxdb.com/docs/v0.8/api/query_language.html.

Grafana installation

Go o:


$ wget http://grafanarel.s3.amazonaws.com/grafana-1.8.1.tar.gz
$ tar xzvf grafana-1.8.1.tar.gz
$ mv grafana-1.8.1 grafana
$ cd grafana
$ mv config.sample.js config.js

We will now edit the Grafana configuration file in order to specify our datasource: InfluxDB (and particularly our database name: "collectd") :


    datasources: {
      collectd: {
        type: 'influxdb',
        url: "http://yourdomain.com:8086/db/collectd",
        username: 'root',
        password: 'root'
      },
    },

Please be careful about specifying a publicly accessible URL because requests will be made by browser client (using Javascript).

Finally, go on http://yourdomain.com/grafana and start adding your charts using your data collected by collectd and retrieved from InfluxDB.

Comments