We cannot change the cards we are dealt, just how we play the hand. - Randy Pausch
How to graph shorewall accounting data
This article will explain what you can do with Shorewall, a gateway/firewall configuration tool for GNU/Linux, and graphite/Grafana.
Shorewall is a gateway/firewall configuration tool for GNU/Linux. It has a very good documentation and is very easy to configure.
In the Shorewall configuration, you can create packet and byte counters with accounting rules so you can gather data precisely on what you need. An interesting option is to use the xtables netfilter addon who will create per IP metrics.
We will add a loc-net table in the /etc/shorewall/accounting config file, handling lan to net and net to lan traffic (rules are not stateful):
We can then print counters with the shorewall show ipa command:
Great ! All we have to do now is to send this data to graphite and we will write a small python script for that.
We will use the daemonize library to run our process in the background (you can install it with pip install daemonize).
You can launch it with the systemd shorewall-ipa-graphite.service unit file.
Finally, we will create a Grafana dashboard with some Collectd metrics (ping and network interfaces) and our shorewall-ipa-graphite.py script metrics in a top ten consumers graphic:
Here is the Metric configuration line, with a Transform: negative-Y series specific overrides on /tx.*/ to separate in and out: