This article will explain what you can do with Shorewall, a gateway/firewall configuration tool for GNU/Linux, and graphite/Grafana.

Shorewall logo

Shorewall is a gateway/firewall configuration tool for GNU/Linux. It has a very good documentation and is very easy to configure.

In the Shorewall configuration, you can create packet and byte counters with accounting rules so you can gather data precisely on what you need. An interesting option is to use the xtables netfilter addon who will create per IP metrics.

We will add a loc-net table in the /etc/shorewall/accounting config file, handling lan to net and net to lan traffic (rules are not stateful):

#ACTION					CHAIN	SOURCE	DEST	PROTO   DPORT   SPORT   USER    MARK    IPSEC
ACCOUNT(loc-net,x.x.0.0/20)		-	eth2    eth1
ACCOUNT(loc-net,x.x.0.0/20)		-	eth1    eth2

We can then print counters with the shorewall show ipa command:

host:~# shorewall show ipa
Shorewall 4.6.4.3 per-IP Accounting at host - 

Showing table: loc-net
IP: x.x.0.4 SRC packets: 480774 bytes: 49865249 DST packets: 0 bytes: 0
IP: x.x.1.100 SRC packets: 33941 bytes: 1899866 DST packets: 43 bytes: 8562
IP: x.x.2.47 SRC packets: 5736 bytes: 1204376 DST packets: 0 bytes: 0
IP: x.x.2.51 SRC packets: 608394 bytes: 139710104 DST packets: 0 bytes: 0
IP: x.x.2.60 SRC packets: 141483 bytes: 30495738 DST packets: 0 bytes: 0
IP: x.x.2.69 SRC packets: 26145 bytes: 3335664 DST packets: 0 bytes: 0
IP: x.x.2.78 SRC packets: 285086 bytes: 68623864 DST packets: 0 bytes: 0
IP: x.x.2.81 SRC packets: 163715 bytes: 11279400 DST packets: 0 bytes: 0
...

Great ! All we have to do now is to send this data to graphite and we will write a small python script for that.

We will use the daemonize library to run our process in the background (you can install it with pip install daemonize).

You can launch it with the systemd shorewall-ipa-graphite.service unit file.

Finally, we will create a Grafana dashboard with some Collectd metrics (ping and network interfaces) and our shorewall-ipa-graphite.py script metrics in a top ten consumers graphic:

shorewall-grafana

Here is the Metric configuration line, with a Transform: negative-Y series specific overrides on /tx.*/ to separate in and out:

aliasSub(absolute(derivative(consolidateBy(sortByMaxima(highestCurrent(firewall.hostname.accounting.perip.*.*_bytes, 10)), 'max'))), '.*perip\.([0-9]*)_([0-9]*)_([0-9]*)_([0-9]*)\.([a-z]*).*', '\5.\3.\4')

Thanks to Eric Leblond for his posts using-ulogd-and-json-output and logging-connection-tracking-event-with-ulogd.