RRD Monitoring tool for Disk usage

Last of my post was presenting a BASH script which monitored the usage of a FlexLM license server.
Here is how to monitor disk usage of some users' personal folders. For that we will use 2 scripts.
These are the crontab entries :




0 0 * * * /root/scripts/taille_utilisateurs
0 20 * * * /root/scripts/creation_graphs

I run the second script 20mn after the first one; it's function of the time that the first script will take to run. I recommend the first time you run the "taille_utilisateurs" script to "time" it and be sure to schedule the second one AFTER all the "du -s" of all folders is done.

Here is the first script "taille_utilisateurs" :
#!/bin/bash

stockage="/backup/vol2/rrd"
users_windows="/backup/vol1/home"
users_unix="/backup/vol2/users"
rep_logs="/root/logs"

creation_graphs(){
for i in `ls $1`
do
user=$(echo $i | sed -e 's/\.old*//')
user=$(echo $user | sed -e 's/\.//')
if [ ! -f $stockage/$2/taille_$user.rrd ]
then
rrdtool create $stockage/$2/taille_$user.rrd -s 86400 -b 1157782169 \
DS:$user:GAUGE:100000:U:U \
RRA:AVERAGE:0.5:1:150
echo "Creation du RRD "$stockage/$2/taille_$user.rrd >> $logfile
fi
done
}

maj_rrd(){
for i in `ls $1`
do
# On calcule la taille du repertoire, puis on met a jour le RRD correspondant avec la valeur
user=$(echo $i | sed -e 's/\.old*//')
user=$(echo $user | sed -e 's/\.//')
taille=$(du -sb $1/$i | awk '{print $1}')
rrdtool update $stockage/$2/taille_$user.rrd N:$taille
echo "Mise a jour du RRD" $stockage/$2/taille_$user.rrd "avec la valeur " $taille "le " `date` >> $logfile
echo $taille" "$i >> $rep_logs/tailles
done

}

logfile=$rep_logs/log-`date "+%d-%m-%Y"`
touch $logfile
echo "Debut du script de calcul des tailles le" `date` > $logfile
creation_graphs $users_unix "unix"
creation_graphs $users_windows "windows"
echo "UNIX" > $rep_logs/tailles
maj_rrd $users_unix "unix"
echo "WINDOWS" >> $rep_logs/tailles
maj_rrd $users_windows "windows"

This first script creates a RRD file per user/folder (150 values stocked, 1 every 24 hours, meaning half a year archive) and then updates the value of this file. Everything is logged in a file, and the sizes of all folders are written in the $rep_logs/tailles file. This file will be used by the second script to do a "Top10" of the most disk-consuming users.

Here is the second script, "creation_graphs" :
#!/bin/bash

stockage="/backup/vol2/rrd"
users_windows="/backup/vol1/home"
users_unix="/backup/vol2/users"
rep_logs="/root/logs"
rep_img="/var/www/html/stats"

top10(){
ligne_win=$(grep -in windows /root/logs/tailles | awk -F':' '{print $1}')
#On trace le Top10 unix
echo "rrdtool graph $rep_img/top10_unix.png \\" > /tmp/graph.sh
echo "-s \"now -4 week\" -e now \\" >> /tmp/graph.sh
echo "--title=\"Top 10 des tailles disque Home UNIX\" \\" >> /tmp/graph.sh
echo "--vertical-label=Octets \\" >> /tmp/graph.sh
echo "--imgformat=PNG \\" >> /tmp/graph.sh
echo "--width=800 \\" >> /tmp/graph.sh
echo "--base=1000 \\" >> /tmp/graph.sh
echo "--height=600 \\" >> /tmp/graph.sh
echo "--interlaced \\" >> /tmp/graph.sh
rang=10
declare -ar couleurs='([0]="#FF0000" [1]="#FF6347" [2]="#FF8C00" [3]="#FF00FF" [4]="#DDA0DD" [5]="#9ACD32" [6]="#008000" [7]="#0000FF" [8]="#6A5ACD" [9]="#48D1CC")'
for i in $(head -$ligne_win /root/logs/tailles | sort -n | tail -10 | awk '{print $2}')
do
user=$i
indice=$(expr $rang - 1)
couleur=${couleurs[$indice]}
echo "DEF:$user=$stockage/unix/taille_$user.rrd:$user:AVERAGE \\" >> /tmp/graph.sh
echo "LINE1:"$user$couleur":\""$rang") Taille de $user\" \\" >> /tmp/graph.sh
echo "VDEF:"$user"_MAX="$user",MAXIMUM \\" >> /tmp/graph.sh
rang=$(expr $rang - 1)
done
. /tmp/graph.sh

#Puis le top 10 windows
echo "rrdtool graph $rep_img/top10_windows.png \\" > /tmp/graph.sh
echo "-s \"now -4 week\" -e now \\" >> /tmp/graph.sh
echo "--title=\"Top 10 des tailles disque Home WINDOWS\" \\" >> /tmp/graph.sh
echo "--vertical-label=Octets \\" >> /tmp/graph.sh
echo "--imgformat=PNG \\" >> /tmp/graph.sh
echo "--width=800 \\" >> /tmp/graph.sh
echo "--base=1000 \\" >> /tmp/graph.sh
echo "--height=600 \\" >> /tmp/graph.sh
echo "--interlaced \\" >> /tmp/graph.sh
rang=10
declare -ar couleurs='([0]="#FF0000" [1]="#FF6347" [2]="#FF8C00" [3]="#FF00FF" [4]="#DDA0DD" [5]="#9ACD32" [6]="#008000" [7]="#0000FF" [8]="#6A5ACD" [9]="#48D1CC")'
for i in $(tail -$ligne_win /root/logs/tailles | sort -n | tail -10 | awk '{print $2}')
do
user=$i
indice=$(expr $rang - 1)
couleur=${couleurs[$indice]}
echo "DEF:$user=$stockage/windows/taille_$user.rrd:$user:AVERAGE \\" >> /tmp/graph.sh
echo "LINE1:"$user$couleur":\""$rang") Taille de $user\" \\" >> /tmp/graph.sh
echo "VDEF:"$user"_MAX="$user",MAXIMUM \\" >> /tmp/graph.sh
rang=$(expr $rang - 1)
done
. /tmp/graph.sh
}

top10

This second script generates a RRD graph corresponding to the Top10 most disk-consuming users.
In order to do this TOP10, we parse the $rep_logs/tailles file.
It's up to you to adapt this to do TOP10-20 TOP20-30 and so on.

As usual, an image of the final RRD graph is at the beginning of this post.

Have fun !