think tank forum

technology » monitoring disks

lucas's avatar
14 years ago
r1, link
lucas
i ❤ demo
does anyone here use scripts to monitor disk usage?

it would be nice to be alerted via email when the capacity of /dev/stripe/st0a goes above 90%.

also, it would be nice to alerted via email if gmirror or gstripe statuses change.

--

bsdnexus cross-post
 
14 years ago
r2, link
arun
keep smiling !
Something like

df | awk '/st0a/ { sub(/%/,"",$5); $5 >= 90 ? system("mail -s \"st0a < 90%\" me@mydomain.com > /dev/null"): 0 }'


would work.
lucas's avatar
14 years ago
link
lucas
i ❤ demo
and just run it daily via cron?
 
14 years ago
link
arun
keep smiling !
yep
lucas's avatar
14 years ago
r2, link
lucas
i ❤ demo
how do you feel about this script?

#!/bin/sh

HOSTNAME=`hostname`;
HOME_USAGE=`df | awk '/usr\/home/ { sub(/%/,"",$5); print $5}'`;
GMIRROR_STATUS=`/sbin/gmirror status | grep "^mirror" | awk {'print $2'} | sort -u`;

if [ $HOME_USAGE -ge 90 ]; then

    df -h | mail -s "$HOSTNAME: /usr/home usage high" root

fi

if [ "$GMIRROR_STATUS" != 'COMPLETE' ]; then

    /sbin/gmirror status | mail -s "$HOSTNAME: raid array incomplete" root

fi
lucas's avatar
14 years ago
r2, link
lucas
i ❤ demo
it seems to work well enough:

To: root@euterpe.wingedleopard.net
Subject: euterpe.wingedleopard.net: raid array incomplete
Message-Id: <20090816100000.982DD409D@euterpe.wingedleopard.net>
Date: Sun, 16 Aug 2009 04:00:00 -0600 (MDT)
From: lucas@euterpe.wingedleopard.net (▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇)

      Name    Status  Components
mirror/gm0  COMPLETE  ad4
                      ad6
mirror/gm1  DEGRADED  ad8
                      ad10 (4%)
lucas's avatar
14 years ago
link
lucas
i ❤ demo
i deactivated ad10 to test the system.

now it's going to take an hour to sync up, which means i'll have thirty emails waiting for me in the morning (i'm running that script via cron every five minutes).

:)
lucas's avatar
14 years ago
link
lucas
i ❤ demo
next level?
lucas's avatar
14 years ago
r2, link
lucas
i ❤ demo
i just got my first non-test (production) alert!

To: root@euterpe.wingedleopard.net
From: lucas@euterpe.wingedleopard.net (▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇)
Date: Sat, 12 Sep 2009 21:30:01 -0600 (MDT)
Subject: euterpe.wingedleopard.net: /usr/home usage high

Filesystem          Size    Used   Avail Capacity  Mounted on
/dev/ad0s1a         496M    252M    204M    55%    /
devfs               1.0K    1.0K      0B   100%    /dev
/dev/ad0s1d         496M     32K    456M     0%    /tmp
/dev/ad0s1f          17G    3.1G     12G    20%    /usr
/dev/ad0s1e         496M     29M    427M     6%    /var
/dev/stripe/st0a    451G    371G     44G    90%    /usr/home


:)

time to buy some disks..
lucas's avatar
14 years ago
link
lucas
i ❤ demo
coupon code for 10% off disks on newegg.com

EMCHDD10B
lucas's avatar
14 years ago
r1, link
lucas
i ❤ demo
the Rosewill RC-209-EX is $40 on newegg.com. :(

i should just invest in a new mobo (and cpu and ram and psu). sigh
lucas's avatar
13 years ago
link
lucas
i ❤ demo
this is boring. atlas is too reliable. i never get emails from it. :(



atlas% crontab -l
*/05 * * * * ./bin/disk_alerts.sh
atlas% cat ./bin/disk_alerts.sh
#!/bin/sh

HOSTNAME=`hostname`;
HOME_USAGE=`df | awk '/usr\/home/ { sub(/%/,"",$5); print $5}'`;
GMIRROR_STATUS=`/sbin/gmirror status | grep "^mirror" | awk {'print $2'} | sort -u`;

if [ $HOME_USAGE -ge 90 ]; then

    df -h | mail -s "$HOSTNAME: /usr/home usage high" root

fi

if [ "$GMIRROR_STATUS" != 'COMPLETE' ]; then

    /sbin/gmirror status | mail -s "$HOSTNAME: raid array incomplete" root

fi

atlas%
lucas's avatar
13 years ago
r1, link
lucas
i ❤ demo
test emails:

To: root@atlas.wingedleopard.net
Subject: atlas.wingedleopard.net: raid array incomplete
Date: Sun, 30 Jan 2011 01:34:00 -0700 (MST)
From: lucas@atlas.wingedleopard.net (▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇)

      Name    Status  Components
mirror/gm9  COMPLETE  ada0
                      ada1
mirror/gm7  COMPLETE  ada2
                      ada3
mirror/gm5  COMPLETE  ada4
                      ada5


To: root@atlas.wingedleopard.net
Subject: atlas.wingedleopard.net: /usr/home usage high
Date: Sun, 30 Jan 2011 01:34:32 -0700 (MST)
From: lucas@atlas.wingedleopard.net (▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇)

Filesystem            Size    Used   Avail Capacity  Mounted on
/dev/mirror/gm9s1a    496M    317M    139M    70%    /
devfs                 1.0K    1.0K      0B   100%    /dev
/dev/mirror/gm9s1d    496M     22K    456M     0%    /tmp
/dev/mirror/gm9s1e    1.9G     13M    1.8G     1%    /var
/dev/mirror/gm9s1f    221G    1.6G    201G     1%    /usr
/dev/concat/gc9a      1.1T    948G    115G    89%    /usr/home