From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oa0-f42.google.com (mail-oa0-f42.google.com [209.85.219.42]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 8655021F196 for ; Thu, 24 Jan 2013 20:44:09 -0800 (PST) Received: by mail-oa0-f42.google.com with SMTP id j1so11151909oag.29 for ; Thu, 24 Jan 2013 20:44:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=i/ofW5oROx1n5ZV8ibVnR5lN1PSY99Jn4F0q+4RR8j4=; b=nhmIp0dEhAiL7qdaKSSRQo3zZa6XkpjItSPgQeOBRcg6HeZXaOUSD0ZsEWvDk/wnWm MjNYKZPiGnQGfq91XuGASFk0Nj4eoFhpXOG7Waazs3m2SEiR+GftFKIGr0od5EocPGoC CC/0PlqHxTXeVrcGtWvjFWUWBbv926oUVUkytYIWUOb45Gaz9zKrCNEK8dbUYhqAkZ+7 9Nn8WiQVEmsO3DTULwb/GBCOr0oLP2Sp02im08HqMD0uaPYpj0K+0ktGTTPrjz9utmrl 7YkUwz3kRUtLAq67UAdxk2gZnxAz+H/gu3K0yMsJ3tDw7SvI07tqWySTSMFxfEX8fkQ2 eM0Q== MIME-Version: 1.0 X-Received: by 10.60.11.67 with SMTP id o3mr3447097oeb.24.1359089048503; Thu, 24 Jan 2013 20:44:08 -0800 (PST) Received: by 10.60.94.239 with HTTP; Thu, 24 Jan 2013 20:44:08 -0800 (PST) In-Reply-To: References: Date: Thu, 24 Jan 2013 20:44:08 -0800 Message-ID: From: Justin Madru To: Sebastian Moeller Content-Type: multipart/alternative; boundary=e89a8fb2066ef445cf04d4159394 Cc: cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] Network device transfer counter loops at 4GB X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jan 2013 04:44:09 -0000 --e89a8fb2066ef445cf04d4159394 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks for the reply. I'll take a look at vnstat. It still seems like making the counters 64bit would be a better out-of-the-box solution, especially since the web-ui is displaying these values which are nearly useless. --Justin On Thu, Jan 24, 2013 at 9:40 AM, Sebastian Moeller wrote: > Hi Justin, > > have a look at vnstat and vnstati if you want to keep a better overview o= f > Rx/Tx volumes over time=E2=80=A6 (Then again, it might not be what you ar= e looking > for) > > best > Sebastian > > > On Jan 24, 2013, at 00:55 , Justin Madru wrote: > > > Hi, > > > > I noticed that the TX/RX counter as reported by ifconfig and in the web > ui loop over at 4GB (2^32). Is this something that could be fixed? > > Just guessing here, but that smells like a 32bit counter which > might be involved to change to 64bit. So you could just periodically read > out the value and compare to your last value to detect overruns (assuming > you sample at a period smaller than the shortest possible roll-over time)= . > Then again this seems what vnstat is doing. > > > Interestingly, the ifb0 device (what ever that is) does not have this > issue. > > > http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb > > > It would be nice to know the amount of data I've transferred at least > for GE00. > > Me =3D broken record: have a look at the vnstat and vnstati packe= d > that are part of the cerowrt repository that Dave maintains (THANKS A LOT= ) > > If you should go the vnstat route here is what I ended up with (nothing > fancy, but it seems to work okay): > > Here is my System->Startup-> local startup (can be pasted into the lucy > text box) > > # Put your custom commands here that should be executed once > # the system init finished. By default this file does nothing. > > #manual vnstat updates... since vnstat logs to tmpfs re-init the database > mkdir /var/lib/vnstat > vnstat -u -i ge00 > > # start the traffic shaping script on startup... > #/usr/sbin/simple_qos.sh > > > # be verbose in dmesg and log > echo 1 1 1 1 > /proc/sys/kernel/printk > # helps unaligned access errors show up in log > #echo 2 > /sys/kernel/debug/mips/unaligned_action > > # to make vnstat robust against reboots (to monitor volume caps) > # this will copy the saved vnstat database over the just initialized (mor= e > or less empty db) > /etc/init.d/vnstatbackup restore > > # minimize the too large tmpfs (to avoid OOM), this is better handled by > adding swap > # might interfere with sys upgrade procedure (default is ram/2 IIRC > mount tmpfs /tmp -t tmpfs -o remount,size=3D16000k,nosuid,nodev > > exit 0 > > > and here is /etc/init.d/vnstatbackup: > > #!/bin/sh > ## Please visit http://wiki.openwrt.org/doc/howto/vnstat > # 20121008sm: automate parsing of vnstat related information > # place this script in /etc/init.d > # make it executable: chmod 755 /etc/init.d/vnstatbackup > # 4 CRON: echo "*/30 * * * * /etc/init.d/vnstatbackup backup" >> > /etc/crontabs/root > # /etc/init.d/cron restart > # 4 INIT add > # /etc/init.d/vnstatbackup restore > # to /etc/rc.local > > > # edit this to point to where you stored the db > BACKUP_BASE_DIR=3D"/home/persistent" # where to store the database > backup > > # for vnstat have a look at /etc/vnstat.conf (code taken from > /etc/init.d/vnstat) > vnstat_option() { > sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^'\"]*\)['\"].*/\1/p" \ > /etc/vnstat.conf > } > VNSTAT_DB_DIR=3D"$(vnstat_option DatabaseDir)" > VNSTAT_BASE_DIR=3D`dirname ${VNSTAT_DB_DIR}` #${VNSTAT_DB_DIR%/*} > VNSTAT_SUB_DIR=3D`basename ${VNSTAT_DB_DIR}` #"vnstat" > > #echo "VNSTAT_DB_DIR ${VNSTAT_DB_DIR}" > #echo "VNSTAT_BASE_DIR ${VNSTAT_BASE_DIR}" > #echo "VNSTAT_SUB_DIR ${VNSTAT_SUB_DIR}" > #echo "BACKUP_BASE_DIR ${BACKUP_BASE_DIR}" > > case $1 in > backup) > cp -r ${VNSTAT_BASE_DIR}/${VNSTAT_SUB_DIR} ${BACKUP_BASE_DIR} > ;; > restore) > cp -r ${BACKUP_BASE_DIR}/${VNSTAT_SUB_DIR} ${VNSTAT_BASE_DIR} > ;; > *) > echo "Please use 'backup' or 'restore' parameter for run"; > ;; > esac > > exit 0 > > Note that I mounted a USB stick under /home to have plenty of space, but > you should also be able to use the persistent overlay part of cerowrt > (where config files and such are stored) but that will stress the preciou= s > flash (a usb stick, by virtue of being replaceable and cheap is just less > precious). > > Finally System->Scheduled Tasks: > > */5 * * * * vnstat -u > */30 * * * * /etc/init.d/vnstatbackup backup > > The first polls the Rx/Tx counters every 5 minutes (if your link is very > fast you might want to reduce that) > The second pushes the vnstat database every 30 minutes to the stick. The > idea here is that you might want vnstat to do something even when the sti= ck > is (accidentally) removed. > > best > Sebastian > > > > > > --Justin Madru > > _______________________________________________ > > Cerowrt-devel mailing list > > Cerowrt-devel@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cerowrt-devel > > --e89a8fb2066ef445cf04d4159394 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for the reply. I'll take a look at vnstat.= It still seems like making the counters 64bit would be a better out-of-the= -box solution, especially since the web-ui is displaying these values which= are nearly useless.

--Justin


=
On Thu, Jan 24, 2013 at 9:40 AM, Sebastian Moell= er <moeller0@gmx.de> wrote:
Hi Justin,

have a look at vnstat and vnstati if you want to keep a better overview of = Rx/Tx volumes over time=E2=80=A6 (Then again, it might not be what you are = looking for)

best
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Sebastian


On Jan 24, 2013, at 00:55 , Justin Madru wrote:

> Hi,
>
> I noticed that the TX/RX counter as reported by ifconfig and in the we= b ui loop over at 4GB (2^32). Is this something that could be fixed?

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Just guessing here, but that smells like = a 32bit counter which might be involved to change to 64bit. So you could ju= st periodically read out the value and compare to your last value to detect= overruns (assuming you sample at a period smaller than the shortest possib= le roll-over time). Then again this seems what vnstat is doing.

> Interestingly, the ifb0 device (what ever that is) does not have this = issue.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 http://www.linuxf= oundation.org/collaborate/workgroups/networking/ifb

> It would be nice to know the amount of data I've transferred at le= ast for GE00.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Me =3D broken record: have a look at the = vnstat and vnstati packed that are part of the cerowrt repository that Dave= maintains (THANKS A LOT)

If you should go the vnstat route here is what I ended up with (nothing fan= cy, but it seems to work okay):

Here is my System->Startup-> local startup (can be pasted into the lu= cy text box)

# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

#manual vnstat updates... since vnstat logs to tmpfs re-init the database mkdir /var/lib/vnstat
vnstat -u -i ge00

# start the traffic shaping script on startup...
#/usr/sbin/simple_qos.sh


# be verbose in dmesg and log
echo 1 1 1 1 > /proc/sys/kernel/printk
# helps unaligned access errors show up in log
#echo 2 > /sys/kernel/debug/mips/unaligned_action

# to make vnstat robust against reboots (to monitor volume caps)
# this will copy the saved vnstat database over the just initialized (more = or less empty db)
/etc/init.d/vnstatbackup restore

# minimize the too large tmpfs (to avoid OOM), this is better handled by ad= ding swap
# might interfere with sys upgrade procedure (default is ram/2 IIRC
mount tmpfs /tmp -t tmpfs -o remount,size=3D16000k,nosuid,nodev

exit 0


and here is /etc/init.d/vnstatbackup:

#!/bin/sh
## Please visit http://wiki.openwrt.org/doc/howto/vnstat
# 20121008sm: automate parsing of vnstat related information
# place this script in /etc/init.d
# make it executable: chmod 755 /etc/init.d/vnstatbackup
# 4 CRON: echo "*/30 * * * * /etc/init.d/vnstatbackup backup" >= ;> /etc/crontabs/root
# =C2=A0 =C2=A0 =C2=A0 /etc/init.d/cron restart
# 4 INIT add
# /etc/init.d/vnstatbackup restore
# to /etc/rc.local


# edit this to point to where you stored the db
BACKUP_BASE_DIR=3D"/home/persistent" =C2=A0 =C2=A0 =C2=A0# where = to store the database backup

# for vnstat have a look at /etc/vnstat.conf (code taken from /etc/init.d/v= nstat)
vnstat_option() {
=C2=A0 =C2=A0 sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^= '\"]*\)['\"].*/\1/p" \
=C2=A0 =C2=A0 /etc/vnstat.conf
}
VNSTAT_DB_DIR=3D"$(vnstat_option DatabaseDir)"
VNSTAT_BASE_DIR=3D`dirname ${VNSTAT_DB_DIR}` #${VNSTAT_DB_DIR%/*}
VNSTAT_SUB_DIR=3D`basename ${VNSTAT_DB_DIR}` #"vnstat"

#echo "VNSTAT_DB_DIR ${VNSTAT_DB_DIR}"
#echo "VNSTAT_BASE_DIR ${VNSTAT_BASE_DIR}"
#echo "VNSTAT_SUB_DIR ${VNSTAT_SUB_DIR}"
#echo "BACKUP_BASE_DIR ${BACKUP_BASE_DIR}"

case $1 in
=C2=A0 =C2=A0 backup)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 cp -r ${VNSTAT_BASE_DIR}/${VNSTAT_SUB_DIR} ${BA= CKUP_BASE_DIR}
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
=C2=A0 =C2=A0 restore)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 cp -r ${BACKUP_BASE_DIR}/${VNSTAT_SUB_DIR} ${VN= STAT_BASE_DIR}
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
=C2=A0 =C2=A0 *)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 echo "Please use 'backup' or '= restore' parameter for run";
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
esac

exit 0

Note that I mounted a USB stick under /home to have plenty of space, but yo= u should also be able to use the persistent overlay part of cerowrt (where = config files and such are stored) but that will stress the precious flash (= a usb stick, by virtue of being replaceable and cheap is just less precious= ).

Finally System->Scheduled Tasks:

*/5 * * * * vnstat -u
*/30 * * * * /etc/init.d/vnstatbackup backup

The first polls the Rx/Tx counters every 5 minutes (if your link is very fa= st you might want to reduce that)
The second pushes the vnstat database every 30 minutes to the stick. The id= ea here is that you might want vnstat to do something even when the stick i= s (accidentally) removed.

best
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Sebastian


>
> --Justin Madru
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@l= ists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel


--e89a8fb2066ef445cf04d4159394--