* [Cerowrt-devel] Network device transfer counter loops at 4GB @ 2013-01-24 8:55 Justin Madru 2013-01-24 17:40 ` Sebastian Moeller 0 siblings, 1 reply; 4+ messages in thread From: Justin Madru @ 2013-01-24 8:55 UTC (permalink / raw) To: cerowrt-devel [-- Attachment #1: Type: text/plain, Size: 317 bytes --] Hi, I noticed that the TX/RX counter as reported by ifconfig and in the web ui loop over at 4GB (2^32). Is this something that could be fixed? Interestingly, the ifb0 device (what ever that is) does not have this issue. It would be nice to know the amount of data I've transferred at least for GE00. --Justin Madru [-- Attachment #2: Type: text/html, Size: 383 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Cerowrt-devel] Network device transfer counter loops at 4GB 2013-01-24 8:55 [Cerowrt-devel] Network device transfer counter loops at 4GB Justin Madru @ 2013-01-24 17:40 ` Sebastian Moeller 2013-01-25 4:44 ` Justin Madru 0 siblings, 1 reply; 4+ messages in thread From: Sebastian Moeller @ 2013-01-24 17:40 UTC (permalink / raw) To: Justin Madru; +Cc: cerowrt-devel Hi Justin, have a look at vnstat and vnstati if you want to keep a better overview of Rx/Tx volumes over time… (Then again, it might not be what you are looking for) best Sebastian On Jan 24, 2013, at 00:55 , Justin Madru wrote: > Hi, > > I noticed that the TX/RX counter as reported by ifconfig and in the web ui loop over at 4GB (2^32). Is this something that could be fixed? Just guessing here, but that smells like a 32bit counter which might be involved to change to 64bit. So you could just periodically read out the value and compare to your last value to detect overruns (assuming you sample at a period smaller than the shortest possible roll-over time). Then again this seems what vnstat is doing. > Interestingly, the ifb0 device (what ever that is) does not have this issue. http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb > It would be nice to know the amount of data I've transferred at least for GE00. Me = broken record: have a look at the vnstat and vnstati packed that are part of the cerowrt repository that Dave maintains (THANKS A LOT) If you should go the vnstat route here is what I ended up with (nothing fancy, but it seems to work okay): Here is my System->Startup-> local startup (can be pasted into the lucy text box) # Put your custom commands here that should be executed once # the system init finished. By default this file does nothing. #manual vnstat updates... since vnstat logs to tmpfs re-init the database mkdir /var/lib/vnstat vnstat -u -i ge00 # start the traffic shaping script on startup... #/usr/sbin/simple_qos.sh # be verbose in dmesg and log echo 1 1 1 1 > /proc/sys/kernel/printk # helps unaligned access errors show up in log #echo 2 > /sys/kernel/debug/mips/unaligned_action # to make vnstat robust against reboots (to monitor volume caps) # this will copy the saved vnstat database over the just initialized (more or less empty db) /etc/init.d/vnstatbackup restore # minimize the too large tmpfs (to avoid OOM), this is better handled by adding swap # might interfere with sys upgrade procedure (default is ram/2 IIRC mount tmpfs /tmp -t tmpfs -o remount,size=16000k,nosuid,nodev exit 0 and here is /etc/init.d/vnstatbackup: #!/bin/sh ## Please visit http://wiki.openwrt.org/doc/howto/vnstat # 20121008sm: automate parsing of vnstat related information # place this script in /etc/init.d # make it executable: chmod 755 /etc/init.d/vnstatbackup # 4 CRON: echo "*/30 * * * * /etc/init.d/vnstatbackup backup" >> /etc/crontabs/root # /etc/init.d/cron restart # 4 INIT add # /etc/init.d/vnstatbackup restore # to /etc/rc.local # edit this to point to where you stored the db BACKUP_BASE_DIR="/home/persistent" # where to store the database backup # for vnstat have a look at /etc/vnstat.conf (code taken from /etc/init.d/vnstat) vnstat_option() { sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^'\"]*\)['\"].*/\1/p" \ /etc/vnstat.conf } VNSTAT_DB_DIR="$(vnstat_option DatabaseDir)" VNSTAT_BASE_DIR=`dirname ${VNSTAT_DB_DIR}` #${VNSTAT_DB_DIR%/*} VNSTAT_SUB_DIR=`basename ${VNSTAT_DB_DIR}` #"vnstat" #echo "VNSTAT_DB_DIR ${VNSTAT_DB_DIR}" #echo "VNSTAT_BASE_DIR ${VNSTAT_BASE_DIR}" #echo "VNSTAT_SUB_DIR ${VNSTAT_SUB_DIR}" #echo "BACKUP_BASE_DIR ${BACKUP_BASE_DIR}" case $1 in backup) cp -r ${VNSTAT_BASE_DIR}/${VNSTAT_SUB_DIR} ${BACKUP_BASE_DIR} ;; restore) cp -r ${BACKUP_BASE_DIR}/${VNSTAT_SUB_DIR} ${VNSTAT_BASE_DIR} ;; *) echo "Please use 'backup' or 'restore' parameter for run"; ;; esac exit 0 Note that I mounted a USB stick under /home to have plenty of space, but you should also be able to use the persistent overlay part of cerowrt (where config files and such are stored) but that will stress the precious flash (a usb stick, by virtue of being replaceable and cheap is just less precious). Finally System->Scheduled Tasks: */5 * * * * vnstat -u */30 * * * * /etc/init.d/vnstatbackup backup The first polls the Rx/Tx counters every 5 minutes (if your link is very fast you might want to reduce that) The second pushes the vnstat database every 30 minutes to the stick. The idea here is that you might want vnstat to do something even when the stick is (accidentally) removed. best Sebastian > > --Justin Madru > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Cerowrt-devel] Network device transfer counter loops at 4GB 2013-01-24 17:40 ` Sebastian Moeller @ 2013-01-25 4:44 ` Justin Madru 2013-01-25 5:01 ` Dave Taht 0 siblings, 1 reply; 4+ messages in thread From: Justin Madru @ 2013-01-25 4:44 UTC (permalink / raw) To: Sebastian Moeller; +Cc: cerowrt-devel [-- Attachment #1: Type: text/plain, Size: 5233 bytes --] Thanks for the reply. I'll take a look at vnstat. It still seems like making the counters 64bit would be a better out-of-the-box solution, especially since the web-ui is displaying these values which are nearly useless. --Justin On Thu, Jan 24, 2013 at 9:40 AM, Sebastian Moeller <moeller0@gmx.de> wrote: > Hi Justin, > > have a look at vnstat and vnstati if you want to keep a better overview of > Rx/Tx volumes over time… (Then again, it might not be what you are looking > for) > > best > Sebastian > > > On Jan 24, 2013, at 00:55 , Justin Madru wrote: > > > Hi, > > > > I noticed that the TX/RX counter as reported by ifconfig and in the web > ui loop over at 4GB (2^32). Is this something that could be fixed? > > Just guessing here, but that smells like a 32bit counter which > might be involved to change to 64bit. So you could just periodically read > out the value and compare to your last value to detect overruns (assuming > you sample at a period smaller than the shortest possible roll-over time). > Then again this seems what vnstat is doing. > > > Interestingly, the ifb0 device (what ever that is) does not have this > issue. > > > http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb > > > It would be nice to know the amount of data I've transferred at least > for GE00. > > Me = broken record: have a look at the vnstat and vnstati packed > that are part of the cerowrt repository that Dave maintains (THANKS A LOT) > > If you should go the vnstat route here is what I ended up with (nothing > fancy, but it seems to work okay): > > Here is my System->Startup-> local startup (can be pasted into the lucy > text box) > > # Put your custom commands here that should be executed once > # the system init finished. By default this file does nothing. > > #manual vnstat updates... since vnstat logs to tmpfs re-init the database > mkdir /var/lib/vnstat > vnstat -u -i ge00 > > # start the traffic shaping script on startup... > #/usr/sbin/simple_qos.sh > > > # be verbose in dmesg and log > echo 1 1 1 1 > /proc/sys/kernel/printk > # helps unaligned access errors show up in log > #echo 2 > /sys/kernel/debug/mips/unaligned_action > > # to make vnstat robust against reboots (to monitor volume caps) > # this will copy the saved vnstat database over the just initialized (more > or less empty db) > /etc/init.d/vnstatbackup restore > > # minimize the too large tmpfs (to avoid OOM), this is better handled by > adding swap > # might interfere with sys upgrade procedure (default is ram/2 IIRC > mount tmpfs /tmp -t tmpfs -o remount,size=16000k,nosuid,nodev > > exit 0 > > > and here is /etc/init.d/vnstatbackup: > > #!/bin/sh > ## Please visit http://wiki.openwrt.org/doc/howto/vnstat > # 20121008sm: automate parsing of vnstat related information > # place this script in /etc/init.d > # make it executable: chmod 755 /etc/init.d/vnstatbackup > # 4 CRON: echo "*/30 * * * * /etc/init.d/vnstatbackup backup" >> > /etc/crontabs/root > # /etc/init.d/cron restart > # 4 INIT add > # /etc/init.d/vnstatbackup restore > # to /etc/rc.local > > > # edit this to point to where you stored the db > BACKUP_BASE_DIR="/home/persistent" # where to store the database > backup > > # for vnstat have a look at /etc/vnstat.conf (code taken from > /etc/init.d/vnstat) > vnstat_option() { > sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^'\"]*\)['\"].*/\1/p" \ > /etc/vnstat.conf > } > VNSTAT_DB_DIR="$(vnstat_option DatabaseDir)" > VNSTAT_BASE_DIR=`dirname ${VNSTAT_DB_DIR}` #${VNSTAT_DB_DIR%/*} > VNSTAT_SUB_DIR=`basename ${VNSTAT_DB_DIR}` #"vnstat" > > #echo "VNSTAT_DB_DIR ${VNSTAT_DB_DIR}" > #echo "VNSTAT_BASE_DIR ${VNSTAT_BASE_DIR}" > #echo "VNSTAT_SUB_DIR ${VNSTAT_SUB_DIR}" > #echo "BACKUP_BASE_DIR ${BACKUP_BASE_DIR}" > > case $1 in > backup) > cp -r ${VNSTAT_BASE_DIR}/${VNSTAT_SUB_DIR} ${BACKUP_BASE_DIR} > ;; > restore) > cp -r ${BACKUP_BASE_DIR}/${VNSTAT_SUB_DIR} ${VNSTAT_BASE_DIR} > ;; > *) > echo "Please use 'backup' or 'restore' parameter for run"; > ;; > esac > > exit 0 > > Note that I mounted a USB stick under /home to have plenty of space, but > you should also be able to use the persistent overlay part of cerowrt > (where config files and such are stored) but that will stress the precious > flash (a usb stick, by virtue of being replaceable and cheap is just less > precious). > > Finally System->Scheduled Tasks: > > */5 * * * * vnstat -u > */30 * * * * /etc/init.d/vnstatbackup backup > > The first polls the Rx/Tx counters every 5 minutes (if your link is very > fast you might want to reduce that) > The second pushes the vnstat database every 30 minutes to the stick. The > idea here is that you might want vnstat to do something even when the stick > is (accidentally) removed. > > best > Sebastian > > > > > > --Justin Madru > > _______________________________________________ > > Cerowrt-devel mailing list > > Cerowrt-devel@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cerowrt-devel > > [-- Attachment #2: Type: text/html, Size: 6407 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Cerowrt-devel] Network device transfer counter loops at 4GB 2013-01-25 4:44 ` Justin Madru @ 2013-01-25 5:01 ` Dave Taht 0 siblings, 0 replies; 4+ messages in thread From: Dave Taht @ 2013-01-25 5:01 UTC (permalink / raw) To: Justin Madru; +Cc: cerowrt-devel [-- Attachment #1: Type: text/plain, Size: 6044 bytes --] The 32 bit counter problem is kind of legacy. I'm not even sure what layer of the stack it's at. Perhaps better counters are in /sys/class/net or elsewhere. On Thu, Jan 24, 2013 at 11:44 PM, Justin Madru <justin.jdm64@gmail.com>wrote: > Thanks for the reply. I'll take a look at vnstat. It still seems like > making the counters 64bit would be a better out-of-the-box solution, > especially since the web-ui is displaying these values which are nearly > useless. > To know someone other than me has pushed more than 4GB through cerowrt is quite comforting, actually.... > > > --Justin > > > On Thu, Jan 24, 2013 at 9:40 AM, Sebastian Moeller <moeller0@gmx.de>wrote: > >> Hi Justin, >> >> have a look at vnstat and vnstati if you want to keep a better overview >> of Rx/Tx volumes over time… (Then again, it might not be what you are >> looking for) >> >> best >> Sebastian >> >> >> On Jan 24, 2013, at 00:55 , Justin Madru wrote: >> >> > Hi, >> > >> > I noticed that the TX/RX counter as reported by ifconfig and in the web >> ui loop over at 4GB (2^32). Is this something that could be fixed? >> >> Just guessing here, but that smells like a 32bit counter which >> might be involved to change to 64bit. So you could just periodically read >> out the value and compare to your last value to detect overruns (assuming >> you sample at a period smaller than the shortest possible roll-over time). >> Then again this seems what vnstat is doing. >> >> > Interestingly, the ifb0 device (what ever that is) does not have this >> issue. >> >> >> http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb >> >> > It would be nice to know the amount of data I've transferred at least >> for GE00. >> >> Me = broken record: have a look at the vnstat and vnstati packed >> that are part of the cerowrt repository that Dave maintains (THANKS A LOT) >> >> If you should go the vnstat route here is what I ended up with (nothing >> fancy, but it seems to work okay): >> >> Here is my System->Startup-> local startup (can be pasted into the lucy >> text box) >> >> # Put your custom commands here that should be executed once >> # the system init finished. By default this file does nothing. >> >> #manual vnstat updates... since vnstat logs to tmpfs re-init the database >> mkdir /var/lib/vnstat >> vnstat -u -i ge00 >> >> # start the traffic shaping script on startup... >> #/usr/sbin/simple_qos.sh >> >> >> # be verbose in dmesg and log >> echo 1 1 1 1 > /proc/sys/kernel/printk >> # helps unaligned access errors show up in log >> #echo 2 > /sys/kernel/debug/mips/unaligned_action >> >> # to make vnstat robust against reboots (to monitor volume caps) >> # this will copy the saved vnstat database over the just initialized >> (more or less empty db) >> /etc/init.d/vnstatbackup restore >> >> # minimize the too large tmpfs (to avoid OOM), this is better handled by >> adding swap >> # might interfere with sys upgrade procedure (default is ram/2 IIRC >> mount tmpfs /tmp -t tmpfs -o remount,size=16000k,nosuid,nodev >> >> exit 0 >> >> >> and here is /etc/init.d/vnstatbackup: >> >> #!/bin/sh >> ## Please visit http://wiki.openwrt.org/doc/howto/vnstat >> # 20121008sm: automate parsing of vnstat related information >> # place this script in /etc/init.d >> # make it executable: chmod 755 /etc/init.d/vnstatbackup >> # 4 CRON: echo "*/30 * * * * /etc/init.d/vnstatbackup backup" >> >> /etc/crontabs/root >> # /etc/init.d/cron restart >> # 4 INIT add >> # /etc/init.d/vnstatbackup restore >> # to /etc/rc.local >> >> >> # edit this to point to where you stored the db >> BACKUP_BASE_DIR="/home/persistent" # where to store the database >> backup >> >> # for vnstat have a look at /etc/vnstat.conf (code taken from >> /etc/init.d/vnstat) >> vnstat_option() { >> sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^'\"]*\)['\"].*/\1/p" \ >> /etc/vnstat.conf >> } >> VNSTAT_DB_DIR="$(vnstat_option DatabaseDir)" >> VNSTAT_BASE_DIR=`dirname ${VNSTAT_DB_DIR}` #${VNSTAT_DB_DIR%/*} >> VNSTAT_SUB_DIR=`basename ${VNSTAT_DB_DIR}` #"vnstat" >> >> #echo "VNSTAT_DB_DIR ${VNSTAT_DB_DIR}" >> #echo "VNSTAT_BASE_DIR ${VNSTAT_BASE_DIR}" >> #echo "VNSTAT_SUB_DIR ${VNSTAT_SUB_DIR}" >> #echo "BACKUP_BASE_DIR ${BACKUP_BASE_DIR}" >> >> case $1 in >> backup) >> cp -r ${VNSTAT_BASE_DIR}/${VNSTAT_SUB_DIR} ${BACKUP_BASE_DIR} >> ;; >> restore) >> cp -r ${BACKUP_BASE_DIR}/${VNSTAT_SUB_DIR} ${VNSTAT_BASE_DIR} >> ;; >> *) >> echo "Please use 'backup' or 'restore' parameter for run"; >> ;; >> esac >> >> exit 0 >> >> Note that I mounted a USB stick under /home to have plenty of space, but >> you should also be able to use the persistent overlay part of cerowrt >> (where config files and such are stored) but that will stress the precious >> flash (a usb stick, by virtue of being replaceable and cheap is just less >> precious). >> >> Finally System->Scheduled Tasks: >> >> */5 * * * * vnstat -u >> */30 * * * * /etc/init.d/vnstatbackup backup >> >> The first polls the Rx/Tx counters every 5 minutes (if your link is very >> fast you might want to reduce that) >> The second pushes the vnstat database every 30 minutes to the stick. The >> idea here is that you might want vnstat to do something even when the stick >> is (accidentally) removed. >> >> best >> Sebastian >> >> >> > >> > --Justin Madru >> > _______________________________________________ >> > Cerowrt-devel mailing list >> > Cerowrt-devel@lists.bufferbloat.net >> > https://lists.bufferbloat.net/listinfo/cerowrt-devel >> >> > > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel > > -- Dave Täht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html [-- Attachment #2: Type: text/html, Size: 7849 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-01-25 5:01 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-01-24 8:55 [Cerowrt-devel] Network device transfer counter loops at 4GB Justin Madru 2013-01-24 17:40 ` Sebastian Moeller 2013-01-25 4:44 ` Justin Madru 2013-01-25 5:01 ` Dave Taht
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox