[Bloat] $106 achieved and flent-farm status
Dave Taht
dave.taht at gmail.com
Mon Feb 4 22:37:14 EST 2019
Thank you mikael and jake, matt and matthew and richard! (and jon, and
dev for trying)
the linode bill is now paid. For the record, peak "earnings" for the
patreon contribution page was $212 in may 2016, and fell below $100 in
May, 2018. I forget when I scaled the number of servers back in
between those times.
In cerowrt's glory days we were burning ~2k a month on the openwrt
build farm, which I had to quit doing 6 months after the google grant
did. Prior to that we had donated servers from ISC, which had to get
out of the free ISP business. I kind of miss running on bare metal,
with tons of disk space. Cloud services are pretty cheap, cloud
storage isn't.
Oy! I remember the headaches and hassles when the number of openwrt
buildslaves dropped to 1 for 6 weeks during a critical phase....
Thankfully that farm looks healthy at the moment:
http://phase1.builds.openwrt.org/buildslaves
Costs on the "flent-farm" continue to drop. Our earliest linode
servers cost $20/month and our two latest ones (nanoservers) cost
$5/month. For "science!" I've been generally unwilling to
update/change these much, the most critical keeping the same kernel
versions they had for the last couple years. I note that linode at
least a year+ ago, started defaulting to a kernel with fq_codel
enabled *by default*, bql just works, irtt, flent, netperf all "just
install" from apt, etc, etc.
It's currently a matter of a few minutes to get a basic flent server
running in the cloud, and sometimes I wish we had more worldwide
coverage - australia - aws - france - china - It turns out arm based
servers are quite cheap nowadays... yep, here I am 6 bucks to the good
and trying to figure out another server on which to spend it. :)
I have not been maintaining the flent network all that well, but your
support inspired me to go fix two of them.
flent-singapore.taht.net: the newest of these servers
de.taht.net: the second newest
now support BBR, cubic, and reno, and have irtt running with support
for 1ms intervals. (thx pete for showing me how on
flent-london.bufferbloat.net)
Linux flent-singapore.taht.net 4.18.16-x86_64-linode118 #1 SMP PREEMPT
Mon Oct 29 15:38:25 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Linux de.taht.net 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13
UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
In the teklibre sub-basement are a few dozen tiny openwrt/armbian
boxes, and two honkin, but ancient 12 core xeon servers ("spaceheater"
and "ceres") with 10TB of storage. I keep a variety of vms on those in
complicated network emulation setups, an indexed copy of all the
source code in the world, my openwrt build system, etc.
and here's a puzzler for you! Both boxes are running ntp yet one box
was still *30* seconds off.
irtt 1ms between california and germany has a mindboggling amount of
loss... in america... ahh... tools...
5. be-3651-cr02.sunnyvale.ca.ibone.comcast.net
71.9% 33 11.4 11.6 9.4 16.0 2.3
6. be-11083-pe02.529bryant.ca.ibone.comcast.net
56.2% 33 13.1 11.3 9.5 14.8 1.6
Min Mean Median Max Stddev
--- ---- ------ --- ------
RTT 166.2ms 175.4ms 174.3ms 232.7ms 4.85ms
send delay 27.9s 27.9s 27.9s 28s 4.07ms
receive delay -27.7s -27.7s -27.7s -27.7s 1.98ms
IPDV (jitter) 434ns 1.46ms 979µs 37.64ms 1.78ms
send IPDV 408ns 1.36ms 954µs 34.92ms 1.51ms
receive IPDV 2ns 378µs 72.2µs 32.98ms 1.08ms
send call time 3.55µs 14.9µs 1.61ms 16.7µs
timer error 0s 19.6µs 5.52ms 68.3µs
server proc. time 710ns 7.67µs 10.33ms 91.6µs
duration: 1m1s (wait 698.1ms)
packets sent/received: 58852/44855 (23.78% loss)
server packets received: 45579/58852 (22.55%/1.59% loss up/down)
bytes sent/received: 3531120/2691300
send/receive rate: 470.8 Kbps / 358.8 Kbps
packet length: 60 bytes
timer stats: 1148/60000 (1.91%) missed, 1.96% error
restarted ntp, and:
Min Mean Median Max Stddev
--- ---- ------ --- ------
RTT 166.3ms 175.9ms 175.1ms 233ms 4.78ms
send delay 83.97ms 92.57ms 91.9ms 137.4ms 4.1ms
receive delay 80.86ms 83.32ms 82.86ms 137.4ms 1.89ms
IPDV (jitter) 316ns 1.44ms 977µs 63.68ms 1.75ms
send IPDV 41ns 1.35ms 933µs 35.49ms 1.5ms
receive IPDV 2ns 345µs 83.8µs 55.06ms 971µs
send call time 3.55µs 16.4µs 1.07ms 11.8µs
timer error 1ns 23.4µs 5.02ms 76.8µs
server proc. time 716ns 8.22µs 13.13ms 117µs
duration: 1m1s (wait 699.1ms)
packets sent/received: 58779/45154 (23.18% loss)
server packets received: 45912/58779 (21.89%/1.65% loss up/down)
bytes sent/received: 3526740/2709240
send/receive rate: 470.2 Kbps / 361.2 Kbps
packet length: 60 bytes
timer stats: 1221/60000 (2.04%) missed, 2.34% error
Verses singapore.
Min Mean Median Max Stddev
--- ---- ------ --- ------
RTT 173.6ms 179.2ms 178.7ms 194.7ms 2.5ms
send delay 113ms 118.5ms 118ms 131.9ms 2.32ms
receive delay 59.7ms 60.68ms 60.45ms 74.75ms 980µs
IPDV (jitter) 203ns 1.24ms 978µs 18.8ms 1.01ms
send IPDV 20ns 1.19ms 983µs 14ms 881µs
receive IPDV 0s 195µs 20.3µs 14.11ms 539µs
send call time 3.67µs 14.7µs 1.75ms 13.2µs
timer error 1ns 21.4µs 7.82ms 93.9µs
server proc. time 680ns 3.27µs 435µs 9.35µs
duration: 1m1s (wait 584ms)
packets sent/received: 58709/55848 (4.87% loss)
server packets received: 55848/58709 (4.87%/0.00% loss up/down)
bytes sent/received: 3522540/3350880
send/receive rate: 469.7 Kbps / 446.8 Kbps
packet length: 60 bytes
timer stats: 1290/59999 (2.15%) missed, 2.14% error
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
More information about the Bloat
mailing list