[Cerowrt-devel] routers you can throw off the back of a truck

Sun Jan 17 19:45:11 EST 2016

Hi everybody,

our mission is to provide internet in humanitarian and crisis
situations to as many people as possible, and to make bandwidth fairly
distributed between users and to have low bufferbloat.

Issue with working out in crisis stations is that you don't know what
kind of uplink you will get, and bandwidth you get is much often
variable than constant. I have tested 3G/4G uplinks and download can
be in and range from 100Mbps to 5 Mbps, especially with mobile nodes
(we sometimes put router, battery and 3G/4G modem in backpack and go
where it is needed).

Is there anyway to configure any queuing technique in Linux kernel
that so it distributes bandwidth equally between users and to keep lag
(bufferbloat) as low as possible, but without needing to define
absolute values for download and upload. For bandwidth distribution I
know that Mikrotik has really awesome technique called PCQ [1] and
AFAIK there is no technique as PCQ in Linux kernel, the closes I have
found is SFQ, if you know any other similar queueing techniques in
Linux kernel please share.

I have been testing different queue setup scripts for fq_codel in
OpenWrt, to see how they work with properly setup bandwidth limits and
also with completely wrong bandwidth limits. Test have been done with
3G/4G USB modems and on 100/10 Mbps fibre optics connection with
TP-LINK WDR3600 router running latest Chaos Calmer OpenWrt.

Also I have also seen that pfSense mentions that HFSC [2] that is
"very effective for VoIP on links that degrade quickly, such as 3G/4G,
but it can be complex to configure and tweak for proper operation." so
I tested HFSC queuing technique with fq_codel with  and results show
that it also doesn't work if bandwith is not configured correctly

So far all fq_codel + queue setup scripts on OpenWrt I have tested
give low bufferbloat results only when banddwidth is configured to
around 90% of available bandwidth, once I setup it to 200% of
available bandwidth I get really bad bufferbloat results, so any
current options just don't work if bandwidth changes.

Please don't assume I know anything, if you have any idea what I could
test or how to get better results in real world conditions please
share...

Here are my current results it they are interesting to anybody.

Fiber connection 100/10 Mbps
no qos: 150/110 ms - 94/10.4 Mbps - http://www.dslreports.com/speedtest/2635531
no qos: 235/132 ms - 94.2/10.4 Mbps -
http://www.dslreports.com/speedtest/2635586
no qos: 160/130 ms - 96.8/10.5 Mbps -
http://www.dslreports.com/speedtest/2635646

Fiber connection: SQM 95000/9500 on eth0.2
fq_codel + nxt_routed_hfsc.qos: 203/42 ms - 95.2/7.2 Mbps -
http://www.dslreports.com/speedtest/2629376
fq_codel + layer_cake.qos: 193/135 ms - 93.1/7.7 Mbps -
http://www.dslreports.com/speedtest/2629461
fq_codel + layer_cake.qos: 201/116 ms - 93.6/8.2 Mbps -
http://www.dslreports.com/speedtest/2629502

Fiber connection: 100/10 Mbps - SQM 90/9 Mbps (90000/9000) on eth0.2
fq_codel + simple: 63/58 ms - 83/4 Mbps -
http://www.dslreports.com/speedtest/2629696
fq_codel + simple: 87/55 ms - 84.2/4.6 Mbps -
http://www.dslreports.com/speedtest/2629939
fq_codel + nxt_routed_hfsc: 130/46 ms - 94.6/6.9 Mbps -
http://www.dslreports.com/speedtest/2630019
fq_codel + nxt_routed_hfsc: 160/38 ms - 93/9.1 Mbps -
http://www.dslreports.com/speedtest/2630082
fq_codel + layer_cake: 140/120 ms - 91.3/6.6 Mbps -
http://www.dslreports.com/speedtest/2629597
fq_codel + layer_cake: 140/190 ms - 92.5/10.2 Mbps -
http://www.dslreports.com/speedtest/2630122
fq_codel + simplest: 41/35 ms - 81.4/3.7 Mbps -
http://www.dslreports.com/speedtest/2629988
fq_codel + simplest: 57/56 ms - 84.5/8.8 Mbps -
http://www.dslreports.com/speedtest/2630228
fq_codel + piece_of_cake: 175/119 ms - 93/7.5 Mbps -
http://www.dslreports.com/speedtest/2629749
fq_codel + piece_of_cake: 215/146 ms - 89.6/11.7 Mbps -
http://www.dslreports.com/speedtest/2630294
fq_codel + simple_pppoe: 61/48 ms - 84.1/8.5 Mbps -
http://www.dslreports.com/speedtest/2630153

Fiber connection: 100/10 Mbps - SQM 200/20 Mbps (200000/20000) on eth0.2
fq_codel + simple: 102/132 ms - 94.8/10.3 Mbps -
http://www.dslreports.com/speedtest/2630347
fq_codel + nxt_routed_hfsc: 150/133 ms - 94.1/10.5 Mbps -
http://www.dslreports.com/speedtest/2630386
fq_codel + simplest: 178/148 ms - 92.7/10.5 Mbps -
http://www.dslreports.com/speedtest/2630488
fq_codel + piece_of_cake: 192/122 ms - 95/10.7 Mbps -
http://www.dslreports.com/speedtest/2635507

Cheers,
Valent.

[1] http://wiki.mikrotik.com/wiki/Manual:Queues_-_PCQ
[2] https://doc.pfsense.org/index.php/Traffic_Shaping_Guide

On Sun, Jan 17, 2016 at 9:31 PM, Outback Dingo <outbackdingo at gmail.com> wrote:
>
>
> On Sun, Jan 17, 2016 at 7:46 PM, Dave Taht <dave.taht at gmail.com> wrote:
>>
>> http://www.meshpoint.me/ design and purpose seems very cool.
>>
>> Given all the nifty new autoconfig stuff in homenet like hnetd and
>> source specific babel - perhaps we are closer than ever before to
>> actually being able to throw this kind off stuff off the back of a
>> truck and have it "just work".
>>
>> If only we had bufferbloat-free lte uplink hardware (has any
>> materialized, perhaps in a mini-pci-e form factor?)
>>
>> It does now seem semi-possible to hack on enodeBs nowadays:
>>
>> http://bellard.org/lte/
>>
>> Are any LTE providers supporting dhcp-pd or some equivalent for
>> getting routable ipv6 subnets?
>>
>> --
>> Dave Täht
>> Let's go make home routers and wifi faster! With better software!
>
>
> There have been "numerous" autoconfiguration setups deployed on OpenWRT by a
> few different people, even myself
> adding in 4G/LTE would be nice, though this has been possible for a while,
> even ive used the BladeRF in a "BTS" environment
> for testing of WhiteSpace wireless VOIP, and Disaster Scenerios, grab an
> Alix APU load OpenWRT, plug a bladeRF into the USB
> load OpenBTS on it and place it out in the wild.... Instant white space
> telco
>
>
>>
>> https://www.gofundme.com/savewifi
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>