[Bloat] I am unable to pinpoint the source of bufferbloat

Sat Feb 9 11:10:23 PST 2013

Hi Jonathan,

On Feb 9, 2013, at 10:15 , Jonathan Morton wrote:

> Latency caused by bufferbloat always appears at the bottleneck device. Usually that is the modem, and you've given no alternative that it could plausibly be. The modems you mention are slightly different model numbers, but that can hide substantial differences in internal configuration.
> 
> For a typical drop-tail queue, the induced latency under load is the size of the buffer divided by the speed of the link draining it. Assuming both modems have a 4Mbit uplink, 550ms is consistent with a 256KB buffer, and 220ms is consistent with a 48KB buffer - neither of which would seem excessively large to a modem builder who hasn't heard of bufferbloat. However with a shared cable infrastructure, it is possible that the uplink is constrained by other users on the same segment, which will skew this calculation.
> 
> To cure it without modifying the modem, you need to move the bottleneck to a point where you can control the buffer. You do this by introducing traffic shaping at slightly below the advertised modem uplink speed on one of your own machines and directing all upstream traffic through it.

	This is great advise, I just want to mention that both openWRT's QOS settings as well as cerowrt's simple_qos.sh still show some buffering in netalyzr (in my case accidentally 550ms on the uplink with a 4000Kbit/s cable connection shaped down to 3880Kbit/s using cerowrt's (3.7.4-3) simple_qos script). But real measurements with real TCP traffic (opening 60 webpages simultaneously while saturating the uplink with a big upload) showed that ping times are only marginally affected by the cable connection running constantly close to full saturation. My point netalyzr's worst case scenario is quite a bit detached from real world performance, and good queue management (using one of the codel derivates) will give excellent real world performance while still showing excessive buffering in netalyzr. Make what you will out of that… (my not-even-started-yet pet project is to separate tcp from udp traffic (or even better responsive/elastic from unresponsive/inesastic traffic)) and treat both to an according dropping strategy, i.e. drop UDP much harder, but given my time constraints that will happen in the next decade….

best
	Sebastian

> 
> - Jonathan Morton
> On Feb 9, 2013 7:27 PM, "Forums1000" <forums1000 at gmail.com> wrote:
> Hi Jonathan and Dave
> 
> My entire LAN-network is gigabit. My cable subscription is 60 megabit down and 4 megabit up. 
> Now, both my routers' WAN-port and the cable modems' LAN port are also gigabit. The router can route LAN to WAN and the other way around (with NAT and connection tracking enabled) in excess of 100 megabit. 
> 
> Now my cable modem is a Motorola Surfboard SV6120E and hers is a Motorola Surfboard CV6181E. My upload lag is 550ms and hers is only 220ms. Moreover, at her place there are Powerplugs in the path limiting her download to 30 megabit instead of 60 megabit. Yet, the upload lag is much lower than mine. There, it also did not matter where I ran Natalyzr, the result was always 220ms of bufferbload.
> 
> Could this still be only the modem?
> 
> 
> On Sat, Feb 9, 2013 at 10:52 AM, Forums1000 <forums1000 at gmail.com> wrote:
> Hi everyone,
> 
> Can anyone give some tips on how to diagnose the sources of bufferbloat? According to the Netalyzr test at http://netalyzr.icsi.berkeley.edu/, I have 550ms of upload bufferbloat. I tried all kinds of stuff on my Windows 7 laptop:
> 
> - For the Intel(R) 82567LF Gigabit Network Connection, I put receive and transmit buffers to the lowest value of 80 (80 bytes? 80 packets? I don't know). I also disabled interrupt moderation. 
> Result? Still 550ms.
> - Then I connected my laptop directly to my cable modem, bypassing my Mikrotik 450G router. Result? Still 550ms of bufferbloat. 
> - Then I put a 100 megabit switch between the cable modem an the laptop (as both cable modem and Intel NIC are gigabit). Result? Still 550ms of upload bufferbloat.
> 
> I'm out of ideas now. It seems I can't do anything at all to lower bufferbloat. Or the Netalyzr test is broken?:-)
> 
> many thanks for your advice,
> Jeroen
> 
> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> 
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat