[Bloat] the side effects of 330ms lag in the real world

[Bloat] the side effects of 330ms lag in the real world

[Dave Taht]

pretty wonderful experiment and video http://livingwithlag.com/

-- 
Dave Täht

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Mikael Abrahamsson]

On Mon, 28 Apr 2014, Dave Taht wrote:

> pretty wonderful experiment and video http://livingwithlag.com/

Just so that everybody realises that this is an advertisement.

Also, what access method has 300 ms access latency, let alone 3 seconds? 
None that I know of, the meaningful comparison would be ADSL2+ at around 
25ms and 3G at around 50-100ms.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Fred Baker (fred)]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]


On Apr 29, 2014, at 3:08 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:

> On Mon, 28 Apr 2014, Dave Taht wrote:
> 
>> pretty wonderful experiment and video http://livingwithlag.com/
> 
> Just so that everybody realises that this is an advertisement.
> 
> Also, what access method has 300 ms access latency, let alone 3 seconds? None that I know of, the meaningful comparison would be ADSL2+ at around 25ms and 3G at around 50-100ms.

Well, we could discuss international communications. I happen to be at Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you:

ping -c 10 swm.pp.se
PING swm.pp.se (212.247.200.143): 56 data bytes
   ...
--- swm.pp.se ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 249.368/342.038/456.376/71.902 ms

3 seconds is unusually high, although naval satcom is frequently in the 1.5-2.5 ms range. But I have a sample that I measured in a Dublin hotel that observed standing latencies of around 280 ms to San Jose, frequent latencies of seven seconds, and a peak of 9286 ms. Yes, the hotel DSL was horrendously misconfigured. It makes a great graphic for presentations.

> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> 
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

----------------------------------------------------
The ignorance of how to use new knowledge stockpiles exponentially. 
   - Marshall McLuhan


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Mikael Abrahamsson]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 571 bytes --]

On Tue, 29 Apr 2014, Fred Baker (fred) wrote:

> Well, we could discuss international communications. I happen to be at 
> Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you:

Yes, but as soon as you hit the long distance network the latency is the 
same regardless of access method. So while I agree that understanding the 
effect of latency is important, it's no longer a meaningful way of selling 
fiber access. If your last-mile is fiber instead of ADSL2+ won't improve 
your long distance latency.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Dave Taht]

On Tue, Apr 29, 2014 at 12:56 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> On Tue, 29 Apr 2014, Fred Baker (fred) wrote:

A couple points here.

1) The video went viral, and garnered over 600,000 new hits in the 12
hours since I posted
 it here.

there is pent up demand for less latency. While the ad conflates
bandwidth with latency,
they could have published their RTTs on their local fiber network,
which is probably a
great deal less than dsl or cable. That counts for a lot when
accessing local services.

2) There is a lot of things an ISP can do to improve apparent latency
on the long haul

A)  co-locating with a major dns server like f-root to reduce dns latency
B)  co-locating with major services like google and netflix

publishing ping times to google for example might be a good tactic.

C) Better peering

>> Well, we could discuss international communications. I happen to be at
>> Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you:
>
>
> Yes, but as soon as you hit the long distance network the latency is the
> same regardless of access method. So while I agree that understanding the
> effect of latency is important, it's no longer a meaningful way of selling
> fiber access. If your last-mile is fiber instead of ADSL2+ won't improve
> your long distance latency.

Well, it chops a great deal from the baseline physical latency, and most
people tend to access resources closer to them than farther away. An
american in paris might want to access the NYT, but Parisians La Monde.

Similarly most major websites are replicated and use CDNs to distribute
their data closer to the user. The physical RTT matters more and more
in the last mile the more resources are co-located in the local data center.

-- 
Dave Täht

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 1388 bytes --]

On Tue, Apr 29, 2014 at 5:46 PM, Dave Taht <dave.taht@gmail.com> wrote:


> > Yes, but as soon as you hit the long distance network the latency is the
> > same regardless of access method. So while I agree that understanding the
> > effect of latency is important, it's no longer a meaningful way of
> selling
> > fiber access. If your last-mile is fiber instead of ADSL2+ won't improve
> > your long distance latency.
>
> Well, it chops a great deal from the baseline physical latency, and most
> people tend to access resources closer to them than farther away. An
> american in paris might want to access the NYT, but Parisians La Monde.
>
> Similarly most major websites are replicated and use CDNs to distribute
> their data closer to the user. The physical RTT matters more and more
> in the last mile the more resources are co-located in the local data
> center.


With my DSL connection, 80% of the latency to "most" things (dns, cdns,
etc) is between the modem and dslam.  That's a place where fiber would fare
far better.  I get 20-25ms to the first router after the dsl modem, and
then akamai and google are within 3-5ms of that.

(and was that american-in-paris comment aimed at me?  ;)

La Monde is, amusingly, about 150ms from me here in Paris.  But
nytimes.comis 270-280...

And the CDN used by lamonde.fr is 60ms away.

And 20-25ms of all of that is DSL overhead.

-Aaron

[-- Attachment #2: Type: text/html, Size: 2017 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]

On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson <swmike@swm.pp.se>wrote:

> On Tue, 29 Apr 2014, Fred Baker (fred) wrote:
>
>  Well, we could discuss international communications. I happen to be at
>> Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you:
>>
>
> Yes, but as soon as you hit the long distance network the latency is the
> same regardless of access method. So while I agree that understanding the
> effect of latency is important, it's no longer a meaningful way of selling
> fiber access. If your last-mile is fiber instead of ADSL2+ won't improve
> your long distance latency.


​FIOS bufferbloat is a problem too.

Measured bufferbloat, symmetric 25/25 service in New Jersey at my inlaw's
house is 200ms (on the ethernet port of the Actiontec router provided by
Verizon).  So latency under load is the usual problem.

Why would you think the GPON guys are any better in principle than cable or
DSL?  Cable and DSL may be somewhat worse, just because it is older and
downward compatibility means that new modems on low bandwidth tiers are
even more grossly over buffered.
You can look at the netalyzr scatter plots in
http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/

Now, if someone gives me real fiber to the home, with a real switch fabric
upstream, rather than gpon life might be somewhat better (if the switches
aren't themselves overbuffered....  But so far, it isn't.
                                      - Jim

                                  - Jim

                                                         - Jim

​

>
>
> --
> Mikael Abrahamsson    email: swmike@swm.pp.se
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
>

[-- Attachment #2: Type: text/html, Size: 3964 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 2360 bytes --]

On Tue, Apr 29, 2014 at 12:44 PM, Jim Gettys <jg@freedesktop.org> wrote:

>
>
>
> On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson <swmike@swm.pp.se>wrote:
>
>> On Tue, 29 Apr 2014, Fred Baker (fred) wrote:
>>
>>  Well, we could discuss international communications. I happen to be at
>>> Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you:
>>>
>>
>> Yes, but as soon as you hit the long distance network the latency is the
>> same regardless of access method. So while I agree that understanding the
>> effect of latency is important, it's no longer a meaningful way of selling
>> fiber access. If your last-mile is fiber instead of ADSL2+ won't improve
>> your long distance latency.
>
>
> ​FIOS bufferbloat is a problem too.
>
> Measured bufferbloat, symmetric 25/25 service in New Jersey at my inlaw's
> house is 200ms (on the ethernet port of the Actiontec router provided by
> Verizon).  So latency under load is the usual problem.
>
> Why would you think the GPON guys are any better in principle than cable
> or DSL?  Cable and DSL may be somewhat worse, just because it is older and
> downward compatibility means that new modems on low bandwidth tiers are
> even more grossly over buffered.
> You can look at the netalyzr scatter plots in
> http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/
>
>
​Oh, I forgot to note: Netalyzr topped out at about 20M​bps: most of why
GPON looks good on those plots is just that the common lowest tier of
service (at the time that data was taken) it can't easily detect the bloat.
But if you perform other tests that don't have a bandwidth limit, the bloat
is there.

This reminds me I should ask Nick Weaver if he has more recent data...
                                             - Jim

Now, if someone gives me real fiber to the home, with a real switch fabric
> upstream, rather than gpon life might be somewhat better (if the switches
> aren't themselves overbuffered....  But so far, it isn't.
>                                       - Jim
>
>
>
>>
>>
>> --
>> Mikael Abrahamsson    email: swmike@swm.pp.se
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 4896 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

Jim Gettys <jg@freedesktop.org> writes:

> Now, if someone gives me real fiber to the home, with a real switch fabric
> upstream, rather than gpon life might be somewhat better (if the switches aren't
> themselves overbuffered.... But so far, it isn't.

As a data point for this, I have fibre to my apartment building and
ethernet into the apartment. I get .5 ms to my upstream gateway and
about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while
running at 100 Mbps...

http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png

However, as that graph shows, it is quite possible to completely avoid
bufferbloat by deploying the right shaping. And in that case fibre
*does* have a significant latency advantage. The best latency I've seen
to the upstream gateway on DSL has been ~12 ms.

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 1272 bytes --]

On Tue, Apr 29, 2014 at 1:01 PM, Toke Høiland-Jørgensen <toke@toke.dk>wrote:

> Jim Gettys <jg@freedesktop.org> writes:
>
> > Now, if someone gives me real fiber to the home, with a real switch
> fabric
> > upstream, rather than gpon life might be somewhat better (if the
> switches aren't
> > themselves overbuffered.... But so far, it isn't.
>
> As a data point for this, I have fibre to my apartment building and
> ethernet into the apartment. I get .5 ms to my upstream gateway and
> about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while
> running at 100 Mbps...
>
> http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png
>
> However, as that graph shows, it is quite possible to completely avoid
> bufferbloat by deploying the right shaping​

And in that case fibre
> *does* have a significant latency advantage. The best latency I've seen
> to the upstream gateway on DSL has been ~12 ms.
>

​Media access is a killer on Cable too, putting the latency floor at around
8ms on my Docsis 3.0 Comcast service, though you can sometimes get lucky
and piggyback. to somewhat lower latency, IIRC conversations with Greg
White about how cable works.
                                       - Jim


> -Toke
>

[-- Attachment #2: Type: text/html, Size: 2328 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Greg White]

[-- Attachment #1: Type: text/plain, Size: 2788 bytes --]

FYI piggybacked requests in DOCSIS eliminate the possibility of request collisions (and resulting backoff/retry).  In DOCSIS 3.0 they will result in lower latency only as a result of eliminating these events.  In a lightly loaded DOCSIS 3.0 network (few neighbors contending for bandwidth) the collision probability will be low anyway, so it won't make much difference.  In a highly utilized network (especially in the case where a lot of neighbors are sending at rates below the piggybacking threshold) it can make a bigger difference.

Packet rates above ~200pps will be sufficient to ensure piggybacking.

I usually consider ~6ms to be the nominal upstream MAC latency, and ~0.7ms to be the equivalent on downstream, making the RTT on the DOCSIS 3.0 link close to 7ms (depending on configuration).

-Greg

From: Jim Gettys <jg@freedesktop.org<mailto:jg@freedesktop.org>>
Date: Tuesday, April 29, 2014 at 11:07 AM
To: Toke Høiland-Jørgensen <toke@toke.dk<mailto:toke@toke.dk>>
Cc: bloat <bloat@lists.bufferbloat.net<mailto:bloat@lists.bufferbloat.net>>, "Fred Baker (fred)" <fred@cisco.com<mailto:fred@cisco.com>>, "aqm@ietf.org<mailto:aqm@ietf.org>" <aqm@ietf.org<mailto:aqm@ietf.org>>, "cerowrt-devel@lists.bufferbloat.net<mailto:cerowrt-devel@lists.bufferbloat.net>" <cerowrt-devel@lists.bufferbloat.net<mailto:cerowrt-devel@lists.bufferbloat.net>>, Mikael Abrahamsson <swmike@swm.pp.se<mailto:swmike@swm.pp.se>>
Subject: Re: [aqm] [Bloat] the side effects of 330ms lag in the real world




On Tue, Apr 29, 2014 at 1:01 PM, Toke Høiland-Jørgensen <toke@toke.dk<mailto:toke@toke.dk>> wrote:
Jim Gettys <jg@freedesktop.org<mailto:jg@freedesktop.org>> writes:

> Now, if someone gives me real fiber to the home, with a real switch fabric
> upstream, rather than gpon life might be somewhat better (if the switches aren't
> themselves overbuffered.... But so far, it isn't.

As a data point for this, I have fibre to my apartment building and
ethernet into the apartment. I get .5 ms to my upstream gateway and
about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while
running at 100 Mbps...

http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png

However, as that graph shows, it is quite possible to completely avoid
bufferbloat by deploying the right shaping​
And in that case fibre
*does* have a significant latency advantage. The best latency I've seen
to the upstream gateway on DSL has been ~12 ms.

​Media access is a killer on Cable too, putting the latency floor at around 8ms on my Docsis 3.0 Comcast service, though you can sometimes get lucky and piggyback. to somewhat lower latency, IIRC conversations with Greg White about how cable works.
                                       - Jim


-Toke


[-- Attachment #2: Type: text/html, Size: 5143 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Dave Taht]

On Tue, Apr 29, 2014 at 10:01 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Jim Gettys <jg@freedesktop.org> writes:
>
>> Now, if someone gives me real fiber to the home, with a real switch fabric
>> upstream, rather than gpon life might be somewhat better (if the switches aren't
>> themselves overbuffered.... But so far, it isn't.
>
> As a data point for this, I have fibre to my apartment building and
> ethernet into the apartment. I get .5 ms to my upstream gateway and
> about 6 ms to Google. Still measured up to ~20 ms of bufferbloat while
> running at 100 Mbps...
>
> http://files.toke.dk/bufferbloat/data/karlstad/cdf_comparison.png

I need to note that what this wonderfully flat CDF for the measurement
stream shows is that short flows under fq_codel leap to the head of
the queue ever better as you get more and more bandwidth available.

The background load flows not shown on this graph are experiencing
5-20ms worth of latency in each direction as per codel's algorithm.

A better test (in progress) would measure typical voip behaviors....

> However, as that graph shows, it is quite possible to completely avoid
> bufferbloat by deploying the right shaping.

It does not "completely avoid bufferbloat", the fq_codel "fast queue"
merely eliminates queuing delay for sparse flows, things like arp, syn,
syn/ack, dns, ntp, etc, as  well as the first packet of any flow that
has not built up a queue yet.

(which is, admittedly, quite a lot of bufferbloat reduction)

The rest of the magic comes from codel.

> And in that case fibre
> *does* have a significant latency advantage. The best latency I've seen
> to the upstream gateway on DSL has been ~12 ms.

And reduced RTT = money.

this piece states observed average RTTs at peak times were 17ms for fiber,
28ms for cable, and 44ms for DSL.

http://www.igvita.com/2012/07/19/latency-the-new-web-performance-bottleneck/

I don't know if the underlying report measures baseline unloaded last mile RTT.

>
> -Toke
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Jan Ceuleers]

On 04/29/2014 07:01 PM, Toke Høiland-Jørgensen wrote:
> However, as that graph shows, it is quite possible to completely avoid
> bufferbloat by deploying the right shaping. And in that case fibre
> *does* have a significant latency advantage. The best latency I've seen
> to the upstream gateway on DSL has been ~12 ms.

I am not an expert, but I believe that this is due to the use of
interleaving. This is a method to improve the strength of forward error
correction by spreading out the effects of impulse noise on DSL lines
across multiple reed-solomon-protected codewords at the expense of latency.

The topic is briefly discussed on the ADSL Wikipedia page.

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Pedro Tumusok]

[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]

On Wed, Apr 30, 2014 at 8:53 AM, Jan Ceuleers <jan.ceuleers@gmail.com>wrote:

> On 04/29/2014 07:01 PM, Toke Høiland-Jørgensen wrote:
> > However, as that graph shows, it is quite possible to completely avoid
> > bufferbloat by deploying the right shaping. And in that case fibre
> > *does* have a significant latency advantage. The best latency I've seen
> > to the upstream gateway on DSL has been ~12 ms.
>
> I am not an expert, but I believe that this is due to the use of
> interleaving. This is a method to improve the strength of forward error
> correction by spreading out the effects of impulse noise on DSL lines
> across multiple reed-solomon-protected codewords at the expense of latency.
>
> The topic is briefly discussed on the ADSL Wikipedia page.
>

Depending on the interleave depth you get extra latency. But there are also
different schemes you can use

Interleave, the one that basically came with ADSL.
Phyr, Broadcom evolution on interleave, that has less latency, I believe,
in most cases.
G.INP, which I have been told is a standardized version of Phyr.

All of these will affect the latency and by how much is also dependant on
the configuration you have done on your DSLAM. To add to the mess, not all
DSLAM's support phyr or g.inp and then up the complexity a bit with CPE
that supports one, two or all three of the technologies.

Ohh some ISPs might even not use interleave and just have the DSLAM setup
to do Fastmode/Fastpath or what the vendor decided to name the feature.

Pedro




> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Best regards / Mvh
Jan Pedro Tumusok

[-- Attachment #2: Type: text/html, Size: 2653 bytes --]

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Mikael Abrahamsson]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1126 bytes --]

On Wed, 30 Apr 2014, Jan Ceuleers wrote:

> On 04/29/2014 07:01 PM, Toke Høiland-Jørgensen wrote:
>> However, as that graph shows, it is quite possible to completely avoid
>> bufferbloat by deploying the right shaping. And in that case fibre
>> *does* have a significant latency advantage. The best latency I've seen
>> to the upstream gateway on DSL has been ~12 ms.
>
> I am not an expert, but I believe that this is due to the use of 
> interleaving. This is a method to improve the strength of forward error 
> correction by spreading out the effects of impulse noise on DSL lines 
> across multiple reed-solomon-protected codewords at the expense of 
> latency.

You're exactly correct. ADSL2+ interleaving can be set to 0 (off), 4, 8 or 
16 milliseconds in the downstream direction and 0(off), 1, 2 or 4 in the 
upstream direction (if memory serves me right, it was 8 years ago I did 
this last).

So to avoid lost packets due to impulse noise, most set this to 16+4, and 
plus the regular encoding delay for ADSL2+, you often end up with around 
25ms RTT to the DSLAM.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Dave Taht]

On Tue, Apr 29, 2014 at 9:44 AM, Jim Gettys <jg@freedesktop.org> wrote:
>
>
>
> On Tue, Apr 29, 2014 at 3:56 AM, Mikael Abrahamsson <swmike@swm.pp.se>
> wrote:
>>
>> On Tue, 29 Apr 2014, Fred Baker (fred) wrote:
>>
>>> Well, we could discuss international communications. I happen to be at
>>> Infocom in Toronto, VPN’d into Cisco San Jose, and did a ping to you:
>>
>>
>> Yes, but as soon as you hit the long distance network the latency is the
>> same regardless of access method. So while I agree that understanding the
>> effect of latency is important, it's no longer a meaningful way of selling
>> fiber access. If your last-mile is fiber instead of ADSL2+ won't improve
>> your long distance latency.
>
>
> FIOS bufferbloat is a problem too.
>
> Measured bufferbloat, symmetric 25/25 service in New Jersey at my inlaw's
> house is 200ms (on the ethernet port of the Actiontec router provided by
> Verizon).  So latency under load is the usual problem.

ESR's link, before and after the cerowrt SQM treatment:

https://www.bufferbloat.net/projects/codel/wiki/RRUL_Rogues_Gallery#Verizon-FIOS-Testing-at-25Mbit-up-and-25Mbit-down

> Why would you think the GPON guys are any better in principle than cable or
> DSL?  Cable and DSL may be somewhat worse, just because it is older and
> downward compatibility means that new modems on low bandwidth tiers are even
> more grossly over buffered.

Well, buffering on the DSLAM or CMTS needs to be more actively
managed. Fixed limits are much like conventional policing, always
either too large or too small to handle sustained or bursty traffic
respectively.

I have been fiddling with Tim Shepard's "udpburst" tool as a quick
means of measuring head end buffering, even with fq_codel present on
the inbound. (It's not suitable for open internet use as yet, but code
in progress can be had or enhanced at
https://github.com/dtaht/isochronous ). I just added ecn and tos
setting support to it.

server: ./udpburst -S -E -D 32 # Server mode, enable ECN marking, set
dscp to 0x20 (CS1)

client:

This is from a 22Mbit down CMTS

d@nuc:~/git/isochronous$ ./udpburst -f 149.20.63.30 -E -C -d -n 400 -s 1400
1400 bytes -- received 382 of 400 -- 365 consecutive 0 ooo 0 dups 2 ect

........................................................................
........................................................................
........................................................................
........................................................................
........................................................................
.... .. .   ... . ...  . ..    ..  .  .
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2   2 2   2       2 2 2   2   2 2 2     2   2 2         2 2     2

or roughly 512k of buffering.

A DSL link (6400 down)

d@puck:~/git/isochronous$ ./udpburst -f snapon.lab.bufferbloat.net -n
100 -C -d -s 1000
1000 bytes -- received 71 of 100 -- 71 consecutive 0 ooo 0 dups 0 ect

......................................................................

 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

or roughly 64k worth of buffering. Interestingly the bandwidth disparity between
the server (gigE in isc.org's co-lo), is so great that fq_codel can't
kick in before
the 64k dslam buffer is overrun.


> You can look at the netalyzr scatter plots in
> http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/
>
> Now, if someone gives me real fiber to the home, with a real switch fabric
> upstream, rather than gpon life might be somewhat better (if the switches
> aren't themselves overbuffered....  But so far, it isn't.
>                                       - Jim
>
>                                   - Jim
>
>                                                          - Jim
>
>>
>>
>> --
>> Mikael Abrahamsson    email: swmike@swm.pp.se
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>
>
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>



-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Mikael Abrahamsson]

On Tue, 29 Apr 2014, Jim Gettys wrote:

> Why would you think the GPON guys are any better in principle than cable or

Fiber != GPON. Fiber in Sweden is 99% active ethernet.

And the advertisement isn't about bufferbloat, I doubt they even know what 
that is... Upside is that the equipment used in Sweden is usually cheap 
ethernet switches which don't have buffering and might also use policing 
instead of shaping which makes bufferbloat problem basically go away, but 
on the other hand creates other problems.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Tristan Seligmann]

On 29 April 2014 09:08, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> Also, what access method has 300 ms access latency, let alone 3 seconds?
> None that I know of, the meaningful comparison would be ADSL2+ at around
> 25ms and 3G at around 50-100ms.

Latencies on my ADSL2+ connection in Johannesburg, South Africa to my
ISP, a host in the UK, and a host in the USA:

--- www.mweb.co.za ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9011ms
rtt min/avg/max/mdev = 33.720/38.594/66.841/9.507 ms

--- anor.aduial.net ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9010ms
rtt min/avg/max/mdev = 197.966/201.981/211.580/4.619 ms

--- speedtest.atlanta.linode.com ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9001ms
rtt min/avg/max/mdev = 285.859/292.451/331.214/13.285 ms

And of course, I have bufferbloat mitigation in play.
-- 
mithrandi, i Ainil en-Balandor, a faer Ambar

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Hal Murray]

> Also, what access method has 300 ms access latency, let alone 3 seconds?
> None that I know of, the meaningful comparison would be ADSL2+ at around
> 25ms and 3G at around 50-100ms. 

I have an old/slow 256Kbit SDSL link.

I frequently see download bloat delays of over 3 seconds.  I haven't seen 4, 
close, but not quite.


-- 
These are my opinions.  I hate spam.

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Michael Spacefalcon]

Hal Murray <hmurray@megapathdsl.net> wrote:

> I have an old/slow 256Kbit SDSL link.

256?  Really?  Which ISP/CLEC is that?  I thought there were no SDSL
providers left besides Covad/Megapath, and the latter has Nokia (former
Diamond Lane) D50 DSLAMs, whose SDSL speed options are 192, 384, 768,
1152 and 1536 kbps - no 256 in that list.

Sent via my 384 kbps SDSL connection.

SF

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Steinar H. Gunderson]

On Tue, Apr 29, 2014 at 11:26:55PM +0000, Michael Spacefalcon wrote:
> Hal Murray <hmurray@megapathdsl.net> wrote:
> 256?  Really?  Which ISP/CLEC is that?

His email address might give a hint? :-)

/* Steinar */
-- 
Homepage: http://www.sesse.net/

Re: [Bloat] [aqm] the side effects of 330ms lag in the real world

[Hal Murray]

> 256?  Really?  Which ISP/CLEC is that?  I thought there were no SDSL
> providers left besides Covad/Megapath, and the latter has Nokia (former
> Diamond Lane) D50 DSLAMs, whose SDSL speed options are 192, 384, 768, 1152
> and 1536 kbps - no 256 in that list. 

Thanks for the heads-up.  I wonder when my brain warped the numbers?  Yes, 
I'm running at 384.

I remember that many years ago I had something slightly faster.  Maybe 416?  
(384+32)  I think that went away when Covad got in financial trouble and 
Megapath had to switch to somebody else.

In addition to your list, my modem's list also includes 2320.



-- 
These are my opinions.  I hate spam.