[Bloat] ultrafast broadband conference june 27-30

[Bloat] ultrafast broadband conference june 27-30

[Dave Taht]

Is anyone going to this? I sure hope the g.fast technologies have
their uplink bloat fixed, at least.

http://www.ultrafastbroadband.nl/

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Bloat] ultrafast broadband conference june 27-30

[Pedro Tumusok]

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

I might attend, depends on workload, but a colleague is going.

So we get access to the presentations at least.

Pedro

On Mon, Jun 13, 2016 at 6:05 PM, Dave Taht <dave.taht@gmail.com> wrote:

> Is anyone going to this? I sure hope the g.fast technologies have
> their uplink bloat fixed, at least.
>
> http://www.ultrafastbroadband.nl/
>
> --
> Dave Täht
> Let's go make home routers and wifi faster! With better software!
> http://blog.cerowrt.org
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Best regards / Mvh
Jan Pedro Tumusok

[-- Attachment #2: Type: text/html, Size: 1495 bytes --]

Re: [Bloat] ultrafast broadband conference june 27-30

[Jan Ceuleers]

On 13/06/16 18:05, Dave Taht wrote:
> Is anyone going to this? I sure hope the g.fast technologies have
> their uplink bloat fixed, at least.

Dave,

(Been trying to send this for the past few days unsuccessfully; perhaps
the DNS gremlins caused it to bounce. Now sending via gmail).

As I'm sure everyone here knows DSL uses interleaving in order to
achieve coding gain in the face of impulse noise. DSL has to operate in
environments that are rich in such noise so although it is possible to
turn interleaving off pretty much every operator has it on, if for no
other reason than to avoid IPTV artefacts.

This does not meet at least my understanding of the definition of
bufferbloat, in that interleaving introduces a static amount of latency
(i.e. latency that does not vary with load).

But it's perhaps also not what you were talking about because you
mentioned the uplink. Not sure what you meant. Still, other than the use
of ATM and coding I'm not aware of buffering in DSL. Certainly no
dynamic buffers that absorb traffic under load.

Could you enlighten me? I know that you're a busy man; a few keywords
for me to stick into google would be great.

Thanks, Jan

Re: [Bloat] ultrafast broadband conference june 27-30

[Jonathan Morton]

> On 18 Jun, 2016, at 11:04, Jan Ceuleers <jan.ceuleers@gmail.com> wrote:
> 
> As I'm sure everyone here knows DSL uses interleaving in order to
> achieve coding gain in the face of impulse noise. DSL has to operate in
> environments that are rich in such noise so although it is possible to
> turn interleaving off pretty much every operator has it on, if for no
> other reason than to avoid IPTV artefacts.
> 
> This does not meet at least my understanding of the definition of
> bufferbloat, in that interleaving introduces a static amount of latency
> (i.e. latency that does not vary with load).
> 
> But it's perhaps also not what you were talking about because you
> mentioned the uplink. Not sure what you meant. Still, other than the use
> of ATM and coding I'm not aware of buffering in DSL. Certainly no
> dynamic buffers that absorb traffic under load.
> 
> Could you enlighten me? I know that you're a busy man; a few keywords
> for me to stick into google would be great.

These buffers are not inherent to the link technology.  If they were, we’d be calling for different link technology.  Rather, they are contained in the devices at each end of the link.  It is the device at the *entry* to the link which matters; a device at the ISP end for the downlink, and at the customer end for the uplink.

Consider a railway with frequent, direct services between two popular cities - take Osaka and Tokyo, connected by the Tokaido Shinkansen, for a realistic example.  Each train can carry several hundred passengers; if there aren’t enough seats for everyone waiting, some passengers will be left on the platform (the days of stuffing commuters onto trains are over, even in Tokyo).

Only so many trains per hour will fit on the track, however fast they travel along it; indeed, the faster they go, the fewer can be scheduled, because braking distance scales with the square of speed, and trains are kept a full braking distance apart.  And it is hard to make trains longer than they are already; the 16-car Series 700 trains are already nearly a quarter-mile long, and passengers just don’t want to walk that far.

These factors limit the passenger-carrying capacity of the entire railway, to the point where a second high-speed railway (the Chūō Shinkansen) is being built between the same two cities, along a different route.  However, it is not yet complete and won’t be for some years.

Now consider the choices made by each station’s architect.  He can only provide a certain number of platforms to hold trains for efficient cleaning and loading - both are very crowded cities.  Leaving customers on the platform is also bad for business, both in terms of revenue and reputation.  So on the fastest trains which run non-stop between the major cities, all the seats are to be reserved for individual passengers.  A reservation can be booked the same day, at the station, but before reaching the platform.

So passengers are directed to a platform where they can always find a seat on the train waiting there - but *until* the train is waiting there, they must wait in the concourse.  So the architect builds a large and pleasant concourse with lots of cafes and bookshops and waiting benches and so on.  There is plenty of space to do so *above* the Shinkansen platforms, not to mention above the platforms of the huge commuter station next door.

This suits tourists and holidaymakers just fine.  They don’t mind waiting as much as an hour in a pleasant concourse at busy times of the day.  It’s all part of the journey.

But to a business traveller, time is money, and he has a meeting appointment in the other city which he Must Not be late for.  (You know these modern Japanese business practices…)

If he regularly has to wait an hour to get a seat on a train, he’s just as likely to turn around and book a flight next time.  It’s almost as fast, door to door, and the extra cost is just a business expense.  If tourists and holidaymakers take up seats that he might have been able to use, the railway is no longer useful to him.

The concourse is the buffer, and the simplistic reservation system represents a basic FIFO queue.

How the railway actually copes with this problem is by charging extra for reserved seats, premier-class seats and for the fastest trains.  This encourages tourists and holidaymakers to take the slightly slower all-stops trains which have a generous allocation of unreserved seats, leaving the fastest trains entirely for people in a genuine hurry.  People also generally make reservations in advance when possible, to ensure they get on a train at their preferred time.  There are still people waiting in the concourse, but they’re not the people who particularly mind waiting.

This solution is roughly equivalent to “fair queuing” or “flow isolation”, though the analogy is imperfect since no money or resource exchange is involved in Internet queuing systems.

Taking the analogy further, let’s look at what happens if too many people are waiting for trains, so the concourse fills up despite its size.  This will always happen if people continuously arrive at a greater rate than the railway’s capacity.  The waiting time grows to five hours, which is intolerable for even the most patient tourists.  The cafes sell out of sandwiches and coffee, and shut their doors.  Passengers who read the writing on the wall and booked a week in advance have to fight through the crowd to reach the platforms.

It’s dangerous to have too many people in an enclosed space; you tend to get stampedes, especially if something alarming happens.  So the police are called in to keep order; they stop people going in (including people with prior reservations, who then miss their train and are angry about it, while that much capacity on departing trains goes unused), and arrest people who have started fighting over who was first in the queue for the ticket office.  Or the toilets - it’s hard to tell which queue is which any more, as it’s just a solid mass of humanity.  Keeping order in such a big concourse is *hard* - the police have difficulty moving in the crowd too.

If the concourse hadn’t been built so large, keeping order would be easier.  The problem would have been spotted earlier, before the waiting time grew so long and people started shoving each other in the queues.  The booking office could sensibly have been placed by the front doors, so people would know how long they had to wait before they even got inside.  People with prior reservations wouldn’t have had so much of a crowd to fight through.  The cafes and bookshops simply set up in the plaza outside, instead of in the concourse itself, and people waiting can circulate more freely.

Similarly, there is an optimum size for a FIFO queue: it is related to the link capacity, the expected RTT of paths using the link, and the number of simultaneous flows.  Specifically (and IIRC), it is Bandwidth*RTT/sqrt(flows). This is in itself a dynamic quantity, but it is reasonably sensible to assume 1 flow and 100ms RTT for a consumer Internet-edge link.  Increasing the buffer size beyond that gives little or no improvement in link utilisation efficiency, and increases delay which is harmful.

 - Jonathan Morton

Re: [Bloat] ultrafast broadband conference june 27-30

[Jan Ceuleers]

On 18/06/16 12:09, Jonathan Morton wrote:
> These buffers are not inherent to the link technology.
> If they were, we’d be calling for different link technology.

Thanks for confirming my suspicions. But this is exactly what Dave was
doing, since he blamed the uplink bloat on "g.fast technologies".

So it seems that you and I agree that it's not G.fast itself that is
bloated, but the lack of proper buffer management in certain (most?) DSL
modems, regardless of whether they support G.fast or some other flavour
of DSL.

Thx, Jan

Re: [Bloat] ultrafast broadband conference june 27-30

[Jonathan Morton]

> On 18 Jun, 2016, at 13:20, Jan Ceuleers <jan.ceuleers@gmail.com> wrote:
> 
>> These buffers are not inherent to the link technology.
>> If they were, we’d be calling for different link technology.
> 
> Thanks for confirming my suspicions. But this is exactly what Dave was
> doing, since he blamed the uplink bloat on "g.fast technologies".
> 
> So it seems that you and I agree that it's not G.fast itself that is
> bloated, but the lack of proper buffer management in certain (most?) DSL
> modems, regardless of whether they support G.fast or some other flavour
> of DSL.

Right.  We’re hoping that G.fast, being a new link technology, has taken on board some of the extensive research into buffer sizing and management, in typical device implementations.  But we’re not holding our breath - the industry is rather stubborn on this point.

 - Jonathan Morton

Re: [Bloat] ultrafast broadband conference june 27-30

[Jan Ceuleers]

On 18/06/16 12:24, Jonathan Morton wrote:
>> So it seems that you and I agree that it's not G.fast itself that is
>> bloated, but the lack of proper buffer management in certain (most?) DSL
>> modems, regardless of whether they support G.fast or some other flavour
>> of DSL.
> 
> Right.  We’re hoping that G.fast, being a new link technology, has
> taken on board some of the extensive research into buffer sizing and
> management, in typical device implementations.  But we’re not holding
> our breath - the industry is rather stubborn on this point.

We're still not on the same page. So let me again try to decode what you
(and perhaps Dave) are saying:

Either you perceive the problem to be located in the link technology
(i.e. DSL generally or only specific flavours of it). If this is the
case what needs to be fixed is the standard so that implementations
thereof will improve.

Or else you perceive the problem to be located in the CPE that implement
DSL, but in the layer above the DSL link layer. In this case what needs
to be fixed is those implementations, probably starting by the reference
firmware written by chipset vendors.

I think it's the latter. If it's the former then indeed don't hold your
breath because the standardisation is done and dusted.

Re: [Bloat] ultrafast broadband conference june 27-30

[Toke Høiland-Jørgensen]

Jan Ceuleers <jan.ceuleers@gmail.com> writes:

> Either you perceive the problem to be located in the link technology
> (i.e. DSL generally or only specific flavours of it). If this is the
> case what needs to be fixed is the standard so that implementations
> thereof will improve.
>
> Or else you perceive the problem to be located in the CPE that implement
> DSL, but in the layer above the DSL link layer. In this case what needs
> to be fixed is those implementations, probably starting by the reference
> firmware written by chipset vendors.
>
> I think it's the latter. If it's the former then indeed don't hold your
> breath because the standardisation is done and dusted.

It is indeed the latter. However, it is correlated with DSL technology
because the equipment tend to be using binary blobs for drivers that
themselves have huge buffers; so even if the device is running Linux,
sticking FQ-CoDel on it doesn't do much good without a software rate
limiter. And also, most devices are owned by the ISP, so the consumer
can't upgrade them anyway.

So while it is nothing inherent in the technology per se, in practice it
is a fairly safe bet to say "ah, you're on DSL? Well, you are probably
suffering from bufferbloat, then".

I know of at least one DSL vendor who supposedly has started paying
attention after pressure from a clueful ISP; not idea if they started
shipping non-bloated products yet.

-Toke

Re: [Bloat] ultrafast broadband conference june 27-30

[Jan Ceuleers]

On 18/06/16 12:51, Toke Høiland-Jørgensen wrote:
> I know of at least one DSL vendor who supposedly has started paying
> attention after pressure from a clueful ISP; not idea if they started
> shipping non-bloated products yet.

Got it.

To get more ISPs interested in this subject we need to get Ookla to take
account of bufferbloat. So long as speedtest.net and the Ookla servers
many ISPs have in their labs ignore latency under load there will not be
any pressure on the CPE industry to clean up its act.

Re: [Bloat] ultrafast broadband conference june 27-30

[Toke Høiland-Jørgensen]

Jan Ceuleers <jan.ceuleers@gmail.com> writes:

> On 18/06/16 12:51, Toke Høiland-Jørgensen wrote:
>> I know of at least one DSL vendor who supposedly has started paying
>> attention after pressure from a clueful ISP; not idea if they started
>> shipping non-bloated products yet.
>
> Got it.
>
> To get more ISPs interested in this subject we need to get Ookla to take
> account of bufferbloat. So long as speedtest.net and the Ookla servers
> many ISPs have in their labs ignore latency under load there will not be
> any pressure on the CPE industry to clean up its act.

Yup. Unfortunately we've had no luck thus far.

https://www.dslreports.com/speedtest is an alternative speedtest that
does include bufferbloat scores.

-Toke

Re: [Bloat] ultrafast broadband conference june 27-30

[Jan Ceuleers]

On 18/06/16 13:14, Toke Høiland-Jørgensen wrote:
> Yup. Unfortunately we've had no luck thus far.
> 
> https://www.dslreports.com/speedtest is an alternative speedtest that
> does include bufferbloat scores.

Yes, but it isn't used by ISPs when they qualify product.

Would it be worth pursuing an RFC on how to properly test the
performance of an internet access circuit and/or of a path across an IP
network? RFC compliance might be a tool to get Ookla to take notice.

Re: [Bloat] ultrafast broadband conference june 27-30

[Jonathan Morton]

> On 18 Jun, 2016, at 14:22, Jan Ceuleers <jan.ceuleers@gmail.com> wrote:
> 
> Would it be worth pursuing an RFC on how to properly test the
> performance of an internet access circuit and/or of a path across an IP
> network? RFC compliance might be a tool to get Ookla to take notice.

I think that could happen, if someone sits down and drafts one.  The “AQM Characterization Guidelines” one is coming along nicely, and is obviously related.

 - Jonathan Morton

Re: [Bloat] ultrafast broadband conference june 27-30

[Dave Taht]

On Sat, Jun 18, 2016 at 6:55 AM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 18 Jun, 2016, at 14:22, Jan Ceuleers <jan.ceuleers@gmail.com> wrote:
>>
>> Would it be worth pursuing an RFC on how to properly test the
>> performance of an internet access circuit and/or of a path across an IP
>> network? RFC compliance might be a tool to get Ookla to take notice.

My answer has become to try to "get in the face" of more existing
standards bodies working at these layers.

Before that, it was "find a way to sit down with the actual driver
developer to go make these *very simple* changes to the underlying
firmware blob". A BQL-like technique is *not rocket science*. My
estimate was less than 3 days worth of engineering for someone
familiar with the firmware source, coupled with someone familiar with
the queue theory.

Merely limiting the maximum number of packets queued in the firmware
to 2x what is needed for interleave at that rate to work, is even
simpler, if less effective.

> I think that could happen, if someone sits down and drafts one.  The “AQM Characterization Guidelines” one is coming along nicely, and is obviously related.
>
>  - Jonathan Morton
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Bloat] ultrafast broadband conference june 27-30

[Dave Taht]

On Sat, Jun 18, 2016 at 4:14 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Jan Ceuleers <jan.ceuleers@gmail.com> writes:
>
>> On 18/06/16 12:51, Toke Høiland-Jørgensen wrote:
>>> I know of at least one DSL vendor who supposedly has started paying
>>> attention after pressure from a clueful ISP; not idea if they started
>>> shipping non-bloated products yet.
>>
>> Got it.
>>
>> To get more ISPs interested in this subject we need to get Ookla to take
>> account of bufferbloat. So long as speedtest.net and the Ookla servers
>> many ISPs have in their labs ignore latency under load there will not be
>> any pressure on the CPE industry to clean up its act.
>
> Yup. Unfortunately we've had no luck thus far.

Correction: free.fr made the requisite changes in their revolution V6
DSL firmware,
and shipped it to customers, 3 months after fq_codel was released.

> https://www.dslreports.com/speedtest is an alternative speedtest that
> does include bufferbloat scores.
>
> -Toke
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org