[Bloat] ultrafast broadband conference june 27-30

[Jonathan Morton]

> On 18 Jun, 2016, at 11:04, Jan Ceuleers <jan.ceuleers at gmail.com> wrote:
> 
> As I'm sure everyone here knows DSL uses interleaving in order to
> achieve coding gain in the face of impulse noise. DSL has to operate in
> environments that are rich in such noise so although it is possible to
> turn interleaving off pretty much every operator has it on, if for no
> other reason than to avoid IPTV artefacts.
> 
> This does not meet at least my understanding of the definition of
> bufferbloat, in that interleaving introduces a static amount of latency
> (i.e. latency that does not vary with load).
> 
> But it's perhaps also not what you were talking about because you
> mentioned the uplink. Not sure what you meant. Still, other than the use
> of ATM and coding I'm not aware of buffering in DSL. Certainly no
> dynamic buffers that absorb traffic under load.
> 
> Could you enlighten me? I know that you're a busy man; a few keywords
> for me to stick into google would be great.

These buffers are not inherent to the link technology.  If they were, we’d be calling for different link technology.  Rather, they are contained in the devices at each end of the link.  It is the device at the *entry* to the link which matters; a device at the ISP end for the downlink, and at the customer end for the uplink.

Consider a railway with frequent, direct services between two popular cities - take Osaka and Tokyo, connected by the Tokaido Shinkansen, for a realistic example.  Each train can carry several hundred passengers; if there aren’t enough seats for everyone waiting, some passengers will be left on the platform (the days of stuffing commuters onto trains are over, even in Tokyo).

Only so many trains per hour will fit on the track, however fast they travel along it; indeed, the faster they go, the fewer can be scheduled, because braking distance scales with the square of speed, and trains are kept a full braking distance apart.  And it is hard to make trains longer than they are already; the 16-car Series 700 trains are already nearly a quarter-mile long, and passengers just don’t want to walk that far.

These factors limit the passenger-carrying capacity of the entire railway, to the point where a second high-speed railway (the Chūō Shinkansen) is being built between the same two cities, along a different route.  However, it is not yet complete and won’t be for some years.

Now consider the choices made by each station’s architect.  He can only provide a certain number of platforms to hold trains for efficient cleaning and loading - both are very crowded cities.  Leaving customers on the platform is also bad for business, both in terms of revenue and reputation.  So on the fastest trains which run non-stop between the major cities, all the seats are to be reserved for individual passengers.  A reservation can be booked the same day, at the station, but before reaching the platform.

So passengers are directed to a platform where they can always find a seat on the train waiting there - but *until* the train is waiting there, they must wait in the concourse.  So the architect builds a large and pleasant concourse with lots of cafes and bookshops and waiting benches and so on.  There is plenty of space to do so *above* the Shinkansen platforms, not to mention above the platforms of the huge commuter station next door.

This suits tourists and holidaymakers just fine.  They don’t mind waiting as much as an hour in a pleasant concourse at busy times of the day.  It’s all part of the journey.

But to a business traveller, time is money, and he has a meeting appointment in the other city which he Must Not be late for.  (You know these modern Japanese business practices…)

If he regularly has to wait an hour to get a seat on a train, he’s just as likely to turn around and book a flight next time.  It’s almost as fast, door to door, and the extra cost is just a business expense.  If tourists and holidaymakers take up seats that he might have been able to use, the railway is no longer useful to him.

The concourse is the buffer, and the simplistic reservation system represents a basic FIFO queue.

How the railway actually copes with this problem is by charging extra for reserved seats, premier-class seats and for the fastest trains.  This encourages tourists and holidaymakers to take the slightly slower all-stops trains which have a generous allocation of unreserved seats, leaving the fastest trains entirely for people in a genuine hurry.  People also generally make reservations in advance when possible, to ensure they get on a train at their preferred time.  There are still people waiting in the concourse, but they’re not the people who particularly mind waiting.

This solution is roughly equivalent to “fair queuing” or “flow isolation”, though the analogy is imperfect since no money or resource exchange is involved in Internet queuing systems.

Taking the analogy further, let’s look at what happens if too many people are waiting for trains, so the concourse fills up despite its size.  This will always happen if people continuously arrive at a greater rate than the railway’s capacity.  The waiting time grows to five hours, which is intolerable for even the most patient tourists.  The cafes sell out of sandwiches and coffee, and shut their doors.  Passengers who read the writing on the wall and booked a week in advance have to fight through the crowd to reach the platforms.

It’s dangerous to have too many people in an enclosed space; you tend to get stampedes, especially if something alarming happens.  So the police are called in to keep order; they stop people going in (including people with prior reservations, who then miss their train and are angry about it, while that much capacity on departing trains goes unused), and arrest people who have started fighting over who was first in the queue for the ticket office.  Or the toilets - it’s hard to tell which queue is which any more, as it’s just a solid mass of humanity.  Keeping order in such a big concourse is *hard* - the police have difficulty moving in the crowd too.

If the concourse hadn’t been built so large, keeping order would be easier.  The problem would have been spotted earlier, before the waiting time grew so long and people started shoving each other in the queues.  The booking office could sensibly have been placed by the front doors, so people would know how long they had to wait before they even got inside.  People with prior reservations wouldn’t have had so much of a crowd to fight through.  The cafes and bookshops simply set up in the plaza outside, instead of in the concourse itself, and people waiting can circulate more freely.

Similarly, there is an optimum size for a FIFO queue: it is related to the link capacity, the expected RTT of paths using the link, and the number of simultaneous flows.  Specifically (and IIRC), it is Bandwidth*RTT/sqrt(flows). This is in itself a dynamic quantity, but it is reasonably sensible to assume 1 flow and 100ms RTT for a consumer Internet-edge link.  Increasing the buffer size beyond that gives little or no improvement in link utilisation efficiency, and increases delay which is harmful.

 - Jonathan Morton