[Ecn-sane] [tsvwg] ECN CE that was ECT(0) incorrectly classified as L4S

[Dave Taht]

On Fri, Aug 2, 2019 at 4:10 AM Dave Taht <dave.taht at gmail.com> wrote:
>
> On Fri, Aug 2, 2019 at 2:47 AM Jonathan Morton <chromatix99 at gmail.com> wrote:
> >
> > > On 2 Aug, 2019, at 9:29 am, <Ruediger.Geib at telekom.de> <Ruediger.Geib at telekom.de> wrote:
> > >
> > > Hi Jonathan,
> > >
> > > could you provide a real world example of links which are consecutively narrower than sender access links?
> > >
> > > I could figure out a small campus network which has a bottleneck at the Internet access and a second one connecting the terminal equipment. But in a small campus network, the individual terminal could very well have a higher LAN access bandwidth, than the campus - Internet connection (and then there's only one bottleneck again).
> > >
> > > There may be a tradeoff between simplicity and general applicability. Awareness of that tradeoff is important. To me, simplicity is the design aim.
> >
> > A progressive narrowing of effective link capacity is very common in consumer Internet access.  Theoretically you can set up a chain of almost unlimited length of consecutively narrowing bottlenecks, such that a line-rate burst injected at the wide end will experience queuing at every intermediate node.  In practice you can expect typically three or more potentially narrowing points:
>
> 0: Container and vm users are frequently using htb + something to keep
> their bandwidths under control.
>
> 0.5: Cloudy providers use "something" to also rate limit traffic.
> Policers and shapers, I assume.

Stuff in the cloud thus far is looking quite jittery; sub-ms marking
thresholds do not look feasible. I have no idea what sorts of
software jitter and burstyness exist in NFV and ddpk based implementatons.

>
> > 1: Core to Access network peering.  Most responsible ISPs regularly upgrade these links to minimise congestion, subject to cost effectiveness.  Some consumer ISPs however are less responsible, and permit regular congestion here, often on a daily and/or weekly cycle according to demand.  Even the responsible ones may experience short-term congestion here due to exceptional events.  Even if the originating server's NIC is slower than the peering link, queuing may occur here if the link is congested overall due to statistical multiplexing.
> >
> > 2: Access to Backhaul provisioning shaper.  Many ISPs have a provisioning shaper to handle "poverty tariffs" with deliberately limited capacity.  It may also be used to impose a sanity check on inbound traffic bursts, to limit backhaul network traffic to that actually deliverable to the customer (especially when the backhaul network is itself subcontracted on a gigabytes-carried basis, as is common in the UK).  In the ADSL context it's often called a BRAS.
> >
> > 3: Backhaul to head-end device.  Generally the backhaul network is sized to support many head-end devices, each of which serves some number of consumer last-mile links.  I'm being deliberately generic here; it could be a CMTS on a cable network, a DSLAM in a phone exchange, a cell tower, or a long-range wifi hub.  In many cases the head-end device shares several subscribers' lines over a common last-mile medium, so there is still some statistical multiplexing.  In the particular case of a cell tower, the subscriber usually gets less link capacity (due to propagation conditions) than his tariff limit.

There are also bursts here from the vpn crypto engine.

> > 4: CPE bufferbloat mitigation shaper.  This is *post-last-mile* ingress shaping, with AQM and FQ, to reduce the effects of the still-ubiquitous dumb FIFOs on the above bottlenecks, especially the head-end device.  The IQrouter is a ready-made CPE device which does this.

I would say "inbound shaper" to be clear. And its far, far wider than
just that commercial product, Nearly everyone using this stuff
shapes both in and outbound - be it untangle, netduma, asus "adaptive
qos", edgerouter's or eero's sqm implemention, all of openwrt, dd-wrt,
and deratives, bsd-based pfsense and opfsense, preseem does a bump in
the wire for WISPs, streamboost, I think the list of off the shelf
home router qos systems that shape inbound is well above 80-90%, and
users have been trained to turn it on in both directions
and off only when they run out of cpu. We see new product sales driven
by the cpu cost of having to shape inbound, too.

Policers have become quite useless in the past decade.

> >
> > 5: LAN, wifi, or powerline link.  Most people now have GigE, but 100base-TX is still in use in cheaper CPE and some low-end

I'd so love to see powerline gain fq and AQM. I see it used a lot in
busy apt buildings to drag stuff from room to room. It can be *awful*

>computers, and this will be a further bottleneck in cases where the last mile is faster.  For example, the Raspberry Pi has only just upgraded to GigE in its latest versions, and used 100base-TX before.  Wifi is also a likely bottleneck, especially if the mobile station is on the opposite side of the house from the base CPE, and is additionally a "bursty MAC" with heavy aggregation.  Some CPE now runs airtime-fairness and FQ-AQM on wifi to help manage that.

"some" includes most of qca's ath9k or ath10k products. (40% of the AP
market?) Prominently known fq-codel for wifi users are ~3m chromebook
users and ~3m google wifi, (
http://flent-newark.bufferbloat.net/~d/Airtime%20based%20queue%20limit%20for%20FQ_CoDel%20in%20wireless%20interface.pdf
) and nearly everyone else in the 802.11s meshy market... meraki is a
known sfq + codel user... starting in 2014...

A large number of wifi APs and p2p links oft have a faster wifi speed
than their 100base-tx link, and thus use the ethernet (with either a
short fifo or fq_codel) to smooth the bursts out. I can point to a few
ubnt products that I know a bit too much about. There's also some
802.11ad and ay.... there is a quite remarkable amount of 100base-tx
gear still being deployed.

If it helps any to those here doing simulation, long ago I put a
"slot" and statistical distribution of busty mac delay feature into
netem. I can say now that that was used to help guide the development
of google "stadia", and certainly "slotting" is a MAJOR tool that I'd
plug into the SCE work to better emulate the real orld behaviors of
bursty macs. google has collected WAY more examples characterizing
real world micro(bursts) than I could ever deal with, and I hope they
publish that data someday for others to use.

> >
> > I think the above collection is not at all exotic.  Not all of these will apply in every case, or at all times in any given case, but it is certainly possible for all five to apply consecutively.

For bursty.

> >  - Jonathan Morton
> > _______________________________________________
> > Ecn-sane mailing list
> > Ecn-sane at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/ecn-sane
>
>
>
> --
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740