[Bloat] Really getting 1G out of ISP?

[Bloat] Really getting 1G out of ISP?

[Stephen Hemminger]

Is there any consumer hardware that can actually keep up and do AQM at 1Gbit.
It seems everyone seems obsessed with gamer Wifi 6. But can only do 300Mbit single
stream with any kind of QoS.

It doesn't help that all the local ISP's claim 10Mbit upload even with 1G download.
Is this a head end provisioning problem or related to Docsis 3.0 (or later) modems?

Re: [Bloat] Really getting 1G out of ISP?

[Sebastian Moeller]

On 22 June 2021 06:00:48 CEST, Stephen Hemminger <stephen@networkplumber.org> wrote:
>Is there any consumer hardware that can actually keep up and do AQM at
>1Gbit.

        Over in the OpenWrt forums the same question pops up routinely once per week. The best answer ATM seems to be a combination of a raspberry pi4B with a decent USB3 gigabit ethernet dongle, a managed switch and any capable (OpenWrt) AP of the user's liking. With 4 arm A72 cores the will traffic shape up to a gigabit as reported by multiple users.


>It seems everyone seems obsessed with gamer Wifi 6. But can only do
>300Mbit single
>stream with any kind of QoS.

IIUC most commercial home routers/APs bet on offload engines to do most of the heavy lifting, but as far as I understand only the NSS cores have a shaper and fq_codel module....


>
>It doesn't help that all the local ISP's claim 10Mbit upload even with
>1G download.
>Is this a head end provisioning problem or related to Docsis 3.0 (or
>later) modems?

For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes .
Over here the incumbent cable isp provisions  fifty Mbps for upstream and plans to increase that to hundred once the upstream is switched to docsis 3.1.
I believe one issue is that since most of the upstream is required for the reverse ACK traffic for the download and hence it can not be oversubscribed too much.... but I think we have real docsis experts on the list, so I will stop my speculation here...

Regards
         Sebastian




>
>_______________________________________________
>Bloat mailing list
>Bloat@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/bloat

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Bloat] Really getting 1G out of ISP?

[Giuseppe De Luca]

Also a PC Engines APU4 will do the job
(https://inonius.net/results/?userId=17996087f5e8 - this is a
1gbit/1gbit, with Openwrt/sqm-scripts set to 900/900.  ISP is Sony NURO
in Japan). Will follow this thread to know if some interesting device
popup :)


https://inonius.net/results/?userId=17996087f5e8

On 6/22/2021 6:12 AM, Sebastian Moeller wrote:
>
> On 22 June 2021 06:00:48 CEST, Stephen Hemminger <stephen@networkplumber.org> wrote:
>> Is there any consumer hardware that can actually keep up and do AQM at
>> 1Gbit.
>          Over in the OpenWrt forums the same question pops up routinely once per week. The best answer ATM seems to be a combination of a raspberry pi4B with a decent USB3 gigabit ethernet dongle, a managed switch and any capable (OpenWrt) AP of the user's liking. With 4 arm A72 cores the will traffic shape up to a gigabit as reported by multiple users.
>
>
>> It seems everyone seems obsessed with gamer Wifi 6. But can only do
>> 300Mbit single
>> stream with any kind of QoS.
> IIUC most commercial home routers/APs bet on offload engines to do most of the heavy lifting, but as far as I understand only the NSS cores have a shaper and fq_codel module....
>
>
>> It doesn't help that all the local ISP's claim 10Mbit upload even with
>> 1G download.
>> Is this a head end provisioning problem or related to Docsis 3.0 (or
>> later) modems?
> For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes .
> Over here the incumbent cable isp provisions  fifty Mbps for upstream and plans to increase that to hundred once the upstream is switched to docsis 3.1.
> I believe one issue is that since most of the upstream is required for the reverse ACK traffic for the download and hence it can not be oversubscribed too much.... but I think we have real docsis experts on the list, so I will stop my speculation here...
>
> Regards
>           Sebastian
>
>
>
>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Really getting 1G out of ISP?

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 2670 bytes --]

I'm running an Odyssey from Seeed Studios (celeron J4125 with dual i211),
and it can handle Cake at 1Gbps on a single core (which it needs to,
because OpenWRT's i211 support still has multiple receive queues disabled).

On Tue, Jun 22, 2021 at 12:44 AM Giuseppe De Luca <dropheaders@gmx.com>
wrote:

> Also a PC Engines APU4 will do the job
> (https://inonius.net/results/?userId=17996087f5e8 - this is a
> 1gbit/1gbit, with Openwrt/sqm-scripts set to 900/900.  ISP is Sony NURO
> in Japan). Will follow this thread to know if some interesting device
> popup :)
>
>
> https://inonius.net/results/?userId=17996087f5e8
>
> On 6/22/2021 6:12 AM, Sebastian Moeller wrote:
> >
> > On 22 June 2021 06:00:48 CEST, Stephen Hemminger <
> stephen@networkplumber.org> wrote:
> >> Is there any consumer hardware that can actually keep up and do AQM at
> >> 1Gbit.
> >          Over in the OpenWrt forums the same question pops up routinely
> once per week. The best answer ATM seems to be a combination of a raspberry
> pi4B with a decent USB3 gigabit ethernet dongle, a managed switch and any
> capable (OpenWrt) AP of the user's liking. With 4 arm A72 cores the will
> traffic shape up to a gigabit as reported by multiple users.
> >
> >
> >> It seems everyone seems obsessed with gamer Wifi 6. But can only do
> >> 300Mbit single
> >> stream with any kind of QoS.
> > IIUC most commercial home routers/APs bet on offload engines to do most
> of the heavy lifting, but as far as I understand only the NSS cores have a
> shaper and fq_codel module....
> >
> >
> >> It doesn't help that all the local ISP's claim 10Mbit upload even with
> >> 1G download.
> >> Is this a head end provisioning problem or related to Docsis 3.0 (or
> >> later) modems?
> > For DOCSIS the issue seems to be an unfortunate frequency split between
> up and downstream and use of lower efficiency coding schemes .
> > Over here the incumbent cable isp provisions  fifty Mbps for upstream
> and plans to increase that to hundred once the upstream is switched to
> docsis 3.1.
> > I believe one issue is that since most of the upstream is required for
> the reverse ACK traffic for the download and hence it can not be
> oversubscribed too much.... but I think we have real docsis experts on the
> list, so I will stop my speculation here...
> >
> > Regards
> >           Sebastian
> >
> >
> >
> >
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 3843 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Dave Taht]

On Tue, Jul 6, 2021 at 3:32 PM Aaron Wood <woody77@gmail.com> wrote:
>
> I'm running an Odyssey from Seeed Studios (celeron J4125 with dual i211), and it can handle Cake at 1Gbps on a single core (which it needs to, because OpenWRT's i211 support still has multiple receive queues disabled).

Not clear if that is shaped or not? Line rate is easy on processors of
that class or better, but shaped?

some points:

On inbound shaping especially it it still best to lock network traffic
to a single core in low end platforms.

Cake itself is not multicore, although the design essentially is. We
did some work towards trying to make it shape across multiple cores
and multiple hardware queues. IF the locking contention could be
minimized (RCU) I felt it possible for a win here, but a bigger win
would be to eliminate "mirred" from the ingress path entirely.

Even multiple transmit queues remains kind of dicy in linux, and
actually tend to slow network processing in most cases I've tried at
gbit line rates. They also add latency, as (1) BQL is MIAD, not AIMD,
so it stays "stuck" at a "good" level for a long time, AND 2) each hw
queue gets an additive fifo at this layer, so where, you might need
only 40k to keep a single hw queue busy, you end up with 160k with 4
hw queues. This problem is getting worse and worse (64 queues are
common in newer hardware, 1000s in really new hardware) and a revisit
to how BQL does things in this case would be useful. Ideally it would
share state (with a cross core variable and atomic locks) as to how
much total buffering was actually needed "down there" across all the
queues, but without trying it, I worry that that would end up costing
a lot of cpu cycles.

Feel free to experiment with multiple transmit queues locked to other
cores with the set-affinity bits in /proc/interrupts. I'm sure these
MUST be useful on some platform, but I think most of the use for
multiple hw queues is when a locally processing application  is
getting the data, not when it is being routed.

Ironically, I guess, the shorter your queues the higher likelihood a
given packet will remain in l2 or even l1 cache.

I
>
> On Tue, Jun 22, 2021 at 12:44 AM Giuseppe De Luca <dropheaders@gmx.com> wrote:
>>
>> Also a PC Engines APU4 will do the job
>> (https://inonius.net/results/?userId=17996087f5e8 - this is a
>> 1gbit/1gbit, with Openwrt/sqm-scripts set to 900/900.  ISP is Sony NURO
>> in Japan). Will follow this thread to know if some interesting device
>> popup :)
>>
>>
>> https://inonius.net/results/?userId=17996087f5e8
>>
>> On 6/22/2021 6:12 AM, Sebastian Moeller wrote:
>> >
>> > On 22 June 2021 06:00:48 CEST, Stephen Hemminger <stephen@networkplumber.org> wrote:
>> >> Is there any consumer hardware that can actually keep up and do AQM at
>> >> 1Gbit.
>> >          Over in the OpenWrt forums the same question pops up routinely once per week. The best answer ATM seems to be a combination of a raspberry pi4B with a decent USB3 gigabit ethernet dongle, a managed switch and any capable (OpenWrt) AP of the user's liking. With 4 arm A72 cores the will traffic shape up to a gigabit as reported by multiple users.
>> >
>> >
>> >> It seems everyone seems obsessed with gamer Wifi 6. But can only do
>> >> 300Mbit single
>> >> stream with any kind of QoS.
>> > IIUC most commercial home routers/APs bet on offload engines to do most of the heavy lifting, but as far as I understand only the NSS cores have a shaper and fq_codel module....
>> >
>> >
>> >> It doesn't help that all the local ISP's claim 10Mbit upload even with
>> >> 1G download.
>> >> Is this a head end provisioning problem or related to Docsis 3.0 (or
>> >> later) modems?
>> > For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes .
>> > Over here the incumbent cable isp provisions  fifty Mbps for upstream and plans to increase that to hundred once the upstream is switched to docsis 3.1.
>> > I believe one issue is that since most of the upstream is required for the reverse ACK traffic for the download and hence it can not be oversubscribed too much.... but I think we have real docsis experts on the list, so I will stop my speculation here...
>> >
>> > Regards
>> >           Sebastian
>> >
>> >
>> >
>> >
>> >> _______________________________________________
>> >> Bloat mailing list
>> >> Bloat@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/bloat
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Latest Podcast:
https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/

Dave Täht CTO, TekLibre, LLC

Re: [Bloat] Really getting 1G out of ISP?

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 7579 bytes --]

On Tue, Jul 6, 2021 at 7:26 PM Dave Taht <dave.taht@gmail.com> wrote:

> On Tue, Jul 6, 2021 at 3:32 PM Aaron Wood <woody77@gmail.com> wrote:
> >
> > I'm running an Odyssey from Seeed Studios (celeron J4125 with dual
> i211), and it can handle Cake at 1Gbps on a single core (which it needs to,
> because OpenWRT's i211 support still has multiple receive queues disabled).
>
> Not clear if that is shaped or not? Line rate is easy on processors of
> that class or better, but shaped?
>

That's shaped.  I can shape 800+, and the kernel ramps the clock rate up to
2.5GHz as needed, IIRC.  I'm guessing that it might thermally limit at some
point, but I haven't had sustained >500Mbps traffic for long enough to
really exercise that.  Although the covid WFH and has definitely increased
the likelihood that I'm hitting >500Mbps downloads.


> some points:
>
> On inbound shaping especially it it still best to lock network traffic
> to a single core in low end platforms.
>
> Cake itself is not multicore, although the design essentially is. We
> did some work towards trying to make it shape across multiple cores
> and multiple hardware queues. IF the locking contention could be
> minimized (RCU) I felt it possible for a win here, but a bigger win
> would be to eliminate "mirred" from the ingress path entirely.
>

I was going to play around with shaping to lower levels across multiple
cores, as many of the loads I deal with are multi-stream, but I always
worry about the ack path, as the provisioned rates are so asymmetric
(35Mbps up).  I'm using `ack-filter-aggressive` on egress to help.  I've
found that the most aggressive ack filtering seems to hurt throughput.




>
> Even multiple transmit queues remains kind of dicy in linux, and
> actually tend to slow network processing in most cases I've tried at
> gbit line rates. They also add latency, as (1) BQL is MIAD, not AIMD,
> so it stays "stuck" at a "good" level for a long time, AND 2) each hw
> queue gets an additive fifo at this layer, so where, you might need
> only 40k to keep a single hw queue busy, you end up with 160k with 4
> hw queues. This problem is getting worse and worse (64 queues are
> common in newer hardware, 1000s in really new hardware) and a revisit
> to how BQL does things in this case would be useful. Ideally it would
> share state (with a cross core variable and atomic locks) as to how
> much total buffering was actually needed "down there" across all the
> queues, but without trying it, I worry that that would end up costing
> a lot of cpu cycles.
>
> Feel free to experiment with multiple transmit queues locked to other
> cores with the set-affinity bits in /proc/interrupts. I'm sure these
> MUST be useful on some platform, but I think most of the use for
> multiple hw queues is when a locally processing application  is
> getting the data, not when it is being routed.
>
> Ironically, I guess, the shorter your queues the higher likelihood a
> given packet will remain in l2 or even l1 cache.
>

I'm pinning all the queues to cores.  Although I've pinned rx/tx for the
same interface to the same cores, with cores 0-1 doing LAN and 2-3 doing
WAN duties...  I may try matching flow directions per core (rx WAN and tx
LAN on the same core).

One separate reason to set affinity on startup is that the reshuffling that
the kernel tries to do will cause things to stumble as the caches all miss.

The note about BQL is interesting...  Is that actually configurable (I
haven't gone looking, before).

OTOH, I've hit a point where trying to squeeze the most out of it just
doesn't seem necessary.  When I was bench-testing it (with local traffic
generation), I could saturate wire rates in both directions with cake
running, and limiting.  So...  Not much of a worry there.  But it's still
inconsistent on live traffic and with a real internet.  I'm not sure if
that is due to the dynamic frequency scaling, or just congestion at the
head-end, or what.

I was going to start a separate thread, but I've been contemplating what
measurements and stats I can long-term monitor to understand the
intermittent stumbles and hangs that I see.  I'm fairly certain that
they're in the "It can't be DNS....  ::sigh:: It's always DNS...."
category, though.  And if that's the case, I should just log all the
queries and look at the response times.  It seems to be marginally better
with dns-over-https (doing happy-eyeballs-like concurrent requests across
google and cloudflare), but I can't be certain.



> I
> >
> > On Tue, Jun 22, 2021 at 12:44 AM Giuseppe De Luca <dropheaders@gmx.com>
> wrote:
> >>
> >> Also a PC Engines APU4 will do the job
> >> (https://inonius.net/results/?userId=17996087f5e8 - this is a
> >> 1gbit/1gbit, with Openwrt/sqm-scripts set to 900/900.  ISP is Sony NURO
> >> in Japan). Will follow this thread to know if some interesting device
> >> popup :)
> >>
> >>
> >> https://inonius.net/results/?userId=17996087f5e8
> >>
> >> On 6/22/2021 6:12 AM, Sebastian Moeller wrote:
> >> >
> >> > On 22 June 2021 06:00:48 CEST, Stephen Hemminger <
> stephen@networkplumber.org> wrote:
> >> >> Is there any consumer hardware that can actually keep up and do AQM
> at
> >> >> 1Gbit.
> >> >          Over in the OpenWrt forums the same question pops up
> routinely once per week. The best answer ATM seems to be a combination of a
> raspberry pi4B with a decent USB3 gigabit ethernet dongle, a managed switch
> and any capable (OpenWrt) AP of the user's liking. With 4 arm A72 cores the
> will traffic shape up to a gigabit as reported by multiple users.
> >> >
> >> >
> >> >> It seems everyone seems obsessed with gamer Wifi 6. But can only do
> >> >> 300Mbit single
> >> >> stream with any kind of QoS.
> >> > IIUC most commercial home routers/APs bet on offload engines to do
> most of the heavy lifting, but as far as I understand only the NSS cores
> have a shaper and fq_codel module....
> >> >
> >> >
> >> >> It doesn't help that all the local ISP's claim 10Mbit upload even
> with
> >> >> 1G download.
> >> >> Is this a head end provisioning problem or related to Docsis 3.0 (or
> >> >> later) modems?
> >> > For DOCSIS the issue seems to be an unfortunate frequency split
> between up and downstream and use of lower efficiency coding schemes .
> >> > Over here the incumbent cable isp provisions  fifty Mbps for upstream
> and plans to increase that to hundred once the upstream is switched to
> docsis 3.1.
> >> > I believe one issue is that since most of the upstream is required
> for the reverse ACK traffic for the download and hence it can not be
> oversubscribed too much.... but I think we have real docsis experts on the
> list, so I will stop my speculation here...
> >> >
> >> > Regards
> >> >           Sebastian
> >> >
> >> >
> >> >
> >> >
> >> >> _______________________________________________
> >> >> Bloat mailing list
> >> >> Bloat@lists.bufferbloat.net
> >> >> https://lists.bufferbloat.net/listinfo/bloat
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
>
>
> --
> Latest Podcast:
> https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/
>
> Dave Täht CTO, TekLibre, LLC
>

[-- Attachment #2: Type: text/html, Size: 10404 bytes --]

Re: [Bloat] [Cake] Really getting 1G out of ISP?

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 7074 bytes --]


As a data point, I run Cake on a "Intel(R) Celeron(R) CPU  N2930  @ 1.83GHz" with 2 cores, and 1 GB/sec cable modem network. My "router board" has two GigE ports, doesn't have WiFi. It uses Fedora 34 Server as its basis, runs dnsmasq for the main LAN serving DNS, DHCP, and running a Hurricane Electric /56 tunnel for v6.
 
Doing testing with RRUL or various high-end web speed tests, I get full 1 GHz (usually >950 Mb/s throughput) download performance through it, and minimal bufferbloat (A+ on the speed tests that measure bufferbloat).. I also get full upload speed with no bufferbloat. 
 
This, I believe, is a much slower board, with fewer cores, than the Odyssey. It never comes close to saturating one of the cores.
 
I long ago gave up on trying to reflash consumer WiFi routers to serve as home gateway. (and now that cpus and memory are incredibly cheap, the proper architecture is not to bundle two unrelated functions into a single processor anyway, just have two boxes for the two functions)
 
I do use them inside my premises as APs. Life is too short. As APs, they are limited by the damn WiFi chipsets and drivers, with their poor packet scheduling, which is not solved by Cake. That's a WiFi layer problem of queuing and scheduling in the MAC layer, and I think the WiFi chip vendors have been clueless for at least a decade, and show no sign of getting a clue, sad to say. They live in proprietary land, and really have no interest in fixing the MAC layer as long as they can claim extreme throughput in an artificial scenario between two points with no cross traffic.
 
 
On Tuesday, July 6, 2021 10:26pm, "Dave Taht" <dave.taht@gmail.com> said:



> On Tue, Jul 6, 2021 at 3:32 PM Aaron Wood <woody77@gmail.com> wrote:
> >
> > I'm running an Odyssey from Seeed Studios (celeron J4125 with dual i211), and
> it can handle Cake at 1Gbps on a single core (which it needs to, because OpenWRT's
> i211 support still has multiple receive queues disabled).
> 
> Not clear if that is shaped or not? Line rate is easy on processors of
> that class or better, but shaped?
> 
> some points:
> 
> On inbound shaping especially it it still best to lock network traffic
> to a single core in low end platforms.
> 
> Cake itself is not multicore, although the design essentially is. We
> did some work towards trying to make it shape across multiple cores
> and multiple hardware queues. IF the locking contention could be
> minimized (RCU) I felt it possible for a win here, but a bigger win
> would be to eliminate "mirred" from the ingress path entirely.
> 
> Even multiple transmit queues remains kind of dicy in linux, and
> actually tend to slow network processing in most cases I've tried at
> gbit line rates. They also add latency, as (1) BQL is MIAD, not AIMD,
> so it stays "stuck" at a "good" level for a long time, AND 2) each hw
> queue gets an additive fifo at this layer, so where, you might need
> only 40k to keep a single hw queue busy, you end up with 160k with 4
> hw queues. This problem is getting worse and worse (64 queues are
> common in newer hardware, 1000s in really new hardware) and a revisit
> to how BQL does things in this case would be useful. Ideally it would
> share state (with a cross core variable and atomic locks) as to how
> much total buffering was actually needed "down there" across all the
> queues, but without trying it, I worry that that would end up costing
> a lot of cpu cycles.
> 
> Feel free to experiment with multiple transmit queues locked to other
> cores with the set-affinity bits in /proc/interrupts. I'm sure these
> MUST be useful on some platform, but I think most of the use for
> multiple hw queues is when a locally processing application is
> getting the data, not when it is being routed.
> 
> Ironically, I guess, the shorter your queues the higher likelihood a
> given packet will remain in l2 or even l1 cache.
> 
> I
> >
> > On Tue, Jun 22, 2021 at 12:44 AM Giuseppe De Luca <dropheaders@gmx.com>
> wrote:
> >>
> >> Also a PC Engines APU4 will do the job
> >> (https://inonius.net/results/?userId=17996087f5e8 - this is a
> >> 1gbit/1gbit, with Openwrt/sqm-scripts set to 900/900. ISP is Sony NURO
> >> in Japan). Will follow this thread to know if some interesting device
> >> popup :)
> >>
> >>
> >> https://inonius.net/results/?userId=17996087f5e8
> >>
> >> On 6/22/2021 6:12 AM, Sebastian Moeller wrote:
> >> >
> >> > On 22 June 2021 06:00:48 CEST, Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >> >> Is there any consumer hardware that can actually keep up and do
> AQM at
> >> >> 1Gbit.
> >> > Over in the OpenWrt forums the same question pops up
> routinely once per week. The best answer ATM seems to be a combination of a
> raspberry pi4B with a decent USB3 gigabit ethernet dongle, a managed switch and
> any capable (OpenWrt) AP of the user's liking. With 4 arm A72 cores the will
> traffic shape up to a gigabit as reported by multiple users.
> >> >
> >> >
> >> >> It seems everyone seems obsessed with gamer Wifi 6. But can only
> do
> >> >> 300Mbit single
> >> >> stream with any kind of QoS.
> >> > IIUC most commercial home routers/APs bet on offload engines to do
> most of the heavy lifting, but as far as I understand only the NSS cores have a
> shaper and fq_codel module....
> >> >
> >> >
> >> >> It doesn't help that all the local ISP's claim 10Mbit upload
> even with
> >> >> 1G download.
> >> >> Is this a head end provisioning problem or related to Docsis 3.0
> (or
> >> >> later) modems?
> >> > For DOCSIS the issue seems to be an unfortunate frequency split
> between up and downstream and use of lower efficiency coding schemes .
> >> > Over here the incumbent cable isp provisions fifty Mbps for
> upstream and plans to increase that to hundred once the upstream is switched to
> docsis 3.1.
> >> > I believe one issue is that since most of the upstream is required
> for the reverse ACK traffic for the download and hence it can not be
> oversubscribed too much.... but I think we have real docsis experts on the list,
> so I will stop my speculation here...
> >> >
> >> > Regards
> >> > Sebastian
> >> >
> >> >
> >> >
> >> >
> >> >> _______________________________________________
> >> >> Bloat mailing list
> >> >> Bloat@lists.bufferbloat.net
> >> >> https://lists.bufferbloat.net/listinfo/bloat
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
> 
> 
> 
> --
> Latest Podcast:
> https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/
> 
> Dave Täht CTO, TekLibre, LLC
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 

[-- Attachment #2: Type: text/html, Size: 9827 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Livingood, Jason]

> For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes.

Performance really takes a big step forward once a person has a D3.1 modem in their home, bringing OFDM and OFDMA as key advancements. Also in flux at the moment is where the upstream split is in cable networks, which are moving to mid-split or high-split designs that bring more upstream bandwidth. As well, over the past 18+ months, most cable networks have added substantially more upstream channels as well as performed quite a number of fiber node splits. And that is just below the physical layer stuff - there's also a lot of work at the software layer for modems and CMTSes that is quite interesting.

JL

Re: [Bloat] Really getting 1G out of ISP?

[Stephen Hemminger]

On Tue, 22 Jun 2021 23:04:18 +0000
"Livingood, Jason" <Jason_Livingood@comcast.com> wrote:

> > For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes.  
> 
> Performance really takes a big step forward once a person has a D3.1 modem in their home, bringing OFDM and OFDMA as key advancements. Also in flux at the moment is where the upstream split is in cable networks, which are moving to mid-split or high-split designs that bring more upstream bandwidth. As well, over the past 18+ months, most cable networks have added substantially more upstream channels as well as performed quite a number of fiber node splits. And that is just below the physical layer stuff - there's also a lot of work at the software layer for modems and CMTSes that is quite interesting.
> 
> JL
> 

Also discovered that the cable modem I am using is one of the ones with the
broken Intel Puma chipset (even a lawsuit about that). Will be changing it today.

Re: [Bloat] Really getting 1G out of ISP?

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

Are these in-flux changes to where the upstream split is why some modems
report DOCSIS 3.1 downstream, but only 3.0 upstream?  (and therefore aren't
enabling AQM on the upstream?)

-Aaron

On Tue, Jun 22, 2021 at 4:04 PM Livingood, Jason via Bloat <
bloat@lists.bufferbloat.net> wrote:

> > For DOCSIS the issue seems to be an unfortunate frequency split between
> up and downstream and use of lower efficiency coding schemes.
>
> Performance really takes a big step forward once a person has a D3.1 modem
> in their home, bringing OFDM and OFDMA as key advancements. Also in flux at
> the moment is where the upstream split is in cable networks, which are
> moving to mid-split or high-split designs that bring more upstream
> bandwidth. As well, over the past 18+ months, most cable networks have
> added substantially more upstream channels as well as performed quite a
> number of fiber node splits. And that is just below the physical layer
> stuff - there's also a lot of work at the software layer for modems and
> CMTSes that is quite interesting.
>
> JL
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 1750 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Wheelock, Ian]

[-- Attachment #1: Type: text/plain, Size: 2801 bytes --]

The support for AQM (PIE) is mandatory in D3.1 modems regardless of the US mode in use (ie sc-qam (3.0) or ofdma (3.1) upstream), so it should be enabled.

Get Outlook for Android<https://aka.ms/AAb9ysg>

________________________________
From: Bloat <bloat-bounces@lists.bufferbloat.net> on behalf of Aaron Wood <woody77@gmail.com>
Sent: Tuesday, July 6, 2021 11:13:59 PM
To: Livingood, Jason <Jason_Livingood@comcast.com>
Cc: bloat@lists.bufferbloat.net <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] Really getting 1G out of ISP?

Are these in-flux changes to where the upstream split is why some modems report DOCSIS 3.1 downstream, but only 3.0 upstream? (and therefore aren't enabling AQM on the upstream?) -Aaron  ‌  ‌  ‌  ‌  ‌  ‌  ‌
External (woody77@gmail.com<mailto:woody77@gmail.com>)
  Report This Email<https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2EyZmFiZGMyYTNjNjZmYjMwOTExZTgxMjQ1MzY3ZmRlLzE2MjU2MDk2NTcuMQ==#key=641fac114ebb55b107a804bc90c84eb7>  FAQ<https://www.inky.com/banner-faq/>  Protection by INKY<https://www.inky.com>

Are these in-flux changes to where the upstream split is why some modems report DOCSIS 3.1 downstream, but only 3.0 upstream?  (and therefore aren't enabling AQM on the upstream?)

-Aaron

On Tue, Jun 22, 2021 at 4:04 PM Livingood, Jason via Bloat <bloat@lists.bufferbloat.net<mailto:bloat@lists.bufferbloat.net>> wrote:
> For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes.

Performance really takes a big step forward once a person has a D3.1 modem in their home, bringing OFDM and OFDMA as key advancements. Also in flux at the moment is where the upstream split is in cable networks, which are moving to mid-split or high-split designs that bring more upstream bandwidth. As well, over the past 18+ months, most cable networks have added substantially more upstream channels as well as performed quite a number of fiber node splits. And that is just below the physical layer stuff - there's also a lot of work at the software layer for modems and CMTSes that is quite interesting.

JL

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<mailto:Bloat@lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/bloat<https://secure-web.cisco.com/12r7ZmOFM8ddVde0m7xhKWGeITD3XE3Ac8lYCXdEv60KV7M2dP0FljB3D2Drve2H_DrbajVB3m_YObAyONExbkEVq1hasawuiwj6gKBkdsagSp44rWmPbd3AYBB3T_gqXzw5lJilH-KCga_T694ymSPplZtkdJyEPAxcl4cd1elFxGzE2cKv0yWsHC3rCkJRNvkbEzEF6BPvlI8csBxJLA8k4wOTd-SpoSvQodsyQp6X8hBDSoo-OtpI1sVvNo84-c585IlURf4pCCiQfyHZvkBMmb_Ee0STbfr_Q3Jqc0qHqn6wOImpTw5-rG0JZktn8QFfPuZHvwR2eSVSXMyNBSg/https%3A%2F%2Flists.bufferbloat.net%2Flistinfo%2Fbloat>

[-- Attachment #2: Type: text/html, Size: 4061 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 2538 bytes --]

I'm sure it's present, as a capability.  I just think it's not being used
(based on what I see for upstream bloat, which is >100ms without running
cake in the router).

On Tue, Jul 6, 2021 at 3:28 PM Wheelock, Ian <ian.wheelock@commscope.com>
wrote:

> The support for AQM (PIE) is mandatory in D3.1 modems regardless of the US
> mode in use (ie sc-qam (3.0) or ofdma (3.1) upstream), so it should be
> enabled.
>
> Get Outlook for Android <https://aka.ms/AAb9ysg>
>
> ------------------------------
> *From:* Bloat <bloat-bounces@lists.bufferbloat.net> on behalf of Aaron
> Wood <woody77@gmail.com>
> *Sent:* Tuesday, July 6, 2021 11:13:59 PM
> *To:* Livingood, Jason <Jason_Livingood@comcast.com>
> *Cc:* bloat@lists.bufferbloat.net <bloat@lists.bufferbloat.net>
> *Subject:* Re: [Bloat] Really getting 1G out of ISP?
>
> Are these in-flux changes to where the upstream split is why some modems
> report DOCSIS 3.1 downstream, but only 3.0 upstream?  (and therefore aren't
> enabling AQM on the upstream?)
>
> -Aaron
>
> On Tue, Jun 22, 2021 at 4:04 PM Livingood, Jason via Bloat <
> bloat@lists.bufferbloat.net> wrote:
>
> > For DOCSIS the issue seems to be an unfortunate frequency split between
> up and downstream and use of lower efficiency coding schemes.
>
> Performance really takes a big step forward once a person has a D3.1 modem
> in their home, bringing OFDM and OFDMA as key advancements. Also in flux at
> the moment is where the upstream split is in cable networks, which are
> moving to mid-split or high-split designs that bring more upstream
> bandwidth. As well, over the past 18+ months, most cable networks have
> added substantially more upstream channels as well as performed quite a
> number of fiber node splits. And that is just below the physical layer
> stuff - there's also a lot of work at the software layer for modems and
> CMTSes that is quite interesting.
>
> JL
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> <https://secure-web.cisco.com/12r7ZmOFM8ddVde0m7xhKWGeITD3XE3Ac8lYCXdEv60KV7M2dP0FljB3D2Drve2H_DrbajVB3m_YObAyONExbkEVq1hasawuiwj6gKBkdsagSp44rWmPbd3AYBB3T_gqXzw5lJilH-KCga_T694ymSPplZtkdJyEPAxcl4cd1elFxGzE2cKv0yWsHC3rCkJRNvkbEzEF6BPvlI8csBxJLA8k4wOTd-SpoSvQodsyQp6X8hBDSoo-OtpI1sVvNo84-c585IlURf4pCCiQfyHZvkBMmb_Ee0STbfr_Q3Jqc0qHqn6wOImpTw5-rG0JZktn8QFfPuZHvwR2eSVSXMyNBSg/https%3A%2F%2Flists.bufferbloat.net%2Flistinfo%2Fbloat>
>
>

[-- Attachment #2: Type: text/html, Size: 4444 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Dave Taht]

On Tue, Jul 6, 2021 at 3:28 PM Wheelock, Ian <ian.wheelock@commscope.com> wrote:
>
> The support for AQM (PIE) is mandatory in D3.1 modems regardless of the US mode in use (ie sc-qam (3.0) or ofdma (3.1) upstream), so it should be enabled.

I am under the distinct impression it is only on for ISP provided
modems for at least one ISP.

> Get Outlook for Android
>
> ________________________________
> From: Bloat <bloat-bounces@lists.bufferbloat.net> on behalf of Aaron Wood <woody77@gmail.com>
> Sent: Tuesday, July 6, 2021 11:13:59 PM
> To: Livingood, Jason <Jason_Livingood@comcast.com>
> Cc: bloat@lists.bufferbloat.net <bloat@lists.bufferbloat.net>
> Subject: Re: [Bloat] Really getting 1G out of ISP?
>
> Are these in-flux changes to where the upstream split is why some modems report DOCSIS 3.1 downstream, but only 3.0 upstream?  (and therefore aren't enabling AQM on the upstream?)
>
> -Aaron
>
> On Tue, Jun 22, 2021 at 4:04 PM Livingood, Jason via Bloat <bloat@lists.bufferbloat.net> wrote:
>
> > For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes.
>
> Performance really takes a big step forward once a person has a D3.1 modem in their home, bringing OFDM and OFDMA as key advancements. Also in flux at the moment is where the upstream split is in cable networks, which are moving to mid-split or high-split designs that bring more upstream bandwidth. As well, over the past 18+ months, most cable networks have added substantially more upstream channels as well as performed quite a number of fiber node splits. And that is just below the physical layer stuff - there's also a lot of work at the software layer for modems and CMTSes that is quite interesting.
>
> JL
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Latest Podcast:
https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/

Dave Täht CTO, TekLibre, LLC

Re: [Bloat] Really getting 1G out of ISP?

[Wheelock, Ian]

[-- Attachment #1: Type: text/plain, Size: 5258 bytes --]


In terms of the standard itself the DOCSIS MULPI spec mandates that AQM must always be enabled on each Best Effort US service flow configured on the CM, along with a default target latency of 10ms.

AQM can be explicitly disabled, using the DISABLE AQM TLV configured on each US service flow configured in the CM Configuration file supplied during registration of the CM to the CMTS. Similarly, explicit configuration using the TARGET LATENCY TLV is required to modify the default 10ms.

It is entirely possible through the mechanics of DOCSIS provisioning that AQM could be enabled or disabled on different CMs or groups of CMs. Doing so would be rather petty and may add additional unnecessary complexity to the provisioning system. Users that own their CMs are still paying for the internet access with the specific ISP, so would likely expect equivalent performance.

User owned CMs connecting to ISPs are generally approved/accepted by that ISP, and a minimum expectation is that they are DOCSIS 3.1 certified (meaning 100% support for US AQM). US AQM processing itself is purely confined within the CM itself and how it deals with each US service flow – it has not material impact on the CMTS, so a view that a non-ISP supplied CM running AQM might somehow negatively impact the CMTS would be without any foundation.

- Ian



From: Dave Taht <dave.taht@gmail.com>
Date: Wednesday 7 July 2021 at 03:11
To: "Wheelock, Ian" <ian.wheelock@commscope.com>
Cc: "Livingood, Jason" <Jason_Livingood@comcast.com>, Aaron Wood <woody77@gmail.com>, "bloat@lists.bufferbloat.net" <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] Really getting 1G out of ISP?

On Tue, Jul 6, 2021 at 3:28 PM Wheelock, Ian wrote: > > The support for AQM (PIE) is mandatory in D3.1 modems regardless of the US mode in use (ie sc-qam (3.0) or ofdma (3
External (dave.taht@gmail.com<mailto:dave.taht@gmail.com>)
  Report This Email<https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2RlMzcwMWRkOGJhZjExZTI5ZWFlMzIzMDZmMzU5YzMwLzE2MjU2MjM4NjAuOA==#key=70c18fa2937a6724bb0094f5a2440dd7>  FAQ<https://www.inky.com/banner-faq/>  Protection by INKY<https://www.inky.com>


On Tue, Jul 6, 2021 at 3:28 PM Wheelock, Ian <ian.wheelock@commscope.com> wrote:

>

> The support for AQM (PIE) is mandatory in D3.1 modems regardless of the US mode in use (ie sc-qam (3.0) or ofdma (3.1) upstream), so it should be enabled.



I am under the distinct impression it is only on for ISP provided

modems for at least one ISP.



> Get Outlook for Android

>

> ________________________________

> From: Bloat <bloat-bounces@lists.bufferbloat.net> on behalf of Aaron Wood <woody77@gmail.com>

> Sent: Tuesday, July 6, 2021 11:13:59 PM

> To: Livingood, Jason <Jason_Livingood@comcast.com>

> Cc: bloat@lists.bufferbloat.net <bloat@lists.bufferbloat.net>

> Subject: Re: [Bloat] Really getting 1G out of ISP?

>

> Are these in-flux changes to where the upstream split is why some modems report DOCSIS 3.1 downstream, but only 3.0 upstream?  (and therefore aren't enabling AQM on the upstream?)

>

> -Aaron

>

> On Tue, Jun 22, 2021 at 4:04 PM Livingood, Jason via Bloat <bloat@lists.bufferbloat.net> wrote:

>

> > For DOCSIS the issue seems to be an unfortunate frequency split between up and downstream and use of lower efficiency coding schemes.

>

> Performance really takes a big step forward once a person has a D3.1 modem in their home, bringing OFDM and OFDMA as key advancements. Also in flux at the moment is where the upstream split is in cable networks, which are moving to mid-split or high-split designs that bring more upstream bandwidth. As well, over the past 18+ months, most cable networks have added substantially more upstream channels as well as performed quite a number of fiber node splits. And that is just below the physical layer stuff - there's also a lot of work at the software layer for modems and CMTSes that is quite interesting.

>

> JL

>

> _______________________________________________

> Bloat mailing list

> Bloat@lists.bufferbloat.net

> https://secure-web.cisco.com/1ONp6C2XmyCfN3ti15R4dj_0N3utHIeQ_etf8rF_YuPX9PY8HyOvcU-M-mKgEiJgb1K_B2h3x6WPCTUCVOrrS4YgUhza41owjh6HB8vTcAKMqNbcVtT6ym7vfN6GtSDKzHepnxefd8HEx0WXLTyAEVveRpFgQDNcL_rSZe1oTsqz3IH8X7dVj53lHVh3osO09T7ot_gEkGzPg-OVa28-RbNJxu-prknTw5jkzcU2Gg2c5Fr3-2v_wATMol4pgnePUhTHAcba998rdwi6xj2Nf1Nn0bkMqja4F58Q30KhsDfhjapErVB7gMbj8ekpAxhy0/https%3A%2F%2Flists.bufferbloat.net%2Flistinfo%2Fbloat

>

> _______________________________________________

> Bloat mailing list

> Bloat@lists.bufferbloat.net

> https://secure-web.cisco.com/1ONp6C2XmyCfN3ti15R4dj_0N3utHIeQ_etf8rF_YuPX9PY8HyOvcU-M-mKgEiJgb1K_B2h3x6WPCTUCVOrrS4YgUhza41owjh6HB8vTcAKMqNbcVtT6ym7vfN6GtSDKzHepnxefd8HEx0WXLTyAEVveRpFgQDNcL_rSZe1oTsqz3IH8X7dVj53lHVh3osO09T7ot_gEkGzPg-OVa28-RbNJxu-prknTw5jkzcU2Gg2c5Fr3-2v_wATMol4pgnePUhTHAcba998rdwi6xj2Nf1Nn0bkMqja4F58Q30KhsDfhjapErVB7gMbj8ekpAxhy0/https%3A%2F%2Flists.bufferbloat.net%2Flistinfo%2Fbloat







--

Latest Podcast:

https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/



Dave Täht CTO, TekLibre, LLC



[-- Attachment #2: Type: text/html, Size: 10797 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Jonathan Morton]

> On 7 Jul, 2021, at 12:27 pm, Wheelock, Ian <ian.wheelock@commscope.com> wrote:
> 
> It is entirely possible through the mechanics of DOCSIS provisioning that AQM could be enabled or disabled on different CMs or groups of CMs. Doing so would be rather petty and may add additional unnecessary complexity to the provisioning system. Users that own their CMs are still paying for the internet access with the specific ISP, so would likely expect equivalent performance.

Entirely true, but for the ISP the matter of whether the subscriber is using a rented or self-owned modem is not entirely petty - it is the difference of a line item on the monthly bill.  I'm sure you can see how the perverse incentives arise with that.

 - Jonathan Morton

Re: [Bloat] Really getting 1G out of ISP?

[Wheelock, Ian]

[-- Attachment #1: Type: text/plain, Size: 2117 bytes --]

Just to pull this back into context – it may well be that one ISP happens to muck around like that – however, rented/leased CMs still make up the majority of DOCSIS devices, so it is very likely that DOCSIS AQM is enabled and operating on the majority of connected DOCSIS 3.1 devices.


From: Jonathan Morton <chromatix99@gmail.com>
Date: Wednesday 7 July 2021 at 17:18
To: "Wheelock, Ian" <ian.wheelock@commscope.com>
Cc: Dave Taht <dave.taht@gmail.com>, "bloat@lists.bufferbloat.net" <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] Really getting 1G out of ISP?

> On 7 Jul, 2021, at 12:27 pm, Wheelock, Ian wrote: > > It is entirely possible through the mechanics of DOCSIS provisioning that AQM could be enabled or disabled on diffe
Caution: External (chromatix99@gmail.com<mailto:chromatix99@gmail.com>)
Potential Sender Forgery   Details<https://shared.outlook.inky.com/details?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2I4NTExOGM0OGM5ZDc2Mjk1MGQ0YTYxNThhNzI5NmU3LzE2MjU2NzQ2ODMuNTI=#key=f830c29ef63b1e22dd732e346fb610f8>
  Report This Email<https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tL2I4NTExOGM0OGM5ZDc2Mjk1MGQ0YTYxNThhNzI5NmU3LzE2MjU2NzQ2ODMuNTI=#key=f830c29ef63b1e22dd732e346fb610f8>  FAQ<https://www.inky.com/banner-faq/>  Protection by INKY<https://www.inky.com>


> On 7 Jul, 2021, at 12:27 pm, Wheelock, Ian <ian.wheelock@commscope.com> wrote:

>

> It is entirely possible through the mechanics of DOCSIS provisioning that AQM could be enabled or disabled on different CMs or groups of CMs. Doing so would be rather petty and may add additional unnecessary complexity to the provisioning system. Users that own their CMs are still paying for the internet access with the specific ISP, so would likely expect equivalent performance.



Entirely true, but for the ISP the matter of whether the subscriber is using a rented or self-owned modem is not entirely petty - it is the difference of a line item on the monthly bill.  I'm sure you can see how the perverse incentives arise with that.



- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 3799 bytes --]

Re: [Bloat] Really getting 1G out of ISP?

[Livingood, Jason]

> It doesn't help that all the local ISP's claim 10Mbit upload even with 1G download. Is this a head end provisioning problem or related to Docsis 3.0 (or later) modems?

I'll cover this in an upcoming technical paper (mid-July I hope). Depending on the DOCSIS version, CMTS, and cable modems involved you may have no AQM, or buffer controls for the cable modem, or AQM (sort of) on the CMTS and in the cable modem. In the Comcast network you should find AQM in the upstream queue on the cable modems for which we have deployed RDK-B software (XB6 and XB7), while other devices would have buffer controls.

JL

Re: [Bloat] Really getting 1G out of ISP?

[Stephen Hemminger]

On Tue, 22 Jun 2021 22:51:18 +0000
"Livingood, Jason" <Jason_Livingood@comcast.com> wrote:

> > It doesn't help that all the local ISP's claim 10Mbit upload even with 1G download. Is this a head end provisioning problem or related to Docsis 3.0 (or later) modems?  
> 
> I'll cover this in an upcoming technical paper (mid-July I hope). Depending on the DOCSIS version, CMTS, and cable modems involved you may have no AQM, or buffer controls for the cable modem, or AQM (sort of) on the CMTS and in the cable modem. In the Comcast network you should find AQM in the upstream queue on the cable modems for which we have deployed RDK-B software (XB6 and XB7), while other devices would have buffer controls.
> 
> JL
> 

Just a short update. The cable modem matters. Updated from Docsis 3.0 modem with bad Intel Puma chipset
to new model with Docsis 3.1 and Broadcom and things are much more stable.

As far as AQM, in this setup; fq_codel does much better than the Cake configuration. With fq_codel can
see 700Mbit download speed. It looks like Cake is using more CPU especialy since the Cake configuration
is using an ifb ingress queue discipline as well.

Re: [Bloat] Really getting 1G out of ISP?

[Sebastian Moeller]

Hi Stephen,


> On Jun 29, 2021, at 21:48, Stephen Hemminger <stephen@networkplumber.org> wrote:
> 
> On Tue, 22 Jun 2021 22:51:18 +0000
> "Livingood, Jason" <Jason_Livingood@comcast.com> wrote:
> 
>>> It doesn't help that all the local ISP's claim 10Mbit upload even with 1G download. Is this a head end provisioning problem or related to Docsis 3.0 (or later) modems?  
>> 
>> I'll cover this in an upcoming technical paper (mid-July I hope). Depending on the DOCSIS version, CMTS, and cable modems involved you may have no AQM, or buffer controls for the cable modem, or AQM (sort of) on the CMTS and in the cable modem. In the Comcast network you should find AQM in the upstream queue on the cable modems for which we have deployed RDK-B software (XB6 and XB7), while other devices would have buffer controls.
>> 
>> JL
>> 
> 
> Just a short update. The cable modem matters. Updated from Docsis 3.0 modem with bad Intel Puma chipset
> to new model with Docsis 3.1 and Broadcom and things are much more stable.

	Glad that helped, rather sad state of affairs that these devices are still in the field. (I am not fan of forced obsolescence or retiring hardware too early, but these devices are simply barely fit for their purpose).

> 
> As far as AQM, in this setup; fq_codel does much better than the Cake configuration. With fq_codel can
> see 700Mbit download speed. It looks like Cake is using more CPU

	Yepp, it turns out that all the additional features cake brings to the table have some computational cost. At some earlier state in development cake was leaner and meaner than HTB+fq_codel, but that did not seem to hold.


> especialy since the Cake configuration
> is using an ifb ingress queue discipline as well.

	Both cake and HTB+fq_codel require tricks for download shaping, either a veth pair or an IFB (both seem to have a similar cost but IFB is much simpler to set up) or for a wired only shaper, one can instantiate the internet-download shaper as egress shaper on the interface towards the LAN. Last I tested on an ancient single core 750 MHz MIPS, avoiding the IFB got a 5% higher shaper limit, so not nothing, but also not a way to have your router punch one weight class higher?

Best Regards
	Sebastian


> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat