* [Ecn-sane] osx ecn
@ 2019-08-27 2:03 Dave Taht
2019-08-27 6:47 ` Mikael Abrahamsson
2019-08-27 6:58 ` Sebastian Moeller
0 siblings, 2 replies; 18+ messages in thread
From: Dave Taht @ 2019-08-27 2:03 UTC (permalink / raw)
To: ECN-Sane
netstat -sp tcp has all kinds of useful stats
7174 client connections attempted to negotiate ECN
4407 client connections successfully negotiated ECN
2192 times graceful fallback to Non-ECN connection
2053 times lost ECN negotiating SYN, followed by retransmission
??
0 server connection attempted to negotiate ECN
0 server connection successfully negotiated ECN
0 time lost ECN negotiating SYN-ACK, followed by retransmission
90 times received congestion experienced (CE) notification
0 time CWR was sent in response to ECE
3298 times sent ECE notification
25 connections received CE atleast once
0 connection received ECE atleast once
830 connections using ECN have seen packet loss but no CE
12 connections using ECN have seen packet loss and CE
13 connections using ECN received CE but no packet loss
9 connections fell back to non-ECN due to SYN-loss
135 connections fell back to non-ECN due to reordering
0 connection fell back to non-ECN due to excessive CE-markings
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 2:03 [Ecn-sane] osx ecn Dave Taht
@ 2019-08-27 6:47 ` Mikael Abrahamsson
2019-08-27 6:54 ` Michael Welzl
2019-08-27 6:58 ` Sebastian Moeller
1 sibling, 1 reply; 18+ messages in thread
From: Mikael Abrahamsson @ 2019-08-27 6:47 UTC (permalink / raw)
To: Dave Taht; +Cc: ECN-Sane
On Mon, 26 Aug 2019, Dave Taht wrote:
> netstat -sp tcp has all kinds of useful stats
I am using the settings from
https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ but I still
get 0 on attempted ECN connections?
https://mailarchive.ietf.org/arch/msg/iccrg/ghBJK_x9UqD6a5RzqWY-FmpPzBc
I obviously had it working before. I'm on 10.14.6, what are you on?
Btw, I checked what happened if I removed the sysctl settings and then it
defaults to "2" instead of "1".
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 6:47 ` Mikael Abrahamsson
@ 2019-08-27 6:54 ` Michael Welzl
2019-08-27 7:02 ` Mikael Abrahamsson
2019-08-27 18:36 ` Dave Taht
0 siblings, 2 replies; 18+ messages in thread
From: Michael Welzl @ 2019-08-27 6:54 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: Dave Taht, ECN-Sane
run it as root - strange, yes, but then the output is very different.
cheers,
michael
Sent from my iPhone
> On 27 Aug 2019, at 08:47, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
>
>> On Mon, 26 Aug 2019, Dave Taht wrote:
>>
>> netstat -sp tcp has all kinds of useful stats
>
> I am using the settings from https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ but I still get 0 on attempted ECN connections?
>
> https://mailarchive.ietf.org/arch/msg/iccrg/ghBJK_x9UqD6a5RzqWY-FmpPzBc
>
> I obviously had it working before. I'm on 10.14.6, what are you on?
>
> Btw, I checked what happened if I removed the sysctl settings and then it defaults to "2" instead of "1".
>
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
> _______________________________________________
> Ecn-sane mailing list
> Ecn-sane@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/ecn-sane
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 2:03 [Ecn-sane] osx ecn Dave Taht
2019-08-27 6:47 ` Mikael Abrahamsson
@ 2019-08-27 6:58 ` Sebastian Moeller
2019-08-27 7:05 ` Jonathan Morton
1 sibling, 1 reply; 18+ messages in thread
From: Sebastian Moeller @ 2019-08-27 6:58 UTC (permalink / raw)
To: Dave Täht; +Cc: ECN-Sane
Hi Dave,
> On Aug 27, 2019, at 04:03, Dave Taht <dave.taht@gmail.com> wrote:
>
> 0 connection fell back to non-ECN due to excessive CE-markings
I wonder whether macosx would be able to gracefully survive a misclassification into a L4S/NQB queue at a dual queue AQM? What is the threshold here?
Best Regards
Sebastian
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 6:54 ` Michael Welzl
@ 2019-08-27 7:02 ` Mikael Abrahamsson
2019-08-27 18:20 ` Dave Taht
2019-08-27 18:36 ` Dave Taht
1 sibling, 1 reply; 18+ messages in thread
From: Mikael Abrahamsson @ 2019-08-27 7:02 UTC (permalink / raw)
To: Michael Welzl; +Cc: Dave Taht, ECN-Sane
On Tue, 27 Aug 2019, Michael Welzl wrote:
> run it as root - strange, yes, but then the output is very different.
Thanks, indeed.
$ sudo netstat -sp tcp | egrep 'CE|EC'
11097 client connections attempted to negotiate ECN
4864 client connections successfully negotiated ECN
3800 times graceful fallback to Non-ECN connection
467 times lost ECN negotiating SYN, followed by retransmission
52 server connections attempted to negotiate ECN
52 server connections successfully negotiated ECN
0 time lost ECN negotiating SYN-ACK, followed by retransmission
427 times received congestion experienced (CE) notification
0 time CWR was sent in response to ECE
6111 times sent ECE notification
15 connections received CE atleast once
0 connection received ECE atleast once
267 connections using ECN have seen packet loss but no CE
1 connection using ECN have seen packet loss and CE
14 connections using ECN received CE but no packet loss
14 connections fell back to non-ECN due to SYN-loss
1 connection fell back to non-ECN due to reordering
0 connection fell back to non-ECN due to excessive CE-markings
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 6:58 ` Sebastian Moeller
@ 2019-08-27 7:05 ` Jonathan Morton
2019-08-27 7:52 ` Sebastian Moeller
0 siblings, 1 reply; 18+ messages in thread
From: Jonathan Morton @ 2019-08-27 7:05 UTC (permalink / raw)
To: Sebastian Moeller; +Cc: Dave Täht, ECN-Sane
> On 27 Aug, 2019, at 9:58 am, Sebastian Moeller <moeller0@gmx.de> wrote:
>
>> 0 connection fell back to non-ECN due to excessive CE-markings
>
> I wonder whether macosx would be able to gracefully survive a misclassification into a L4S/NQB queue at a dual queue AQM? What is the threshold here?
No, I don't think so. L4S assumes the DCTCP response function which stabilises at 2 CE marks per RTT, which is still a fairly low percentage if the cwnd is high. I doubt that would be detected as "excessive" in this context. Also, because the RFC-3168 response to CE is much sharper, the actual marking rate will be lower, as the flow won't stay in the saturating regime where marking occurs.
- Jonathan Morton
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 7:05 ` Jonathan Morton
@ 2019-08-27 7:52 ` Sebastian Moeller
0 siblings, 0 replies; 18+ messages in thread
From: Sebastian Moeller @ 2019-08-27 7:52 UTC (permalink / raw)
To: Jonathan Morton; +Cc: Dave Täht, ECN-Sane
Hi Jonathan,
> On Aug 27, 2019, at 09:05, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 27 Aug, 2019, at 9:58 am, Sebastian Moeller <moeller0@gmx.de> wrote:
>>
>>> 0 connection fell back to non-ECN due to excessive CE-markings
>>
>> I wonder whether macosx would be able to gracefully survive a misclassification into a L4S/NQB queue at a dual queue AQM? What is the threshold here?
>
> No, I don't think so. L4S assumes the DCTCP response function which stabilises at 2 CE marks per RTT, which is still a fairly low percentage if the cwnd is high. I doubt that would be detected as "excessive" in this context. Also, because the RFC-3168 response to CE is much sharper, the actual marking rate will be lower, as the flow won't stay in the saturating regime where marking occurs.
>
> - Jonathan Morton
thanks for the detailed answer.
Best Regards
Sebastian
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 7:02 ` Mikael Abrahamsson
@ 2019-08-27 18:20 ` Dave Taht
0 siblings, 0 replies; 18+ messages in thread
From: Dave Taht @ 2019-08-27 18:20 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: Michael Welzl, ECN-Sane
An interesing piece of this is how often the syn attempt is failing.
2192 times graceful fallback to Non-ECN connection
2053 times lost ECN negotiating SYN, followed by retransmission
??
Now, there are plenty of reasons to lose syns (most firewalls rate limit these),
besides the ecn flag.
In my case during this boot cycle I've been engaged in a fun
experiment - and one day perhaps a paper - Going to various coffee
shops in the area and running the attached script. The results thus
far are universally dismal,
except for google starbucks which is excellent - but the packet
captures seem to indicate something in addition to
or other than than fq_codel in use (dpi maybe? aruba is the next hop)
AND it did not negotiate ecn when I last tried.
Wifi spread initially through the coffee shop phenomenon, and I've
"lept over the counter" a couple times
to help configure the to-the-isp qos system when I found an owner with
clue, which helps a lot. When I showed 'em the before/after two bought
me lunch (and I can highly recommend https://www.cemitas1.com/ in
davenport for food, music, view & wifi) if that's a motivator for more
folk here "to get out more", please do so. Table service is a nice
thing to have when in deep hack mode and socializing more helps too. :
daves-Air-6:los_gatos_starbucks d$ more testit.sh
#!/bin/sh
T="Los_Gatos_Starbucks"
F="flent -x -H flent-fremont.bufferbloat.net -t $T" # pick a closer
flent server and on linux use --socket-stats
$F --te=download_streams=4 tcp_ndown
$F --te=upload_streams=4 tcp_nup
$F --te=upload_streams=4 tcp_2up_square
$F rrul
$F rrul_be
On Tue, Aug 27, 2019 at 12:02 AM Mikael Abrahamsson <swmike@swm.pp.se> wrote:
>
> On Tue, 27 Aug 2019, Michael Welzl wrote:
>
> > run it as root - strange, yes, but then the output is very different.
>
> Thanks, indeed.
>
> $ sudo netstat -sp tcp | egrep 'CE|EC'
> 11097 client connections attempted to negotiate ECN
> 4864 client connections successfully negotiated ECN
> 3800 times graceful fallback to Non-ECN connection
> 467 times lost ECN negotiating SYN, followed by retransmission
> 52 server connections attempted to negotiate ECN
> 52 server connections successfully negotiated ECN
> 0 time lost ECN negotiating SYN-ACK, followed by retransmission
> 427 times received congestion experienced (CE) notification
> 0 time CWR was sent in response to ECE
> 6111 times sent ECE notification
> 15 connections received CE atleast once
> 0 connection received ECE atleast once
> 267 connections using ECN have seen packet loss but no CE
> 1 connection using ECN have seen packet loss and CE
> 14 connections using ECN received CE but no packet loss
> 14 connections fell back to non-ECN due to SYN-loss
> 1 connection fell back to non-ECN due to reordering
> 0 connection fell back to non-ECN due to excessive CE-markings
>
>
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 6:54 ` Michael Welzl
2019-08-27 7:02 ` Mikael Abrahamsson
@ 2019-08-27 18:36 ` Dave Taht
2019-08-27 19:00 ` Michael Welzl
1 sibling, 1 reply; 18+ messages in thread
From: Dave Taht @ 2019-08-27 18:36 UTC (permalink / raw)
To: Michael Welzl; +Cc: Mikael Abrahamsson, ECN-Sane
On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote:
>
> run it as root - strange, yes, but then the output is very different.
>
> cheers,
> michael
>
>
> Sent from my iPhone
speaking of iphones I have no idea what they do nowadays, any odds you
could do a packet
cap either on it or at a server or via wifi on a middlebox
> > On 27 Aug 2019, at 08:47, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> >
> >> On Mon, 26 Aug 2019, Dave Taht wrote:
> >>
> >> netstat -sp tcp has all kinds of useful stats
> >
> > I am using the settings from https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ but I still get 0 on attempted ECN connections?
> >
> > https://mailarchive.ietf.org/arch/msg/iccrg/ghBJK_x9UqD6a5RzqWY-FmpPzBc
> >
> > I obviously had it working before. I'm on 10.14.6, what are you on?
> >
> > Btw, I checked what happened if I removed the sysctl settings and then it defaults to "2" instead of "1".
> >
> > --
> > Mikael Abrahamsson email: swmike@swm.pp.se
> > _______________________________________________
> > Ecn-sane mailing list
> > Ecn-sane@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/ecn-sane
>
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 18:36 ` Dave Taht
@ 2019-08-27 19:00 ` Michael Welzl
2019-08-28 23:29 ` Ryan Mounce
0 siblings, 1 reply; 18+ messages in thread
From: Michael Welzl @ 2019-08-27 19:00 UTC (permalink / raw)
To: Dave Taht; +Cc: Mikael Abrahamsson, ECN-Sane
[-- Attachment #1: Type: text/plain, Size: 764 bytes --]
> On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
> On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no <mailto:michawe@ifi.uio.no>> wrote:
>>
>> run it as root - strange, yes, but then the output is very different.
>>
>> cheers,
>> michael
>>
>>
>> Sent from my iPhone
>
> speaking of iphones I have no idea what they do nowadays, any odds you
> could do a packet
> cap either on it or at a server or via wifi on a middlebox
I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on my iPhone, and as a result, I could ssh into it.
So I’m guessing: if you do this with a current one, you can probably still ssh into it and then perhaps run netstat.
Cheers,
Michael
[-- Attachment #2: Type: text/html, Size: 5669 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-27 19:00 ` Michael Welzl
@ 2019-08-28 23:29 ` Ryan Mounce
2019-08-28 23:45 ` Dave Taht
0 siblings, 1 reply; 18+ messages in thread
From: Ryan Mounce @ 2019-08-28 23:29 UTC (permalink / raw)
To: Michael Welzl; +Cc: Dave Taht, ECN-Sane
[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]
On Wed, 28 Aug 2019 at 04:30, Michael Welzl <michawe@ifi.uio.no> wrote:
>
>
> On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
> On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote:
>
>
> run it as root - strange, yes, but then the output is very different.
>
> cheers,
> michael
>
>
> Sent from my iPhone
>
>
> speaking of iphones I have no idea what they do nowadays, any odds you
> could do a packet
> cap either on it or at a server or via wifi on a middlebox
>
>
> I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on
> my iPhone, and as a result, I could ssh into it.
> So I’m guessing: if you do this with a current one, you can probably still
> ssh into it and then perhaps run netstat.
>
> Cheers,
> Michael
>
You sure can :)
https://gist.github.com/rmounce/74dd5e92e6304688219024349bfaad0e
-Ryan
> --
Regards,
Ryan Mounce
ryan@mounce.com.au
0415 799 929
Sent from mobile
[-- Attachment #2: Type: text/html, Size: 5469 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-28 23:29 ` Ryan Mounce
@ 2019-08-28 23:45 ` Dave Taht
2019-08-29 0:55 ` Dave Taht
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Dave Taht @ 2019-08-28 23:45 UTC (permalink / raw)
To: Ryan Mounce; +Cc: Michael Welzl, ECN-Sane
Very cool, thank you for jailbreaking.
I'm a little puzzled over the cwr/ece ratio?
25 times received congestion experienced (CE) notification
37 times CWR was sent in response to ECE
619 times sent ECE notification
1029 connections received CE atleast once
And um, er, while you are here busting your warranty for the sake of
science, I'm dying to know what
netstat -I each_interface_you_have_especially_lte -qq
shows. They have a really short queue and some form of flow control
and I've never seen anything trigger except though a vm
my laptop:
en0:
[ sched: FQ_CODEL qlength: 0/128 ]
[ pkts: 0 bytes: 0 dropped pkts: 1087 bytes: 177812 ]
=====================================================
[ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max: 8 ]
[ queued pkts: 0 bytes: 0 ]
[ dequeued pkts: 36767 bytes: 5237639 ]
[ budget: 0 target qdelay: 10.00 msec update
interval:100.00 msec ]
[ flow control: 0 feedback: 0 stalls: 0 failed: 0 ]
[ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ]
[ flows total: 0 new: 0 old: 0 ]
[ throttle on: 0 off: 0 drop: 0 ]
=====================================================
[ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max: 6 ]
[ queued pkts: 0 bytes: 0 ]
[ dequeued pkts: 96836 bytes: 36619772 ]
[ budget: 0 target qdelay: 10.00 msec update
interval:100.00 msec ]
[ flow control: 0 feedback: 0 stalls: 0 failed: 0 ]
[ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ]
[ flows total: 0 new: 0 old: 0 ]
[ throttle on: 0 off: 0 drop: 0 ]
=====================================================
[ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ]
[ queued pkts: 0 bytes: 0 ]
[ dequeued pkts: 5212250 bytes: 2820548356 ]
[ budget: 0 target qdelay: 10.00 msec update
interval:100.00 msec ]
[ flow control: 30 feedback: 30 stalls: 0 failed: 0 ]
[ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ]
[ flows total: 0 new: 0 old: 0 ]
[ throttle on: 0 off: 0 drop: 0 ]
On Wed, Aug 28, 2019 at 4:29 PM Ryan Mounce <ryan@mounce.com.au> wrote:
>
> On Wed, 28 Aug 2019 at 04:30, Michael Welzl <michawe@ifi.uio.no> wrote:
>>
>>
>>
>> On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote:
>>
>>
>> run it as root - strange, yes, but then the output is very different.
>>
>> cheers,
>> michael
>>
>>
>> Sent from my iPhone
>>
>>
>> speaking of iphones I have no idea what they do nowadays, any odds you
>> could do a packet
>> cap either on it or at a server or via wifi on a middlebox
>>
>>
>> I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on my iPhone, and as a result, I could ssh into it.
>> So I’m guessing: if you do this with a current one, you can probably still ssh into it and then perhaps run netstat.
>>
>> Cheers,
>> Michael
>
>
> You sure can :)
>
> https://gist.github.com/rmounce/74dd5e92e6304688219024349bfaad0e
>
> -Ryan
>
>>
> --
> Regards,
> Ryan Mounce
>
> ryan@mounce.com.au
> 0415 799 929
>
> Sent from mobile
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-28 23:45 ` Dave Taht
@ 2019-08-29 0:55 ` Dave Taht
2019-08-29 3:28 ` Mikael Abrahamsson
2019-08-29 8:08 ` Jonathan Morton
2019-08-29 10:39 ` Ryan Mounce
2 siblings, 1 reply; 18+ messages in thread
From: Dave Taht @ 2019-08-29 0:55 UTC (permalink / raw)
To: Ryan Mounce; +Cc: Michael Welzl, ECN-Sane
fq_codel on osx ethernet (I'd only seen it on wifi before)
https://discussions.apple.com/thread/250589601
On Wed, Aug 28, 2019 at 4:45 PM Dave Taht <dave.taht@gmail.com> wrote:
>
> Very cool, thank you for jailbreaking.
>
> I'm a little puzzled over the cwr/ece ratio?
>
> 25 times received congestion experienced (CE) notification
> 37 times CWR was sent in response to ECE
> 619 times sent ECE notification
> 1029 connections received CE atleast once
>
> And um, er, while you are here busting your warranty for the sake of
> science, I'm dying to know what
>
> netstat -I each_interface_you_have_especially_lte -qq
>
> shows. They have a really short queue and some form of flow control
> and I've never seen anything trigger except though a vm
>
> my laptop:
>
> en0:
>
> [ sched: FQ_CODEL qlength: 0/128 ]
>
> [ pkts: 0 bytes: 0 dropped pkts: 1087 bytes: 177812 ]
>
> =====================================================
>
> [ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max: 8 ]
>
> [ queued pkts: 0 bytes: 0 ]
>
> [ dequeued pkts: 36767 bytes: 5237639 ]
>
> [ budget: 0 target qdelay: 10.00 msec update
> interval:100.00 msec ]
>
> [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ]
>
> [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ]
>
> [ flows total: 0 new: 0 old: 0 ]
>
> [ throttle on: 0 off: 0 drop: 0 ]
>
> =====================================================
>
> [ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max: 6 ]
>
> [ queued pkts: 0 bytes: 0 ]
>
> [ dequeued pkts: 96836 bytes: 36619772 ]
>
> [ budget: 0 target qdelay: 10.00 msec update
> interval:100.00 msec ]
>
> [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ]
>
> [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ]
>
> [ flows total: 0 new: 0 old: 0 ]
>
> [ throttle on: 0 off: 0 drop: 0 ]
>
> =====================================================
>
> [ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ]
>
> [ queued pkts: 0 bytes: 0 ]
>
> [ dequeued pkts: 5212250 bytes: 2820548356 ]
>
> [ budget: 0 target qdelay: 10.00 msec update
> interval:100.00 msec ]
>
> [ flow control: 30 feedback: 30 stalls: 0 failed: 0 ]
>
> [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ]
>
> [ flows total: 0 new: 0 old: 0 ]
>
> [ throttle on: 0 off: 0 drop: 0 ]
>
> On Wed, Aug 28, 2019 at 4:29 PM Ryan Mounce <ryan@mounce.com.au> wrote:
> >
> > On Wed, 28 Aug 2019 at 04:30, Michael Welzl <michawe@ifi.uio.no> wrote:
> >>
> >>
> >>
> >> On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote:
> >>
> >> On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote:
> >>
> >>
> >> run it as root - strange, yes, but then the output is very different.
> >>
> >> cheers,
> >> michael
> >>
> >>
> >> Sent from my iPhone
> >>
> >>
> >> speaking of iphones I have no idea what they do nowadays, any odds you
> >> could do a packet
> >> cap either on it or at a server or via wifi on a middlebox
> >>
> >>
> >> I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on my iPhone, and as a result, I could ssh into it.
> >> So I’m guessing: if you do this with a current one, you can probably still ssh into it and then perhaps run netstat.
> >>
> >> Cheers,
> >> Michael
> >
> >
> > You sure can :)
> >
> > https://gist.github.com/rmounce/74dd5e92e6304688219024349bfaad0e
> >
> > -Ryan
> >
> >>
> > --
> > Regards,
> > Ryan Mounce
> >
> > ryan@mounce.com.au
> > 0415 799 929
> >
> > Sent from mobile
>
>
>
> --
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-205-9740
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-29 0:55 ` Dave Taht
@ 2019-08-29 3:28 ` Mikael Abrahamsson
2019-08-29 4:43 ` Dave Taht
0 siblings, 1 reply; 18+ messages in thread
From: Mikael Abrahamsson @ 2019-08-29 3:28 UTC (permalink / raw)
To: Dave Taht; +Cc: Ryan Mounce, ECN-Sane
On Wed, 28 Aug 2019, Dave Taht wrote:
> fq_codel on osx ethernet (I'd only seen it on wifi before)
>
> https://discussions.apple.com/thread/250589601
On my iMac running 10.14.6 both en0 (wired ethernet) and en1 (wifi) has
FQ_CODEL enabled. Interesting!
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-29 3:28 ` Mikael Abrahamsson
@ 2019-08-29 4:43 ` Dave Taht
2019-08-29 10:30 ` Mikael Abrahamsson
0 siblings, 1 reply; 18+ messages in thread
From: Dave Taht @ 2019-08-29 4:43 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: Ryan Mounce, ECN-Sane
On Wed, Aug 28, 2019 at 8:28 PM Mikael Abrahamsson <swmike@swm.pp.se> wrote:
>
> On Wed, 28 Aug 2019, Dave Taht wrote:
>
> > fq_codel on osx ethernet (I'd only seen it on wifi before)
> >
> > https://discussions.apple.com/thread/250589601
>
> On my iMac running 10.14.6 both en0 (wired ethernet) and en1 (wifi) has
> FQ_CODEL enabled. Interesting!
Only microsoft and cablelabs are left for world domination. I should
go bug some folk I know in Microsoft.
Thing is I've not seen it really trigger on wifi at least according to
the stats. I see "flow control".
Could you hit it really hard with flent locally and see what happens?
flent -H myownlanboxalsoatgige --te=upload_streams=200 tcp_nup
and poll the netstat option.
My guess is it just triggers on vms
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-28 23:45 ` Dave Taht
2019-08-29 0:55 ` Dave Taht
@ 2019-08-29 8:08 ` Jonathan Morton
2019-08-29 10:39 ` Ryan Mounce
2 siblings, 0 replies; 18+ messages in thread
From: Jonathan Morton @ 2019-08-29 8:08 UTC (permalink / raw)
To: Dave Taht; +Cc: Ryan Mounce, ECN-Sane
> On 29 Aug, 2019, at 2:45 am, Dave Taht <dave.taht@gmail.com> wrote:
>
> I'm a little puzzled over the cwr/ece ratio?
>
> 25 times received congestion experienced (CE) notification
This is at the IP layer, on received data.
> 619 times sent ECE notification
This is at the TCP layer, acking received data. (ECE is sticky until CWR, so the ratio of the above is related to the average cwnd at congestion onset.)
> 37 times CWR was sent in response to ECE
This is at the TCP layer, on *sent* data, and therefore independent of the other two.
- Jonathan Morton
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-29 4:43 ` Dave Taht
@ 2019-08-29 10:30 ` Mikael Abrahamsson
0 siblings, 0 replies; 18+ messages in thread
From: Mikael Abrahamsson @ 2019-08-29 10:30 UTC (permalink / raw)
To: Dave Taht; +Cc: ECN-Sane
On Wed, 28 Aug 2019, Dave Taht wrote:
> Could you hit it really hard with flent locally and see what happens?
>
> flent -H myownlanboxalsoatgige --te=upload_streams=200 tcp_nup
>
> and poll the netstat option.
>
> My guess is it just triggers on vms
I ran two concurrent sessions against two machines at home, each with 100
streams (just to make sure I wasn't bottlenecking anything else apart from
the iMac port).
TCP upload sum : 473.50 464.35 Mbits/s 304
TCP upload sum : 487.96 475.68 Mbits/s 304
My iMac has 30 hours uptime, so there's some other traffic in there as
well.
en0: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu
1500 index 7
eflags=400009c0<ACCEPT_RTADV,TXSTART,RXPOLL,ARPLL,FASTLN_ON>
options=10b<RXCSUM,TXCSUM,VLAN_HWTAGGING,AV>
...
nd6 options=201<PERFORMNUD,DAD>
media: autoselect (1000baseT <full-duplex>)
status: active
type: Ethernet
link quality: 100 (good)
state availability: 0 (true)
scheduler: FQ_CODEL
link rate: 1.00 Gbps
qosmarking enabled: yes mode: none
low power mode: disabled
tcp:
75183587 packets sent
59332651 data packets (3521003642 byte)
1387664 data packets (1738258246 bytes) retransmitted
0 resend initiated by MTU discovery
13380244 ack-only packets (765 delayed)
0 URG only packet
102 window probe packets
1072241 window update packets
23163 control packets
571945 data packets sent after flow control
61 challenge ACKs sent due to unexpected SYN
4 challenge ACKs sent due to unexpected RST
18314 checksummed in software
5196 segments (272795 bytes) over IPv4
13118 segments (1494655 bytes) over IPv6
45437408 packets received
20885349 acks (for 3523699640 byte)
2369692 duplicate acks
0 ack for unsent data
26469614 packets (3766989895 byte) received in-sequence
4634 completely duplicate packets (845184 bytes)
55 old duplicate packets
0 received packet dropped due to low memory
1 packet with some dup. data (9 bytes duped)
15064 out-of-order packets (21167582 bytes)
0 packet (0 byte) of data after window
0 window probe
300216 window update packets
808 packets recovered after loss
1584 packets received after close
4 bad resets
0 discarded for bad checksum
17245 checksummed in software
4655 segments (319983 bytes) over IPv4
12590 segments (1379223 bytes) over IPv6
0 discarded for bad header offset field
0 discarded because packet too short
13206 connection requests
83 connection accepts
15 bad connection attempts
0 listen queue overflow
12426 connections established (including accepts)
13687 connections closed (including 3494 drops)
2003 connections updated cached RTT on close
2003 connections updated cached RTT variance on close
283 connections updated cached ssthresh on close
2335 connections initialized RTT from route cache
2335 connections initialized RTT variance from route cache
291 connections initialized ssthresh from route cache
28 embryonic connections dropped
38165161 segments updated rtt (of 8172122 attempts)
60615 retransmit timeouts
123 connections dropped by rexmit timeout
0 connection dropped after retransmitting FIN
18 unnecessary packet retransmissionss
102 persist timeouts
0 connection dropped by persist timeout
18871 keepalive timeouts
18764 keepalive probes sent
20 connections dropped by keepalive
11710123 correct ACK header predictions
21748077 correct data packet header predictions
159802 SACK recovery episodes
984157 segment rexmits in SACK recovery episodes
1244806263 byte rexmits in SACK recovery episodes
3356069 SACK options (SACK blocks) received
16314 SACK options (SACK blocks) sent
0 SACK scoreboard overflow
0 LRO coalesced packet
0 time LRO flow table was full
0 collision in LRO flow table
0 time LRO coalesced 2 packets
0 time LRO coalesced 3 or 4 packets
0 time LRO coalesced 5 or more packets
321561 limited transmits done
1550 early retransmits done
9623 times cumulative ack advanced along with SACK
1069 probe timeouts
83 times retransmit timeout triggered after probe
0 time probe packets were sent for an interface
0 time couldn't send probe packets for an interface
13 times fast recovery after tail loss
186 times recovered last packet
96 SACK based rescue retransmits
8180 client connections attempted to negotiate ECN
3961 client connections successfully negotiated ECN
3489 times graceful fallback to Non-ECN connection
417 times lost ECN negotiating SYN, followed by retransmission
30 server connections attempted to negotiate ECN
30 server connections successfully negotiated ECN
0 time lost ECN negotiating SYN-ACK, followed by retransmission
23 times received congestion experienced (CE) notification
16 times CWR was sent in response to ECE
4998 times sent ECE notification
14 connections received CE atleast once
8 connections received ECE atleast once
308 connections using ECN have seen packet loss but no CE
2 connections using ECN have seen packet loss and CE
20 connections using ECN received CE but no packet loss
2 connections fell back to non-ECN due to SYN-loss
472 connections fell back to non-ECN due to reordering
0 connection fell back to non-ECN due to excessive CE-markings
0 connection fell back caused by connection drop due to RST
4 connections fell back due to drop after multiple retransmits
0 connection fell back due to RST after SYN
846 times packet reordering was detected on a connection
183836 times transmitted packets were reordered
145032 times fast recovery was delayed to handle reordering
183836 times retransmission was avoided by delaying recovery
0 retransmission not needed
22347 retransmissions due to tail loss
1146 times DSACK option was sent
8877 times DSACK option was received
97 times DSACK was disabled on a connection
761 times recovered from bad retransmission using DSACK
125 times ignored DSACK due to ack loss
94 times ignored old DSACK options
7 times PMTU Blackhole detection, size reverted
5 connections were dropped after long sleep
1 connection had stretch ack algorithm disabled
0 time a TFO-cookie has been announced
0 SYN with data and a valid TFO-cookie have been received
0 SYN with TFO-cookie-request received
0 time an invalid TFO-cookie has been received
0 time we requested a TFO-cookie
0 time the peer announced a TFO-cookie
0 time we combined SYN with data and a TFO-cookie
0 time our SYN with data has been acknowledged
0 time a connection-attempt with TFO fell back to regular TCP
0 time a TFO-connection blackhole'd
0 time a TFO-cookie we sent was wrong
0 time did not received a TFO-cookie we asked for
0 time TFO got disabled due to heuristicsn
0 time TFO got blackholed in the sending direction
0 time maximum segment size was changed to default
0 time maximum segment size was changed to medium
0 time maximum segment size was changed to low
842010 timer drifts less or equal to 1 ms
10837 timer drifts less or equal to 10 ms
2 timer drifts less or equal to 20 ms
5 timer drifts less or equal to 50 ms
3 timer drifts less or equal to 100 ms
6 timer drifts less or equal to 200 ms
0 timer drift less or equal to 500 ms
0 timer drift less or equal to 1000 ms
0 timer drift greater than to 1000 ms
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn
2019-08-28 23:45 ` Dave Taht
2019-08-29 0:55 ` Dave Taht
2019-08-29 8:08 ` Jonathan Morton
@ 2019-08-29 10:39 ` Ryan Mounce
2 siblings, 0 replies; 18+ messages in thread
From: Ryan Mounce @ 2019-08-29 10:39 UTC (permalink / raw)
To: Dave Taht; +Cc: Michael Welzl, ECN-Sane
On Thu, 29 Aug 2019 at 09:15, Dave Taht <dave.taht@gmail.com> wrote:
>
> Very cool, thank you for jailbreaking.
>
> I'm a little puzzled over the cwr/ece ratio?
>
> 25 times received congestion experienced (CE) notification
> 37 times CWR was sent in response to ECE
> 619 times sent ECE notification
> 1029 connections received CE atleast once
>
> And um, er, while you are here busting your warranty for the sake of
> science, I'm dying to know what
>
> netstat -I each_interface_you_have_especially_lte -qq
>
> shows. They have a really short queue and some form of flow control
> and I've never seen anything trigger except though a vm
https://gist.github.com/rmounce/dbb05a0bad40c9e37776d15b7d53bdc9
en0 is WiFi
pdp_ip0 is LTE data
pdp_ip1 is VoLTE
This is an iPhone X w/ Qualcomm modem. FWIW I'm on Telstra, which was
(is?) one of the few networks whitelisted by Apple for their cellular
ECN experiment a few years ago.
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2019-08-29 10:39 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-27 2:03 [Ecn-sane] osx ecn Dave Taht
2019-08-27 6:47 ` Mikael Abrahamsson
2019-08-27 6:54 ` Michael Welzl
2019-08-27 7:02 ` Mikael Abrahamsson
2019-08-27 18:20 ` Dave Taht
2019-08-27 18:36 ` Dave Taht
2019-08-27 19:00 ` Michael Welzl
2019-08-28 23:29 ` Ryan Mounce
2019-08-28 23:45 ` Dave Taht
2019-08-29 0:55 ` Dave Taht
2019-08-29 3:28 ` Mikael Abrahamsson
2019-08-29 4:43 ` Dave Taht
2019-08-29 10:30 ` Mikael Abrahamsson
2019-08-29 8:08 ` Jonathan Morton
2019-08-29 10:39 ` Ryan Mounce
2019-08-27 6:58 ` Sebastian Moeller
2019-08-27 7:05 ` Jonathan Morton
2019-08-27 7:52 ` Sebastian Moeller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox