* [Ecn-sane] osx ecn @ 2019-08-27 2:03 Dave Taht 2019-08-27 6:47 ` Mikael Abrahamsson 2019-08-27 6:58 ` Sebastian Moeller 0 siblings, 2 replies; 18+ messages in thread From: Dave Taht @ 2019-08-27 2:03 UTC (permalink / raw) To: ECN-Sane netstat -sp tcp has all kinds of useful stats 7174 client connections attempted to negotiate ECN 4407 client connections successfully negotiated ECN 2192 times graceful fallback to Non-ECN connection 2053 times lost ECN negotiating SYN, followed by retransmission ?? 0 server connection attempted to negotiate ECN 0 server connection successfully negotiated ECN 0 time lost ECN negotiating SYN-ACK, followed by retransmission 90 times received congestion experienced (CE) notification 0 time CWR was sent in response to ECE 3298 times sent ECE notification 25 connections received CE atleast once 0 connection received ECE atleast once 830 connections using ECN have seen packet loss but no CE 12 connections using ECN have seen packet loss and CE 13 connections using ECN received CE but no packet loss 9 connections fell back to non-ECN due to SYN-loss 135 connections fell back to non-ECN due to reordering 0 connection fell back to non-ECN due to excessive CE-markings -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 2:03 [Ecn-sane] osx ecn Dave Taht @ 2019-08-27 6:47 ` Mikael Abrahamsson 2019-08-27 6:54 ` Michael Welzl 2019-08-27 6:58 ` Sebastian Moeller 1 sibling, 1 reply; 18+ messages in thread From: Mikael Abrahamsson @ 2019-08-27 6:47 UTC (permalink / raw) To: Dave Taht; +Cc: ECN-Sane On Mon, 26 Aug 2019, Dave Taht wrote: > netstat -sp tcp has all kinds of useful stats I am using the settings from https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ but I still get 0 on attempted ECN connections? https://mailarchive.ietf.org/arch/msg/iccrg/ghBJK_x9UqD6a5RzqWY-FmpPzBc I obviously had it working before. I'm on 10.14.6, what are you on? Btw, I checked what happened if I removed the sysctl settings and then it defaults to "2" instead of "1". -- Mikael Abrahamsson email: swmike@swm.pp.se ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 6:47 ` Mikael Abrahamsson @ 2019-08-27 6:54 ` Michael Welzl 2019-08-27 7:02 ` Mikael Abrahamsson 2019-08-27 18:36 ` Dave Taht 0 siblings, 2 replies; 18+ messages in thread From: Michael Welzl @ 2019-08-27 6:54 UTC (permalink / raw) To: Mikael Abrahamsson; +Cc: Dave Taht, ECN-Sane run it as root - strange, yes, but then the output is very different. cheers, michael Sent from my iPhone > On 27 Aug 2019, at 08:47, Mikael Abrahamsson <swmike@swm.pp.se> wrote: > >> On Mon, 26 Aug 2019, Dave Taht wrote: >> >> netstat -sp tcp has all kinds of useful stats > > I am using the settings from https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ but I still get 0 on attempted ECN connections? > > https://mailarchive.ietf.org/arch/msg/iccrg/ghBJK_x9UqD6a5RzqWY-FmpPzBc > > I obviously had it working before. I'm on 10.14.6, what are you on? > > Btw, I checked what happened if I removed the sysctl settings and then it defaults to "2" instead of "1". > > -- > Mikael Abrahamsson email: swmike@swm.pp.se > _______________________________________________ > Ecn-sane mailing list > Ecn-sane@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/ecn-sane ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 6:54 ` Michael Welzl @ 2019-08-27 7:02 ` Mikael Abrahamsson 2019-08-27 18:20 ` Dave Taht 2019-08-27 18:36 ` Dave Taht 1 sibling, 1 reply; 18+ messages in thread From: Mikael Abrahamsson @ 2019-08-27 7:02 UTC (permalink / raw) To: Michael Welzl; +Cc: Dave Taht, ECN-Sane On Tue, 27 Aug 2019, Michael Welzl wrote: > run it as root - strange, yes, but then the output is very different. Thanks, indeed. $ sudo netstat -sp tcp | egrep 'CE|EC' 11097 client connections attempted to negotiate ECN 4864 client connections successfully negotiated ECN 3800 times graceful fallback to Non-ECN connection 467 times lost ECN negotiating SYN, followed by retransmission 52 server connections attempted to negotiate ECN 52 server connections successfully negotiated ECN 0 time lost ECN negotiating SYN-ACK, followed by retransmission 427 times received congestion experienced (CE) notification 0 time CWR was sent in response to ECE 6111 times sent ECE notification 15 connections received CE atleast once 0 connection received ECE atleast once 267 connections using ECN have seen packet loss but no CE 1 connection using ECN have seen packet loss and CE 14 connections using ECN received CE but no packet loss 14 connections fell back to non-ECN due to SYN-loss 1 connection fell back to non-ECN due to reordering 0 connection fell back to non-ECN due to excessive CE-markings -- Mikael Abrahamsson email: swmike@swm.pp.se ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 7:02 ` Mikael Abrahamsson @ 2019-08-27 18:20 ` Dave Taht 0 siblings, 0 replies; 18+ messages in thread From: Dave Taht @ 2019-08-27 18:20 UTC (permalink / raw) To: Mikael Abrahamsson; +Cc: Michael Welzl, ECN-Sane An interesing piece of this is how often the syn attempt is failing. 2192 times graceful fallback to Non-ECN connection 2053 times lost ECN negotiating SYN, followed by retransmission ?? Now, there are plenty of reasons to lose syns (most firewalls rate limit these), besides the ecn flag. In my case during this boot cycle I've been engaged in a fun experiment - and one day perhaps a paper - Going to various coffee shops in the area and running the attached script. The results thus far are universally dismal, except for google starbucks which is excellent - but the packet captures seem to indicate something in addition to or other than than fq_codel in use (dpi maybe? aruba is the next hop) AND it did not negotiate ecn when I last tried. Wifi spread initially through the coffee shop phenomenon, and I've "lept over the counter" a couple times to help configure the to-the-isp qos system when I found an owner with clue, which helps a lot. When I showed 'em the before/after two bought me lunch (and I can highly recommend https://www.cemitas1.com/ in davenport for food, music, view & wifi) if that's a motivator for more folk here "to get out more", please do so. Table service is a nice thing to have when in deep hack mode and socializing more helps too. : daves-Air-6:los_gatos_starbucks d$ more testit.sh #!/bin/sh T="Los_Gatos_Starbucks" F="flent -x -H flent-fremont.bufferbloat.net -t $T" # pick a closer flent server and on linux use --socket-stats $F --te=download_streams=4 tcp_ndown $F --te=upload_streams=4 tcp_nup $F --te=upload_streams=4 tcp_2up_square $F rrul $F rrul_be On Tue, Aug 27, 2019 at 12:02 AM Mikael Abrahamsson <swmike@swm.pp.se> wrote: > > On Tue, 27 Aug 2019, Michael Welzl wrote: > > > run it as root - strange, yes, but then the output is very different. > > Thanks, indeed. > > $ sudo netstat -sp tcp | egrep 'CE|EC' > 11097 client connections attempted to negotiate ECN > 4864 client connections successfully negotiated ECN > 3800 times graceful fallback to Non-ECN connection > 467 times lost ECN negotiating SYN, followed by retransmission > 52 server connections attempted to negotiate ECN > 52 server connections successfully negotiated ECN > 0 time lost ECN negotiating SYN-ACK, followed by retransmission > 427 times received congestion experienced (CE) notification > 0 time CWR was sent in response to ECE > 6111 times sent ECE notification > 15 connections received CE atleast once > 0 connection received ECE atleast once > 267 connections using ECN have seen packet loss but no CE > 1 connection using ECN have seen packet loss and CE > 14 connections using ECN received CE but no packet loss > 14 connections fell back to non-ECN due to SYN-loss > 1 connection fell back to non-ECN due to reordering > 0 connection fell back to non-ECN due to excessive CE-markings > > > -- > Mikael Abrahamsson email: swmike@swm.pp.se -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 6:54 ` Michael Welzl 2019-08-27 7:02 ` Mikael Abrahamsson @ 2019-08-27 18:36 ` Dave Taht 2019-08-27 19:00 ` Michael Welzl 1 sibling, 1 reply; 18+ messages in thread From: Dave Taht @ 2019-08-27 18:36 UTC (permalink / raw) To: Michael Welzl; +Cc: Mikael Abrahamsson, ECN-Sane On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote: > > run it as root - strange, yes, but then the output is very different. > > cheers, > michael > > > Sent from my iPhone speaking of iphones I have no idea what they do nowadays, any odds you could do a packet cap either on it or at a server or via wifi on a middlebox > > On 27 Aug 2019, at 08:47, Mikael Abrahamsson <swmike@swm.pp.se> wrote: > > > >> On Mon, 26 Aug 2019, Dave Taht wrote: > >> > >> netstat -sp tcp has all kinds of useful stats > > > > I am using the settings from https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ but I still get 0 on attempted ECN connections? > > > > https://mailarchive.ietf.org/arch/msg/iccrg/ghBJK_x9UqD6a5RzqWY-FmpPzBc > > > > I obviously had it working before. I'm on 10.14.6, what are you on? > > > > Btw, I checked what happened if I removed the sysctl settings and then it defaults to "2" instead of "1". > > > > -- > > Mikael Abrahamsson email: swmike@swm.pp.se > > _______________________________________________ > > Ecn-sane mailing list > > Ecn-sane@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/ecn-sane > -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 18:36 ` Dave Taht @ 2019-08-27 19:00 ` Michael Welzl 2019-08-28 23:29 ` Ryan Mounce 0 siblings, 1 reply; 18+ messages in thread From: Michael Welzl @ 2019-08-27 19:00 UTC (permalink / raw) To: Dave Taht; +Cc: Mikael Abrahamsson, ECN-Sane [-- Attachment #1: Type: text/plain, Size: 764 bytes --] > On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote: > > On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no <mailto:michawe@ifi.uio.no>> wrote: >> >> run it as root - strange, yes, but then the output is very different. >> >> cheers, >> michael >> >> >> Sent from my iPhone > > speaking of iphones I have no idea what they do nowadays, any odds you > could do a packet > cap either on it or at a server or via wifi on a middlebox I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on my iPhone, and as a result, I could ssh into it. So I’m guessing: if you do this with a current one, you can probably still ssh into it and then perhaps run netstat. Cheers, Michael [-- Attachment #2: Type: text/html, Size: 5669 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 19:00 ` Michael Welzl @ 2019-08-28 23:29 ` Ryan Mounce 2019-08-28 23:45 ` Dave Taht 0 siblings, 1 reply; 18+ messages in thread From: Ryan Mounce @ 2019-08-28 23:29 UTC (permalink / raw) To: Michael Welzl; +Cc: Dave Taht, ECN-Sane [-- Attachment #1: Type: text/plain, Size: 1010 bytes --] On Wed, 28 Aug 2019 at 04:30, Michael Welzl <michawe@ifi.uio.no> wrote: > > > On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote: > > On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote: > > > run it as root - strange, yes, but then the output is very different. > > cheers, > michael > > > Sent from my iPhone > > > speaking of iphones I have no idea what they do nowadays, any odds you > could do a packet > cap either on it or at a server or via wifi on a middlebox > > > I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on > my iPhone, and as a result, I could ssh into it. > So I’m guessing: if you do this with a current one, you can probably still > ssh into it and then perhaps run netstat. > > Cheers, > Michael > You sure can :) https://gist.github.com/rmounce/74dd5e92e6304688219024349bfaad0e -Ryan > -- Regards, Ryan Mounce ryan@mounce.com.au 0415 799 929 Sent from mobile [-- Attachment #2: Type: text/html, Size: 5469 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-28 23:29 ` Ryan Mounce @ 2019-08-28 23:45 ` Dave Taht 2019-08-29 0:55 ` Dave Taht ` (2 more replies) 0 siblings, 3 replies; 18+ messages in thread From: Dave Taht @ 2019-08-28 23:45 UTC (permalink / raw) To: Ryan Mounce; +Cc: Michael Welzl, ECN-Sane Very cool, thank you for jailbreaking. I'm a little puzzled over the cwr/ece ratio? 25 times received congestion experienced (CE) notification 37 times CWR was sent in response to ECE 619 times sent ECE notification 1029 connections received CE atleast once And um, er, while you are here busting your warranty for the sake of science, I'm dying to know what netstat -I each_interface_you_have_especially_lte -qq shows. They have a really short queue and some form of flow control and I've never seen anything trigger except though a vm my laptop: en0: [ sched: FQ_CODEL qlength: 0/128 ] [ pkts: 0 bytes: 0 dropped pkts: 1087 bytes: 177812 ] ===================================================== [ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max: 8 ] [ queued pkts: 0 bytes: 0 ] [ dequeued pkts: 36767 bytes: 5237639 ] [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] [ flows total: 0 new: 0 old: 0 ] [ throttle on: 0 off: 0 drop: 0 ] ===================================================== [ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max: 6 ] [ queued pkts: 0 bytes: 0 ] [ dequeued pkts: 96836 bytes: 36619772 ] [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] [ flows total: 0 new: 0 old: 0 ] [ throttle on: 0 off: 0 drop: 0 ] ===================================================== [ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ] [ queued pkts: 0 bytes: 0 ] [ dequeued pkts: 5212250 bytes: 2820548356 ] [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] [ flow control: 30 feedback: 30 stalls: 0 failed: 0 ] [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] [ flows total: 0 new: 0 old: 0 ] [ throttle on: 0 off: 0 drop: 0 ] On Wed, Aug 28, 2019 at 4:29 PM Ryan Mounce <ryan@mounce.com.au> wrote: > > On Wed, 28 Aug 2019 at 04:30, Michael Welzl <michawe@ifi.uio.no> wrote: >> >> >> >> On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote: >> >> On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote: >> >> >> run it as root - strange, yes, but then the output is very different. >> >> cheers, >> michael >> >> >> Sent from my iPhone >> >> >> speaking of iphones I have no idea what they do nowadays, any odds you >> could do a packet >> cap either on it or at a server or via wifi on a middlebox >> >> >> I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on my iPhone, and as a result, I could ssh into it. >> So I’m guessing: if you do this with a current one, you can probably still ssh into it and then perhaps run netstat. >> >> Cheers, >> Michael > > > You sure can :) > > https://gist.github.com/rmounce/74dd5e92e6304688219024349bfaad0e > > -Ryan > >> > -- > Regards, > Ryan Mounce > > ryan@mounce.com.au > 0415 799 929 > > Sent from mobile -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-28 23:45 ` Dave Taht @ 2019-08-29 0:55 ` Dave Taht 2019-08-29 3:28 ` Mikael Abrahamsson 2019-08-29 8:08 ` Jonathan Morton 2019-08-29 10:39 ` Ryan Mounce 2 siblings, 1 reply; 18+ messages in thread From: Dave Taht @ 2019-08-29 0:55 UTC (permalink / raw) To: Ryan Mounce; +Cc: Michael Welzl, ECN-Sane fq_codel on osx ethernet (I'd only seen it on wifi before) https://discussions.apple.com/thread/250589601 On Wed, Aug 28, 2019 at 4:45 PM Dave Taht <dave.taht@gmail.com> wrote: > > Very cool, thank you for jailbreaking. > > I'm a little puzzled over the cwr/ece ratio? > > 25 times received congestion experienced (CE) notification > 37 times CWR was sent in response to ECE > 619 times sent ECE notification > 1029 connections received CE atleast once > > And um, er, while you are here busting your warranty for the sake of > science, I'm dying to know what > > netstat -I each_interface_you_have_especially_lte -qq > > shows. They have a really short queue and some form of flow control > and I've never seen anything trigger except though a vm > > my laptop: > > en0: > > [ sched: FQ_CODEL qlength: 0/128 ] > > [ pkts: 0 bytes: 0 dropped pkts: 1087 bytes: 177812 ] > > ===================================================== > > [ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max: 8 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 36767 bytes: 5237639 ] > > [ budget: 0 target qdelay: 10.00 msec update > interval:100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > ===================================================== > > [ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max: 6 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 96836 bytes: 36619772 ] > > [ budget: 0 target qdelay: 10.00 msec update > interval:100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > ===================================================== > > [ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 5212250 bytes: 2820548356 ] > > [ budget: 0 target qdelay: 10.00 msec update > interval:100.00 msec ] > > [ flow control: 30 feedback: 30 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > On Wed, Aug 28, 2019 at 4:29 PM Ryan Mounce <ryan@mounce.com.au> wrote: > > > > On Wed, 28 Aug 2019 at 04:30, Michael Welzl <michawe@ifi.uio.no> wrote: > >> > >> > >> > >> On Aug 27, 2019, at 8:36 PM, Dave Taht <dave.taht@gmail.com> wrote: > >> > >> On Mon, Aug 26, 2019 at 11:54 PM Michael Welzl <michawe@ifi.uio.no> wrote: > >> > >> > >> run it as root - strange, yes, but then the output is very different. > >> > >> cheers, > >> michael > >> > >> > >> Sent from my iPhone > >> > >> > >> speaking of iphones I have no idea what they do nowadays, any odds you > >> could do a packet > >> cap either on it or at a server or via wifi on a middlebox > >> > >> > >> I wouldn’t know … but I can guess: some years ago, I did a “jailbreak” on my iPhone, and as a result, I could ssh into it. > >> So I’m guessing: if you do this with a current one, you can probably still ssh into it and then perhaps run netstat. > >> > >> Cheers, > >> Michael > > > > > > You sure can :) > > > > https://gist.github.com/rmounce/74dd5e92e6304688219024349bfaad0e > > > > -Ryan > > > >> > > -- > > Regards, > > Ryan Mounce > > > > ryan@mounce.com.au > > 0415 799 929 > > > > Sent from mobile > > > > -- > > Dave Täht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-29 0:55 ` Dave Taht @ 2019-08-29 3:28 ` Mikael Abrahamsson 2019-08-29 4:43 ` Dave Taht 0 siblings, 1 reply; 18+ messages in thread From: Mikael Abrahamsson @ 2019-08-29 3:28 UTC (permalink / raw) To: Dave Taht; +Cc: Ryan Mounce, ECN-Sane On Wed, 28 Aug 2019, Dave Taht wrote: > fq_codel on osx ethernet (I'd only seen it on wifi before) > > https://discussions.apple.com/thread/250589601 On my iMac running 10.14.6 both en0 (wired ethernet) and en1 (wifi) has FQ_CODEL enabled. Interesting! -- Mikael Abrahamsson email: swmike@swm.pp.se ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-29 3:28 ` Mikael Abrahamsson @ 2019-08-29 4:43 ` Dave Taht 2019-08-29 10:30 ` Mikael Abrahamsson 0 siblings, 1 reply; 18+ messages in thread From: Dave Taht @ 2019-08-29 4:43 UTC (permalink / raw) To: Mikael Abrahamsson; +Cc: Ryan Mounce, ECN-Sane On Wed, Aug 28, 2019 at 8:28 PM Mikael Abrahamsson <swmike@swm.pp.se> wrote: > > On Wed, 28 Aug 2019, Dave Taht wrote: > > > fq_codel on osx ethernet (I'd only seen it on wifi before) > > > > https://discussions.apple.com/thread/250589601 > > On my iMac running 10.14.6 both en0 (wired ethernet) and en1 (wifi) has > FQ_CODEL enabled. Interesting! Only microsoft and cablelabs are left for world domination. I should go bug some folk I know in Microsoft. Thing is I've not seen it really trigger on wifi at least according to the stats. I see "flow control". Could you hit it really hard with flent locally and see what happens? flent -H myownlanboxalsoatgige --te=upload_streams=200 tcp_nup and poll the netstat option. My guess is it just triggers on vms > -- > Mikael Abrahamsson email: swmike@swm.pp.se -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-29 4:43 ` Dave Taht @ 2019-08-29 10:30 ` Mikael Abrahamsson 0 siblings, 0 replies; 18+ messages in thread From: Mikael Abrahamsson @ 2019-08-29 10:30 UTC (permalink / raw) To: Dave Taht; +Cc: ECN-Sane On Wed, 28 Aug 2019, Dave Taht wrote: > Could you hit it really hard with flent locally and see what happens? > > flent -H myownlanboxalsoatgige --te=upload_streams=200 tcp_nup > > and poll the netstat option. > > My guess is it just triggers on vms I ran two concurrent sessions against two machines at home, each with 100 streams (just to make sure I wasn't bottlenecking anything else apart from the iMac port). TCP upload sum : 473.50 464.35 Mbits/s 304 TCP upload sum : 487.96 475.68 Mbits/s 304 My iMac has 30 hours uptime, so there's some other traffic in there as well. en0: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 index 7 eflags=400009c0<ACCEPT_RTADV,TXSTART,RXPOLL,ARPLL,FASTLN_ON> options=10b<RXCSUM,TXCSUM,VLAN_HWTAGGING,AV> ... nd6 options=201<PERFORMNUD,DAD> media: autoselect (1000baseT <full-duplex>) status: active type: Ethernet link quality: 100 (good) state availability: 0 (true) scheduler: FQ_CODEL link rate: 1.00 Gbps qosmarking enabled: yes mode: none low power mode: disabled tcp: 75183587 packets sent 59332651 data packets (3521003642 byte) 1387664 data packets (1738258246 bytes) retransmitted 0 resend initiated by MTU discovery 13380244 ack-only packets (765 delayed) 0 URG only packet 102 window probe packets 1072241 window update packets 23163 control packets 571945 data packets sent after flow control 61 challenge ACKs sent due to unexpected SYN 4 challenge ACKs sent due to unexpected RST 18314 checksummed in software 5196 segments (272795 bytes) over IPv4 13118 segments (1494655 bytes) over IPv6 45437408 packets received 20885349 acks (for 3523699640 byte) 2369692 duplicate acks 0 ack for unsent data 26469614 packets (3766989895 byte) received in-sequence 4634 completely duplicate packets (845184 bytes) 55 old duplicate packets 0 received packet dropped due to low memory 1 packet with some dup. data (9 bytes duped) 15064 out-of-order packets (21167582 bytes) 0 packet (0 byte) of data after window 0 window probe 300216 window update packets 808 packets recovered after loss 1584 packets received after close 4 bad resets 0 discarded for bad checksum 17245 checksummed in software 4655 segments (319983 bytes) over IPv4 12590 segments (1379223 bytes) over IPv6 0 discarded for bad header offset field 0 discarded because packet too short 13206 connection requests 83 connection accepts 15 bad connection attempts 0 listen queue overflow 12426 connections established (including accepts) 13687 connections closed (including 3494 drops) 2003 connections updated cached RTT on close 2003 connections updated cached RTT variance on close 283 connections updated cached ssthresh on close 2335 connections initialized RTT from route cache 2335 connections initialized RTT variance from route cache 291 connections initialized ssthresh from route cache 28 embryonic connections dropped 38165161 segments updated rtt (of 8172122 attempts) 60615 retransmit timeouts 123 connections dropped by rexmit timeout 0 connection dropped after retransmitting FIN 18 unnecessary packet retransmissionss 102 persist timeouts 0 connection dropped by persist timeout 18871 keepalive timeouts 18764 keepalive probes sent 20 connections dropped by keepalive 11710123 correct ACK header predictions 21748077 correct data packet header predictions 159802 SACK recovery episodes 984157 segment rexmits in SACK recovery episodes 1244806263 byte rexmits in SACK recovery episodes 3356069 SACK options (SACK blocks) received 16314 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 LRO coalesced packet 0 time LRO flow table was full 0 collision in LRO flow table 0 time LRO coalesced 2 packets 0 time LRO coalesced 3 or 4 packets 0 time LRO coalesced 5 or more packets 321561 limited transmits done 1550 early retransmits done 9623 times cumulative ack advanced along with SACK 1069 probe timeouts 83 times retransmit timeout triggered after probe 0 time probe packets were sent for an interface 0 time couldn't send probe packets for an interface 13 times fast recovery after tail loss 186 times recovered last packet 96 SACK based rescue retransmits 8180 client connections attempted to negotiate ECN 3961 client connections successfully negotiated ECN 3489 times graceful fallback to Non-ECN connection 417 times lost ECN negotiating SYN, followed by retransmission 30 server connections attempted to negotiate ECN 30 server connections successfully negotiated ECN 0 time lost ECN negotiating SYN-ACK, followed by retransmission 23 times received congestion experienced (CE) notification 16 times CWR was sent in response to ECE 4998 times sent ECE notification 14 connections received CE atleast once 8 connections received ECE atleast once 308 connections using ECN have seen packet loss but no CE 2 connections using ECN have seen packet loss and CE 20 connections using ECN received CE but no packet loss 2 connections fell back to non-ECN due to SYN-loss 472 connections fell back to non-ECN due to reordering 0 connection fell back to non-ECN due to excessive CE-markings 0 connection fell back caused by connection drop due to RST 4 connections fell back due to drop after multiple retransmits 0 connection fell back due to RST after SYN 846 times packet reordering was detected on a connection 183836 times transmitted packets were reordered 145032 times fast recovery was delayed to handle reordering 183836 times retransmission was avoided by delaying recovery 0 retransmission not needed 22347 retransmissions due to tail loss 1146 times DSACK option was sent 8877 times DSACK option was received 97 times DSACK was disabled on a connection 761 times recovered from bad retransmission using DSACK 125 times ignored DSACK due to ack loss 94 times ignored old DSACK options 7 times PMTU Blackhole detection, size reverted 5 connections were dropped after long sleep 1 connection had stretch ack algorithm disabled 0 time a TFO-cookie has been announced 0 SYN with data and a valid TFO-cookie have been received 0 SYN with TFO-cookie-request received 0 time an invalid TFO-cookie has been received 0 time we requested a TFO-cookie 0 time the peer announced a TFO-cookie 0 time we combined SYN with data and a TFO-cookie 0 time our SYN with data has been acknowledged 0 time a connection-attempt with TFO fell back to regular TCP 0 time a TFO-connection blackhole'd 0 time a TFO-cookie we sent was wrong 0 time did not received a TFO-cookie we asked for 0 time TFO got disabled due to heuristicsn 0 time TFO got blackholed in the sending direction 0 time maximum segment size was changed to default 0 time maximum segment size was changed to medium 0 time maximum segment size was changed to low 842010 timer drifts less or equal to 1 ms 10837 timer drifts less or equal to 10 ms 2 timer drifts less or equal to 20 ms 5 timer drifts less or equal to 50 ms 3 timer drifts less or equal to 100 ms 6 timer drifts less or equal to 200 ms 0 timer drift less or equal to 500 ms 0 timer drift less or equal to 1000 ms 0 timer drift greater than to 1000 ms -- Mikael Abrahamsson email: swmike@swm.pp.se ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-28 23:45 ` Dave Taht 2019-08-29 0:55 ` Dave Taht @ 2019-08-29 8:08 ` Jonathan Morton 2019-08-29 10:39 ` Ryan Mounce 2 siblings, 0 replies; 18+ messages in thread From: Jonathan Morton @ 2019-08-29 8:08 UTC (permalink / raw) To: Dave Taht; +Cc: Ryan Mounce, ECN-Sane > On 29 Aug, 2019, at 2:45 am, Dave Taht <dave.taht@gmail.com> wrote: > > I'm a little puzzled over the cwr/ece ratio? > > 25 times received congestion experienced (CE) notification This is at the IP layer, on received data. > 619 times sent ECE notification This is at the TCP layer, acking received data. (ECE is sticky until CWR, so the ratio of the above is related to the average cwnd at congestion onset.) > 37 times CWR was sent in response to ECE This is at the TCP layer, on *sent* data, and therefore independent of the other two. - Jonathan Morton ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-28 23:45 ` Dave Taht 2019-08-29 0:55 ` Dave Taht 2019-08-29 8:08 ` Jonathan Morton @ 2019-08-29 10:39 ` Ryan Mounce 2 siblings, 0 replies; 18+ messages in thread From: Ryan Mounce @ 2019-08-29 10:39 UTC (permalink / raw) To: Dave Taht; +Cc: Michael Welzl, ECN-Sane On Thu, 29 Aug 2019 at 09:15, Dave Taht <dave.taht@gmail.com> wrote: > > Very cool, thank you for jailbreaking. > > I'm a little puzzled over the cwr/ece ratio? > > 25 times received congestion experienced (CE) notification > 37 times CWR was sent in response to ECE > 619 times sent ECE notification > 1029 connections received CE atleast once > > And um, er, while you are here busting your warranty for the sake of > science, I'm dying to know what > > netstat -I each_interface_you_have_especially_lte -qq > > shows. They have a really short queue and some form of flow control > and I've never seen anything trigger except though a vm https://gist.github.com/rmounce/dbb05a0bad40c9e37776d15b7d53bdc9 en0 is WiFi pdp_ip0 is LTE data pdp_ip1 is VoLTE This is an iPhone X w/ Qualcomm modem. FWIW I'm on Telstra, which was (is?) one of the few networks whitelisted by Apple for their cellular ECN experiment a few years ago. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 2:03 [Ecn-sane] osx ecn Dave Taht 2019-08-27 6:47 ` Mikael Abrahamsson @ 2019-08-27 6:58 ` Sebastian Moeller 2019-08-27 7:05 ` Jonathan Morton 1 sibling, 1 reply; 18+ messages in thread From: Sebastian Moeller @ 2019-08-27 6:58 UTC (permalink / raw) To: Dave Täht; +Cc: ECN-Sane Hi Dave, > On Aug 27, 2019, at 04:03, Dave Taht <dave.taht@gmail.com> wrote: > > 0 connection fell back to non-ECN due to excessive CE-markings I wonder whether macosx would be able to gracefully survive a misclassification into a L4S/NQB queue at a dual queue AQM? What is the threshold here? Best Regards Sebastian ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 6:58 ` Sebastian Moeller @ 2019-08-27 7:05 ` Jonathan Morton 2019-08-27 7:52 ` Sebastian Moeller 0 siblings, 1 reply; 18+ messages in thread From: Jonathan Morton @ 2019-08-27 7:05 UTC (permalink / raw) To: Sebastian Moeller; +Cc: Dave Täht, ECN-Sane > On 27 Aug, 2019, at 9:58 am, Sebastian Moeller <moeller0@gmx.de> wrote: > >> 0 connection fell back to non-ECN due to excessive CE-markings > > I wonder whether macosx would be able to gracefully survive a misclassification into a L4S/NQB queue at a dual queue AQM? What is the threshold here? No, I don't think so. L4S assumes the DCTCP response function which stabilises at 2 CE marks per RTT, which is still a fairly low percentage if the cwnd is high. I doubt that would be detected as "excessive" in this context. Also, because the RFC-3168 response to CE is much sharper, the actual marking rate will be lower, as the flow won't stay in the saturating regime where marking occurs. - Jonathan Morton ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Ecn-sane] osx ecn 2019-08-27 7:05 ` Jonathan Morton @ 2019-08-27 7:52 ` Sebastian Moeller 0 siblings, 0 replies; 18+ messages in thread From: Sebastian Moeller @ 2019-08-27 7:52 UTC (permalink / raw) To: Jonathan Morton; +Cc: Dave Täht, ECN-Sane Hi Jonathan, > On Aug 27, 2019, at 09:05, Jonathan Morton <chromatix99@gmail.com> wrote: > >> On 27 Aug, 2019, at 9:58 am, Sebastian Moeller <moeller0@gmx.de> wrote: >> >>> 0 connection fell back to non-ECN due to excessive CE-markings >> >> I wonder whether macosx would be able to gracefully survive a misclassification into a L4S/NQB queue at a dual queue AQM? What is the threshold here? > > No, I don't think so. L4S assumes the DCTCP response function which stabilises at 2 CE marks per RTT, which is still a fairly low percentage if the cwnd is high. I doubt that would be detected as "excessive" in this context. Also, because the RFC-3168 response to CE is much sharper, the actual marking rate will be lower, as the flow won't stay in the saturating regime where marking occurs. > > - Jonathan Morton thanks for the detailed answer. Best Regards Sebastian ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2019-08-29 10:39 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-08-27 2:03 [Ecn-sane] osx ecn Dave Taht 2019-08-27 6:47 ` Mikael Abrahamsson 2019-08-27 6:54 ` Michael Welzl 2019-08-27 7:02 ` Mikael Abrahamsson 2019-08-27 18:20 ` Dave Taht 2019-08-27 18:36 ` Dave Taht 2019-08-27 19:00 ` Michael Welzl 2019-08-28 23:29 ` Ryan Mounce 2019-08-28 23:45 ` Dave Taht 2019-08-29 0:55 ` Dave Taht 2019-08-29 3:28 ` Mikael Abrahamsson 2019-08-29 4:43 ` Dave Taht 2019-08-29 10:30 ` Mikael Abrahamsson 2019-08-29 8:08 ` Jonathan Morton 2019-08-29 10:39 ` Ryan Mounce 2019-08-27 6:58 ` Sebastian Moeller 2019-08-27 7:05 ` Jonathan Morton 2019-08-27 7:52 ` Sebastian Moeller
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox