* [Ecn-sane] results of two simple ECN tests @ 2019-02-17 11:26 Pete Heist 2019-02-17 13:02 ` Sebastian Moeller 0 siblings, 1 reply; 6+ messages in thread From: Pete Heist @ 2019-02-17 11:26 UTC (permalink / raw) To: ecn-sane [-- Attachment #1: Type: text/plain, Size: 973 bytes --] Attached are some scripts that run two simple tests of ECN with veth devices, with and without ECN. The topology is: client - middlebox (20Mbit htb+fq_codel egress both ways) - net (40ms netem delay both ways, i.e. 80ms RTT) - server Here are some results from the APU2 with Debian 9 / kernel 4.9.0-8: Test 1 (“One vs one”, two clients uploads competing, one flow each for 60 seconds, measure total data transferred): No ECN, 63.2 + 63.5 transferred = 126.7MB ECN, 63.2 + 61.5 transferred = 124.7MB Test 2 (“One vs pulses”, client #1: upload for 60 seconds, client #2: 40x 1M uploads sequentially (iperf -n 1M), measure client #1 data transferred): No ECN, 63.2 MB transferred ECN, 65.0 MB transferred Can anyone suggest changes to this test or a better test that would more clearly show the benefit of ECN? I guess we’d want more congestion and the cost of each lost packet to be higher, meaning higher RTTs and more clients? Pete [-- Attachment #2: veth_ecn.tar.gz --] [-- Type: application/x-gzip, Size: 5391 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Ecn-sane] results of two simple ECN tests 2019-02-17 11:26 [Ecn-sane] results of two simple ECN tests Pete Heist @ 2019-02-17 13:02 ` Sebastian Moeller 2019-02-17 20:57 ` Pete Heist 0 siblings, 1 reply; 6+ messages in thread From: Sebastian Moeller @ 2019-02-17 13:02 UTC (permalink / raw) To: ecn-sane, Pete Heist [-- Attachment #1: Type: text/plain, Size: 1185 bytes --] Did you use SACK? On February 17, 2019 12:26:51 PM GMT+01:00, Pete Heist <pete@heistp.net> wrote: >Attached are some scripts that run two simple tests of ECN with veth >devices, with and without ECN. The topology is: > >client - middlebox (20Mbit htb+fq_codel egress both ways) - net (40ms >netem delay both ways, i.e. 80ms RTT) - server > >Here are some results from the APU2 with Debian 9 / kernel 4.9.0-8: > >Test 1 (“One vs one”, two clients uploads competing, one flow each for >60 seconds, measure total data transferred): > > No ECN, 63.2 + 63.5 transferred = 126.7MB > ECN, 63.2 + 61.5 transferred = 124.7MB > >Test 2 (“One vs pulses”, client #1: upload for 60 seconds, client #2: >40x 1M uploads sequentially (iperf -n 1M), measure client #1 data >transferred): > > No ECN, 63.2 MB transferred > ECN, 65.0 MB transferred > >Can anyone suggest changes to this test or a better test that would >more clearly show the benefit of ECN? I guess we’d want more congestion >and the cost of each lost packet to be higher, meaning higher RTTs and >more clients? > >Pete -- Sent from my Android device with K-9 Mail. Please excuse my brevity. [-- Attachment #2: Type: text/html, Size: 1440 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Ecn-sane] results of two simple ECN tests 2019-02-17 13:02 ` Sebastian Moeller @ 2019-02-17 20:57 ` Pete Heist 2019-02-17 21:07 ` Sebastian Moeller 0 siblings, 1 reply; 6+ messages in thread From: Pete Heist @ 2019-02-17 20:57 UTC (permalink / raw) To: Sebastian Moeller; +Cc: ecn-sane [-- Attachment #1: Type: text/plain, Size: 1597 bytes --] Yes, it's enabled by default. I think I'm just measuring the wrong thing. ECN seems to be about reducing TCP RTT and jitter, not increasing throughput per se. I'll rather compare packet captures with it on and off to look for an improvement in the TCP RTT spikes typically associated with drops. > On 17 Feb 2019, at 14:02, Sebastian Moeller <moeller0@gmx.de> wrote: > > Did you use SACK? > >> On February 17, 2019 12:26:51 PM GMT+01:00, Pete Heist <pete@heistp.net> wrote: >> Attached are some scripts that run two simple tests of ECN with veth devices, with and without ECN. The topology is: >> >> client - middlebox (20Mbit htb+fq_codel egress both ways) - net (40ms netem delay both ways, i.e. 80ms RTT) - server >> >> Here are some results from the APU2 with Debian 9 / kernel 4.9.0-8: >> >> Test 1 (“One vs one”, two clients uploads competing, one flow each for 60 seconds, measure total data transferred): >> >> No ECN, 63.2 + 63.5 transferred = 126.7MB >> ECN, 63.2 + 61.5 transferred = 124.7MB >> >> Test 2 (“One vs pulses”, client #1: upload for 60 seconds, client #2: 40x 1M uploads sequentially (iperf -n 1M), measure client #1 data transferred): >> >> No ECN, 63.2 MB transferred >> ECN, 65.0 MB transferred >> >> Can anyone suggest changes to this test or a better test that would more clearly show the benefit of ECN? I guess we’d want more congestion and the cost of each lost packet to be higher, meaning higher RTTs and more clients? >> >> Pete > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. [-- Attachment #2: Type: text/html, Size: 2054 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Ecn-sane] results of two simple ECN tests 2019-02-17 20:57 ` Pete Heist @ 2019-02-17 21:07 ` Sebastian Moeller 2019-02-18 10:33 ` Pete Heist 0 siblings, 1 reply; 6+ messages in thread From: Sebastian Moeller @ 2019-02-17 21:07 UTC (permalink / raw) To: Pete Heist; +Cc: ecn-sane Hi Pete, > On Feb 17, 2019, at 21:57, Pete Heist <pete@heistp.net> wrote: > > Yes, it's enabled by default. I think I'm just measuring the wrong thing. ECN seems to be about reducing TCP RTT and jitter, not increasing throughput per se. But in your test, a reduced TCP RTT should result in a higher throughput, no? > I'll rather compare packet captures with it on and off to look for an improvement in the TCP RTT spikes typically associated with drops. Well, the big danger of dropping packets is that you might stall a flow (say, by dropping enough consecutive packets to drive the flow into RTO) something much less likely with SACK (at least that is my understanding of one of SACKs promises). For post-bottleneck shaping there is also the issue that ECN-marking an incoming packet will at least not have wasted the "transmit-slot" that was occupied for the transmit. But given that TCP was designed to interpret lost packets as a sign of having exceeded capacity I am not that amazed that it still does a decent job doing so ;) I believe there is an argument for giving ECN capable flows a lower marking probability than non-ECN flows would get a drop probability, but since that is easily gamed (end-points negotiating ECN but simply not slowing down on receiving marks) it is not an option for the wide-internet and hence ECN should not give much improvement in throughput (although it will reduce the number of retransmitted packets). I wonder how this would change if you would reconfigure the shaper to half the bandwidth in the middle of the > > On 17 Feb 2019, at 14:02, Sebastian Moeller <moeller0@gmx.de> wrote: > >> Did you use SACK? >> >> On February 17, 2019 12:26:51 PM GMT+01:00, Pete Heist <pete@heistp.net> wrote: >> Attached are some scripts that run two simple tests of ECN with veth devices, with and without ECN. The topology is: >> >> client - middlebox (20Mbit htb+fq_codel egress both ways) - net (40ms netem delay both ways, i.e. 80ms RTT) - server >> >> Here are some results from the APU2 with Debian 9 / kernel 4.9.0-8: >> >> Test 1 (“One vs one”, two clients uploads competing, one flow each for 60 seconds, measure total data transferred): >> >> No ECN, 63.2 + 63.5 transferred = 126.7MB >> ECN, 63.2 + 61.5 transferred = 124.7MB >> >> Test 2 (“One vs pulses”, client #1: upload for 60 seconds, client #2: 40x 1M uploads sequentially (iperf -n 1M), measure client #1 data transferred): >> >> No ECN, 63.2 MB transferred >> ECN, 65.0 MB transferred >> >> Can anyone suggest changes to this test or a better test that would more clearly show the benefit of ECN? I guess we’d want more congestion and the cost of each lost packet to be higher, meaning higher RTTs and more clients? >> >> Pete >> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Ecn-sane] results of two simple ECN tests 2019-02-17 21:07 ` Sebastian Moeller @ 2019-02-18 10:33 ` Pete Heist 2019-02-18 19:24 ` Dave Taht 0 siblings, 1 reply; 6+ messages in thread From: Pete Heist @ 2019-02-18 10:33 UTC (permalink / raw) To: Sebastian Moeller; +Cc: ecn-sane [-- Attachment #1: Type: text/plain, Size: 2676 bytes --] > On Feb 17, 2019, at 10:07 PM, Sebastian Moeller <moeller0@gmx.de> wrote: > > But in your test, a reduced TCP RTT should result in a higher throughput, no? Theoretically, but I think the difference may be only marginal at modern bandwidths. For example, during my 20Mbit, 10 second “one vs one” iperf test, 23770 segments are sent and only 9 are dropped. ECN saves those 9 segments, but that’s only 13626 bytes (0.038%), plus what ever side effects there may be. My current test with iperf isn’t sensitive enough to measure a corresponding difference in throughput. Note that at https://en.wikipedia.org/wiki/Explicit_Congestion_Notification#Effects_on_performance <https://en.wikipedia.org/wiki/Explicit_Congestion_Notification#Effects_on_performance>, it claims "Effects of ECN on bulk throughput are less clear”, which references a 2003 paper "Marek Malowidzki, Simulation-based Study of ECN Performance in RED Networks”. >> I'll rather compare packet captures with it on and off to look for an improvement in the TCP RTT spikes typically associated with drops. > > Well, the big danger of dropping packets is that you might stall a flow (say, by dropping enough consecutive packets to drive the flow into RTO) something much less likely with SACK (at least that is my understanding of one of SACKs promises). That may be, but my simple simulation doesn’t reproduce that case. I’ve updated it and made some TCP RTT graphs, which does show a clearer difference with ECN. All the files and pcaps are here: https://www.heistp.net/downloads/veth_ecn/ <https://www.heistp.net/downloads/veth_ecn/> Compare these two one-vs-one RTT graphs and the difference with ECN enabled can be seen: https://www.heistp.net/downloads/veth_ecn/client_one_vs_one_noecn_rtt.png <https://www.heistp.net/downloads/veth_ecn/client_one_vs_one_noecn_rtt.png> https://www.heistp.net/downloads/veth_ecn/client_one_vs_one_ecn_rtt.png <https://www.heistp.net/downloads/veth_ecn/client_one_vs_one_ecn_rtt.png> Similar for one-vs-pulses: https://www.heistp.net/downloads/veth_ecn/client_one_vs_pulses_noecn_rtt.png <https://www.heistp.net/downloads/veth_ecn/client_one_vs_pulses_noecn_rtt.png> https://www.heistp.net/downloads/veth_ecn/client_one_vs_pulses_ecn_rtt.png <https://www.heistp.net/downloads/veth_ecn/client_one_vs_pulses_ecn_rtt.png> The graphs of TCP window are also more appealing in the ECN case, at least. Now, re-reading Dave’s posts about why the ECN-sane project was started, there appear to be some pathological cases. This simple test doesn’t get to those. For now just wanted to get in touch with some basics. :) [-- Attachment #2: Type: text/html, Size: 4019 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Ecn-sane] results of two simple ECN tests 2019-02-18 10:33 ` Pete Heist @ 2019-02-18 19:24 ` Dave Taht 0 siblings, 0 replies; 6+ messages in thread From: Dave Taht @ 2019-02-18 19:24 UTC (permalink / raw) To: Pete Heist; +Cc: Sebastian Moeller, ecn-sane I appreciate the shot at going back to basics and the packet capture driven tests. ECN does help keep latency down at longer RTTs. But the bandwidth improvements are rather minimal in this test case, and basically a function of what would have been the drop rate. The pathological cases I was discussing at one point (100 flows through a short low bandwidth link) dramatically increase delay, cause starvation for other flows (including new flows trying to start). ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-02-18 19:24 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-02-17 11:26 [Ecn-sane] results of two simple ECN tests Pete Heist 2019-02-17 13:02 ` Sebastian Moeller 2019-02-17 20:57 ` Pete Heist 2019-02-17 21:07 ` Sebastian Moeller 2019-02-18 10:33 ` Pete Heist 2019-02-18 19:24 ` Dave Taht
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
