* [Bloat] different speeds on different ports? (benchmarking fun) @ 2017-08-30 5:15 Aaron Wood 2017-09-21 2:06 ` Dave Taht ` (4 more replies) 0 siblings, 5 replies; 11+ messages in thread From: Aaron Wood @ 2017-08-30 5:15 UTC (permalink / raw) To: bloat [-- Attachment #1.1: Type: text/plain, Size: 1302 bytes --] I don't have a full writeup yet, but wanted to ask if people on here have run into this. I'm seeing a disparity between flent and the dslreports speed tests. On my connection at home (Comcast 150/12), I figured it was something related to the test implementations, but minor. But on a connect at a friend with business-class Comcast (300/12), we're seeing a huge difference. Flent can't seem to achieve more than 120Mbps, often with an early, couple-second hump at a much higher speed. But dslreports' speed tests gets the full 300Mbps. In looking closer at my connection, with sqm (cake) turned off, I'm seeing ~180Mbps download with 500ms of bufferbloat when I use the dslreports test ( http://www.dslreports.com/speedtest/20805152). Yet flent can't come close to that, even with the tcp_12down test: The current hypothesis that we have is that this is due to either traffic class, or the ports that traffic are running on. I've ruled out the ping streams, as a parallel set of netperf tcp_maerts downloads has the same 120Mbps roof. It would be interesting if we could run some netperf tests using port 80/443 for the listening socket for the data connection (although if doing deep-packet inspection, we might need to use an actual HTTP transfer). -Aaron [-- Attachment #1.2: Type: text/html, Size: 1611 bytes --] [-- Attachment #2: raw.totals.png --] [-- Type: image/png, Size: 257944 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-08-30 5:15 [Bloat] different speeds on different ports? (benchmarking fun) Aaron Wood @ 2017-09-21 2:06 ` Dave Taht 2017-09-21 2:34 ` Dave Taht ` (3 subsequent siblings) 4 siblings, 0 replies; 11+ messages in thread From: Dave Taht @ 2017-09-21 2:06 UTC (permalink / raw) To: Aaron Wood; +Cc: bloat [-- Attachment #1: Type: text/plain, Size: 214 bytes --] Well, at these higher speeds the distance to the server and the quality of the server and its cloud connection start to matter. I'm assuming you are using flent-fremont which should only be 10-12 ms away from you? [-- Attachment #2: Type: text/html, Size: 243 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-08-30 5:15 [Bloat] different speeds on different ports? (benchmarking fun) Aaron Wood 2017-09-21 2:06 ` Dave Taht @ 2017-09-21 2:34 ` Dave Taht 2017-09-21 10:58 ` Stefan Alfredsson ` (2 subsequent siblings) 4 siblings, 0 replies; 11+ messages in thread From: Dave Taht @ 2017-09-21 2:34 UTC (permalink / raw) To: Aaron Wood; +Cc: bloat A packet capture from dslreports might be useful also. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-08-30 5:15 [Bloat] different speeds on different ports? (benchmarking fun) Aaron Wood 2017-09-21 2:06 ` Dave Taht 2017-09-21 2:34 ` Dave Taht @ 2017-09-21 10:58 ` Stefan Alfredsson 2017-09-21 11:16 ` Toke Høiland-Jørgensen 2017-09-21 14:50 ` Colin Dearborn 4 siblings, 0 replies; 11+ messages in thread From: Stefan Alfredsson @ 2017-09-21 10:58 UTC (permalink / raw) To: bloat On 2017-08-30 07:15, Aaron Wood wrote: > > The current hypothesis that we have is that this is due to either > traffic class, or the ports that traffic are running on. I've ruled > out the ping streams, as a parallel set of netperf tcp_maerts > downloads has the same 120Mbps roof. Also think of server differentiation (maybe full capacity to known measurement servers, shaped for other destinations). Does dslreports have flent/netperf responders on their servers as well? > > It would be interesting if we could run some netperf tests using port > 80/443 for the listening socket for the data connection (although if > doing deep-packet inspection, we might need to use an actual HTTP > transfer). UDP 443 (QUIC) would also be interesting for comparison. /Stefan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-08-30 5:15 [Bloat] different speeds on different ports? (benchmarking fun) Aaron Wood ` (2 preceding siblings ...) 2017-09-21 10:58 ` Stefan Alfredsson @ 2017-09-21 11:16 ` Toke Høiland-Jørgensen 2017-09-21 12:51 ` Sebastian Moeller 2017-09-22 0:26 ` David Lang 2017-09-21 14:50 ` Colin Dearborn 4 siblings, 2 replies; 11+ messages in thread From: Toke Høiland-Jørgensen @ 2017-09-21 11:16 UTC (permalink / raw) To: Aaron Wood, bloat > It would be interesting if we could run some netperf tests using port > 80/443 for the listening socket for the data connection (although if > doing deep-packet inspection, we might need to use an actual HTTP > transfer). Trouble with this is that netserver would have to run as root to be able to bind to the ports; it does that dynamically, so binding at startup and dropping privileges won't work. Also it would mean that no other services could run on those ports. For your use case, my guess would also be differences in the server connection (or peering), though. I do believe dslreports will open connections to multiple servers; you could try running one of the rtt_fair Flent tests against multiple servers to see if you can push more data that way... -Toke ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-09-21 11:16 ` Toke Høiland-Jørgensen @ 2017-09-21 12:51 ` Sebastian Moeller 2017-09-22 0:26 ` David Lang 1 sibling, 0 replies; 11+ messages in thread From: Sebastian Moeller @ 2017-09-21 12:51 UTC (permalink / raw) To: Toke Høiland-Jørgensen; +Cc: Aaron Wood, bloat Hi Toke, Stefan, Aaron, > On Sep 21, 2017, at 13:16, Toke Høiland-Jørgensen <toke@toke.dk> wrote: > >> It would be interesting if we could run some netperf tests using port >> 80/443 for the listening socket for the data connection (although if >> doing deep-packet inspection, we might need to use an actual HTTP >> transfer). > > Trouble with this is that netserver would have to run as root to be able > to bind to the ports; it does that dynamically, so binding at startup > and dropping privileges won't work. Also it would mean that no other > services could run on those ports. > > For your use case, my guess would also be differences in the server > connection (or peering), though. I do believe dslreports will open > connections to multiple servers; you could try running one of the > rtt_fair Flent tests against multiple servers to see if you can push > more data that way... After a free registration you can actually select the servers you want to test against as well as the number of concurrent streams. There even is a command line version of the dslreports speedtest (https://www.dslreports.com/forum/speedtestbinary) which offers the following options: bash-3.2$ ./dslrcli-darwin-amd64 --help NAME: Dslrcli - Test network speed USAGE: dslrcli-darwin-amd64 [global options] command [command options] [arguments...] VERSION: 0.1 - 13-Dec-2016 AUTHOR(S): DSLReports.com <www.dslreports.com> COMMANDS: daemonmode, W run in daemonmode help, h Shows a list of commands or help for one command GLOBAL OPTIONS: --debug, -d --printlog, -l Prints the log on the screen. --uname value Your DslReports.com username - OPTIONAL. --upassword value Your DslReports.com password - OPTIONAL. --machinename value A machine name for the daemon mode. (default: "machine1") --apikey value Your DslReports.com API Key - OPTIONAL. --up value Number of streams to use for upload tests (default: "4") --uploadtime value Time limit to end the upload test in. (default: "15") --down value Number of streams to use for download tests. (default: "6") --hz value The frequency (in hz) at which speed samples are to be taken. (default: 1) --downloadtime value Time limit to end the download test in. (default: "15") --perserver value The number of streams per server. (default: "3") --totalstreams value The total number of streams to use for the speedtests - OPTIONAL (default: "X") --comment value, -c value Include comment with test result. --output value, -o value Specify type of output . 'json' and 'csv' are currently supported. (default: "default") --upload-only Performs the Upload test only. --download-only Performs the Download test only. --daemon Runs the DslReports client in daemon mode. --daemon-cmd value Daemon sub command install, uninstall. --authfile value A json file containing username and password for authentication. --platform value Specify a platform (GPRS, 3G, 4G, WiFi, Wireless, Satellite, DSL, Cable, Fiber, Unsure). (default: "Unsure") --latlong value Specify a latitude longitude of the region to select servers from. In the form -latlong a,b --ipv4 IPv4 --ipv6 IPv6 --help, -h show help --version, -v print the version So you can specify the number of servers (--totalstreams MM; totalstreams/perserver shoukld equal number of servers) as well as the number of flows per server (--perserver NN). Anyway, no back to the regularly scheduled programming... Best Regards Sebastian > > -Toke > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-09-21 11:16 ` Toke Høiland-Jørgensen 2017-09-21 12:51 ` Sebastian Moeller @ 2017-09-22 0:26 ` David Lang 1 sibling, 0 replies; 11+ messages in thread From: David Lang @ 2017-09-22 0:26 UTC (permalink / raw) To: Toke Høiland-Jørgensen; +Cc: Aaron Wood, bloat [-- Attachment #1: Type: TEXT/PLAIN, Size: 710 bytes --] On Thu, 21 Sep 2017, Toke Høiland-Jørgensen wrote: >> It would be interesting if we could run some netperf tests using port >> 80/443 for the listening socket for the data connection (although if >> doing deep-packet inspection, we might need to use an actual HTTP >> transfer). > > Trouble with this is that netserver would have to run as root to be able > to bind to the ports; it does that dynamically, so binding at startup > and dropping privileges won't work. you can also give the binary the capability to bind to low ports without running as root. setcap 'cap_net_bind_service=+ep' /path/to/program > Also it would mean that no other services could run on those ports. that is true. David Lang ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-08-30 5:15 [Bloat] different speeds on different ports? (benchmarking fun) Aaron Wood ` (3 preceding siblings ...) 2017-09-21 11:16 ` Toke Høiland-Jørgensen @ 2017-09-21 14:50 ` Colin Dearborn 2017-09-21 15:13 ` Aaron Wood 4 siblings, 1 reply; 11+ messages in thread From: Colin Dearborn @ 2017-09-21 14:50 UTC (permalink / raw) To: Aaron Wood, bloat [-- Attachment #1.1: Type: text/plain, Size: 1817 bytes --] This is my guess. DSL reports uses many streams from different servers to achieve these speeds. I’m assuming flent is a single stream, so you’re at the mercy of TCP receive windows and latency limiting how fast you can go on that single stream. From: Bloat [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Aaron Wood Sent: Tuesday, August 29, 2017 11:16 PM To: bloat <bloat@lists.bufferbloat.net> Subject: [Bloat] different speeds on different ports? (benchmarking fun) I don't have a full writeup yet, but wanted to ask if people on here have run into this. I'm seeing a disparity between flent and the dslreports speed tests. On my connection at home (Comcast 150/12), I figured it was something related to the test implementations, but minor. But on a connect at a friend with business-class Comcast (300/12), we're seeing a huge difference. Flent can't seem to achieve more than 120Mbps, often with an early, couple-second hump at a much higher speed. But dslreports' speed tests gets the full 300Mbps. In looking closer at my connection, with sqm (cake) turned off, I'm seeing ~180Mbps download with 500ms of bufferbloat when I use the dslreports test (http://www.dslreports.com/speedtest/20805152). Yet flent can't come close to that, even with the tcp_12down test: [cid:image002.png@01D332B6.95FED960] The current hypothesis that we have is that this is due to either traffic class, or the ports that traffic are running on. I've ruled out the ping streams, as a parallel set of netperf tcp_maerts downloads has the same 120Mbps roof. It would be interesting if we could run some netperf tests using port 80/443 for the listening socket for the data connection (although if doing deep-packet inspection, we might need to use an actual HTTP transfer). -Aaron [-- Attachment #1.2: Type: text/html, Size: 5546 bytes --] [-- Attachment #2: image002.png --] [-- Type: image/png, Size: 38956 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-09-21 14:50 ` Colin Dearborn @ 2017-09-21 15:13 ` Aaron Wood 2017-09-21 15:16 ` Aaron Wood 0 siblings, 1 reply; 11+ messages in thread From: Aaron Wood @ 2017-09-21 15:13 UTC (permalink / raw) To: Colin Dearborn; +Cc: bloat [-- Attachment #1.1: Type: text/plain, Size: 2205 bytes --] I'd wondered about single vs. multiple, but I'm getting pretty consistent speeds from the flent-fremont node irrespective of the number of streams that I use (1, 4, 12, etc). On Thu, Sep 21, 2017 at 7:50 AM, Colin Dearborn <Colin.Dearborn@sjrb.ca> wrote: > This is my guess. > > DSL reports uses many streams from different servers to achieve these > speeds. > > I’m assuming flent is a single stream, so you’re at the mercy of TCP > receive windows and latency limiting how fast you can go on that single > stream. > > > > *From:* Bloat [mailto:bloat-bounces@lists.bufferbloat.net] *On Behalf Of *Aaron > Wood > *Sent:* Tuesday, August 29, 2017 11:16 PM > *To:* bloat <bloat@lists.bufferbloat.net> > *Subject:* [Bloat] different speeds on different ports? (benchmarking fun) > > > > I don't have a full writeup yet, but wanted to ask if people on here have > run into this. > > > > I'm seeing a disparity between flent and the dslreports speed tests. On > my connection at home (Comcast 150/12), I figured it was something related > to the test implementations, but minor. But on a connect at a friend with > business-class Comcast (300/12), we're seeing a huge difference. Flent > can't seem to achieve more than 120Mbps, often with an early, couple-second > hump at a much higher speed. But dslreports' speed tests gets the full > 300Mbps. > > > > In looking closer at my connection, with sqm (cake) turned off, I'm seeing > ~180Mbps download with 500ms of bufferbloat when I use the dslreports test ( > http://www.dslreports.com/speedtest/20805152). > > > > Yet flent can't come close to that, even with the tcp_12down test: > > > > > The current hypothesis that we have is that this is due to either traffic > class, or the ports that traffic are running on. I've ruled out the ping > streams, as a parallel set of netperf tcp_maerts downloads has the same > 120Mbps roof. > > > > It would be interesting if we could run some netperf tests using port > 80/443 for the listening socket for the data connection (although if doing > deep-packet inspection, we might need to use an actual HTTP transfer). > > > > -Aaron > [-- Attachment #1.2: Type: text/html, Size: 4424 bytes --] [-- Attachment #2: image002.png --] [-- Type: image/png, Size: 38956 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-09-21 15:13 ` Aaron Wood @ 2017-09-21 15:16 ` Aaron Wood 2017-09-23 2:51 ` Aaron Wood 0 siblings, 1 reply; 11+ messages in thread From: Aaron Wood @ 2017-09-21 15:16 UTC (permalink / raw) To: Colin Dearborn; +Cc: bloat [-- Attachment #1.1: Type: text/plain, Size: 2825 bytes --] The friend of mine that I've been working with brought up a cloud node somewhere with ubuntu and netperf on it, and from another location (business internet) able to consistently get better throughput from his cloud node setup than from the flent-fremont node. We're starting to think that it's something about that node in particular. It seems to have a 125Mbps cap (so I guess about a 140-150Mbps line-rate cap?). What kind of node is it running on? On Thu, Sep 21, 2017 at 8:13 AM, Aaron Wood <woody77@gmail.com> wrote: > I'd wondered about single vs. multiple, but I'm getting pretty consistent > speeds from the flent-fremont node irrespective of the number of streams > that I use (1, 4, 12, etc). > > On Thu, Sep 21, 2017 at 7:50 AM, Colin Dearborn <Colin.Dearborn@sjrb.ca> > wrote: > >> This is my guess. >> >> DSL reports uses many streams from different servers to achieve these >> speeds. >> >> I’m assuming flent is a single stream, so you’re at the mercy of TCP >> receive windows and latency limiting how fast you can go on that single >> stream. >> >> >> >> *From:* Bloat [mailto:bloat-bounces@lists.bufferbloat.net] *On Behalf Of >> *Aaron Wood >> *Sent:* Tuesday, August 29, 2017 11:16 PM >> *To:* bloat <bloat@lists.bufferbloat.net> >> *Subject:* [Bloat] different speeds on different ports? (benchmarking >> fun) >> >> >> >> I don't have a full writeup yet, but wanted to ask if people on here have >> run into this. >> >> >> >> I'm seeing a disparity between flent and the dslreports speed tests. On >> my connection at home (Comcast 150/12), I figured it was something related >> to the test implementations, but minor. But on a connect at a friend with >> business-class Comcast (300/12), we're seeing a huge difference. Flent >> can't seem to achieve more than 120Mbps, often with an early, couple-second >> hump at a much higher speed. But dslreports' speed tests gets the full >> 300Mbps. >> >> >> >> In looking closer at my connection, with sqm (cake) turned off, I'm >> seeing ~180Mbps download with 500ms of bufferbloat when I use the >> dslreports test (http://www.dslreports.com/speedtest/20805152). >> >> >> >> Yet flent can't come close to that, even with the tcp_12down test: >> >> >> >> >> The current hypothesis that we have is that this is due to either traffic >> class, or the ports that traffic are running on. I've ruled out the ping >> streams, as a parallel set of netperf tcp_maerts downloads has the same >> 120Mbps roof. >> >> >> >> It would be interesting if we could run some netperf tests using port >> 80/443 for the listening socket for the data connection (although if doing >> deep-packet inspection, we might need to use an actual HTTP transfer). >> >> >> >> -Aaron >> > > [-- Attachment #1.2: Type: text/html, Size: 5396 bytes --] [-- Attachment #2: image002.png --] [-- Type: image/png, Size: 38956 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Bloat] different speeds on different ports? (benchmarking fun) 2017-09-21 15:16 ` Aaron Wood @ 2017-09-23 2:51 ` Aaron Wood 0 siblings, 0 replies; 11+ messages in thread From: Aaron Wood @ 2017-09-23 2:51 UTC (permalink / raw) Cc: bloat [-- Attachment #1.1: Type: text/plain, Size: 3282 bytes --] I dug out the info from the friend of mine doing this testing, and he was about to get the dslreports_8dn test to deliver ~800Mbps from his server, but only ~122Mbps from flent-freemont.bufferbloat.net. Which is awfully suspicious that this is a setup issue with the node in fremont. On Thu, Sep 21, 2017 at 8:16 AM, Aaron Wood <woody77@gmail.com> wrote: > The friend of mine that I've been working with brought up a cloud node > somewhere with ubuntu and netperf on it, and from another location > (business internet) able to consistently get better throughput from his > cloud node setup than from the flent-fremont node. We're starting to think > that it's something about that node in particular. It seems to have a > 125Mbps cap (so I guess about a 140-150Mbps line-rate cap?). > > What kind of node is it running on? > > On Thu, Sep 21, 2017 at 8:13 AM, Aaron Wood <woody77@gmail.com> wrote: > >> I'd wondered about single vs. multiple, but I'm getting pretty consistent >> speeds from the flent-fremont node irrespective of the number of streams >> that I use (1, 4, 12, etc). >> >> On Thu, Sep 21, 2017 at 7:50 AM, Colin Dearborn <Colin.Dearborn@sjrb.ca> >> wrote: >> >>> This is my guess. >>> >>> DSL reports uses many streams from different servers to achieve these >>> speeds. >>> >>> I’m assuming flent is a single stream, so you’re at the mercy of TCP >>> receive windows and latency limiting how fast you can go on that single >>> stream. >>> >>> >>> >>> *From:* Bloat [mailto:bloat-bounces@lists.bufferbloat.net] *On Behalf >>> Of *Aaron Wood >>> *Sent:* Tuesday, August 29, 2017 11:16 PM >>> *To:* bloat <bloat@lists.bufferbloat.net> >>> *Subject:* [Bloat] different speeds on different ports? (benchmarking >>> fun) >>> >>> >>> >>> I don't have a full writeup yet, but wanted to ask if people on here >>> have run into this. >>> >>> >>> >>> I'm seeing a disparity between flent and the dslreports speed tests. On >>> my connection at home (Comcast 150/12), I figured it was something related >>> to the test implementations, but minor. But on a connect at a friend with >>> business-class Comcast (300/12), we're seeing a huge difference. Flent >>> can't seem to achieve more than 120Mbps, often with an early, couple-second >>> hump at a much higher speed. But dslreports' speed tests gets the full >>> 300Mbps. >>> >>> >>> >>> In looking closer at my connection, with sqm (cake) turned off, I'm >>> seeing ~180Mbps download with 500ms of bufferbloat when I use the >>> dslreports test (http://www.dslreports.com/speedtest/20805152). >>> >>> >>> >>> Yet flent can't come close to that, even with the tcp_12down test: >>> >>> >>> >>> >>> The current hypothesis that we have is that this is due to either >>> traffic class, or the ports that traffic are running on. I've ruled out >>> the ping streams, as a parallel set of netperf tcp_maerts downloads has the >>> same 120Mbps roof. >>> >>> >>> >>> It would be interesting if we could run some netperf tests using port >>> 80/443 for the listening socket for the data connection (although if doing >>> deep-packet inspection, we might need to use an actual HTTP transfer). >>> >>> >>> >>> -Aaron >>> >> >> > [-- Attachment #1.2: Type: text/html, Size: 6257 bytes --] [-- Attachment #2: image002.png --] [-- Type: image/png, Size: 38956 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2017-09-23 2:51 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-08-30 5:15 [Bloat] different speeds on different ports? (benchmarking fun) Aaron Wood 2017-09-21 2:06 ` Dave Taht 2017-09-21 2:34 ` Dave Taht 2017-09-21 10:58 ` Stefan Alfredsson 2017-09-21 11:16 ` Toke Høiland-Jørgensen 2017-09-21 12:51 ` Sebastian Moeller 2017-09-22 0:26 ` David Lang 2017-09-21 14:50 ` Colin Dearborn 2017-09-21 15:13 ` Aaron Wood 2017-09-21 15:16 ` Aaron Wood 2017-09-23 2:51 ` Aaron Wood
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox