From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AAF493BA8D; Thu, 9 Feb 2017 03:35:04 -0500 (EST) Received: by mail-wm0-x242.google.com with SMTP id v77so1712267wmv.0; Thu, 09 Feb 2017 00:35:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=RxyodILiYbquQSqshNktWshfn6ifurXYdairwxRj9Ms=; b=Fma4V8japaqckiOpaDnvX4XIkEemyp76MpGZZXpCojc2pW5JD9DBXv/h8b8k5SvT/b 8JynGLQKVRhv05gpMpOkqoIdatxpynw4KbXMazzDSNTAEtL+r9t2pxjoaySIyImkKaYM Kw7LXYErVD+3CRxKI++8yUXx8M6yW+dZeGkVxWlob3hW42iSuKKOxwCWp0AYnlsffNBF 8Wp8PgZyy14iZEY/LmTDRxXmyaotnI92R0hmX85gg/I5zsgRK5HfkhrCkSyB1C2RX9hL 6NWKvbxxrrVmOBfIjN8VvXbS3eHL0wXD6Z8DcgonJGb3TYNGmzLbZWvHf3vIHUzZ8poC RLDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=RxyodILiYbquQSqshNktWshfn6ifurXYdairwxRj9Ms=; b=pP/Up/55fV/UY7aicCydJxkN9sCc5kV3Ue/VdGfbVvemzv916VA3SKMYANFHpUD9Jr IkEajpe3tO5pwcfnN/Bs6aghY56wcpfEiDxxzh96Wh8ZoMxlVM7lyBBkPtoJZbIsayOo goFW6Xh02WdfZLQIHJpzqIuWcWgj9X4gueZpTqQAJldoCq1hGYaIWSKKuIlB+SD4i66L 16B5TR44CYr9jI0o8o/eOSQcgshncApB3rdBT9OM3iPWU8MRnfielFq0smHcWWSNf2gx neYMQTbGX5km20PyzNxhEDMnhwR/btUl3DSs5ok+L4nKfGuTPSjv6F2GITcD4UKPnVe1 Kbow== X-Gm-Message-State: AMke39kwh7bm3exohxBX+l5o4a9kM2+yamgmUTVuuVC5cM0N1NDHUoMHoR7fUwaswtu/XQ== X-Received: by 10.28.148.76 with SMTP id w73mr21563892wmd.74.1486629303244; Thu, 09 Feb 2017 00:35:03 -0800 (PST) Received: from [10.72.0.34] (h-1169.lbcfree.net. [185.99.119.68]) by smtp.gmail.com with ESMTPSA id i189sm7605397wmg.7.2017.02.09.00.35.01 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 09 Feb 2017 00:35:02 -0800 (PST) Content-Type: multipart/alternative; boundary="Apple-Mail=_054DEAC2-F116-4FAF-93D2-6F4F5B79C4C4" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Pete Heist In-Reply-To: Date: Thu, 9 Feb 2017 09:35:02 +0100 Cc: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Cake List , make-wifi-fast@lists.bufferbloat.net Message-Id: <35E9BB45-75D4-43CB-ACB3-C42AF1F38EA1@gmail.com> References: <32C42951-373F-4D90-8936-AA07039E5D73@gmail.com> <877f5c2pew.fsf@toke.dk> <878tpqge5g.fsf@toke.dk> <877f52rz68.fsf@toke.dk> <87bmucu0gs.fsf@toke.dk> To: Dave Taht X-Mailer: Apple Mail (2.3124) Subject: Re: [Make-wifi-fast] Flent results for point-to-point Wi-Fi on LEDE/OM2P-HS available X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2017 08:35:05 -0000 --Apple-Mail=_054DEAC2-F116-4FAF-93D2-6F4F5B79C4C4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Feb 8, 2017, at 5:35 PM, Dave Taht wrote: >=20 > On Wed, Feb 8, 2017 at 8:11 AM, Toke H=C3=B8iland-J=C3=B8rgensen = > wrote: >>=20 >> It's not so much running the test *from* the router, as it is having = the >> server component (netserver) run on a router to test. To do that = it'll >> have to be C, basically... >=20 > I note that I usually run tests *through* the router rather than *to* > the router, and most of my targets for this have evolved to be arm > (odroid c2), and x86 platforms, so I could get past a gbit. Me too so far. The OM2P-HS (520 MHz MIPS 74Kc, similar to the NSM5 I=E2=80= =99m about to test), was unable to max out the link rate even with = iperf3, so I knew I had to run flent on separate hardware. Right now I = have a 4 node config: flent-client/router =E2=80=94 station =E2=80=94 AP =E2=80=94 = router/flent-server When I test the Alix router, which is pretty low end, I=E2=80=99ll go to = a 6 node config because I don=E2=80=99t think it will bear running flent = while routing and managing the queue: flent-client =E2=80=94 router =E2=80=94 station =E2=80=94 AP =E2=80=94 = router =E2=80=94 flent-server It seems that flent should be run on sufficiently overpowered hardware = (basically any even semi-modern Intel hardware) relative to what=E2=80=99s= being tested. Maybe there are cases where it needs to run on embedded = devices that I=E2=80=99m not thinking of. > Of note I've always regarded exposing d-itg and some sort of monotonic > voip-like test to the internet as too dangerous without a solid 3 way > handshake, and leveraging existing voip and webrtc/videoconferencing > servers (like freeswitch) way too hard to setup and measure. (there > are plenty of tools to generate rtp-like packets). I haven=E2=80=99t gotten d-itg to run so far. I haven=E2=80=99t looked = into it much, but for some reason it seems like ITGRecv only wants to = listen for signaling (port 9000) on tcp6, even with an explicit bind = address: sysadmin@mbp:~$ sudo ITGRecv -a 10.0.1.2 ITGRecv version 2.8.1 (r1023) Compile-time options: sctp dccp bursty multiport Press Ctrl-C to terminate ... sysadmin@mbp:~$ netstat -a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address = State =20 tcp 0 0 localhost:domain 0.0.0.0:* = LISTEN =20 tcp 0 0 0.0.0.0:ssh 0.0.0.0:* = LISTEN =20 tcp 0 0 mbp:ssh 10.72.0.34:60739 = ESTABLISHED tcp 0 172 mbp:ssh 10.72.0.34:60768 = ESTABLISHED tcp6 0 0 [::]:ssh [::]:* = LISTEN =20 tcp6 0 0 [::]:12865 [::]:* = LISTEN =20 tcp6 0 0 [::]:9000 [::]:* = LISTEN =20 udp 0 0 localhost:domain 0.0.0.0:* = =20 udp 0 0 0.0.0.0:bootpc 0.0.0.0:* = =20 > I am unfond of the current ping and udp tests in the rrul component > because they don't look like these traffic types, and drawing > conclusions from their behavior as if they were is wrong. Also, with > very low RTTs, the measurement flows tend to skew the tests - you'll > consistently see "lower bandwidth" measured when you cut an RTT from > 100 to 1ms via active queue management, when what is really happening > is 100x more measurement traffic. I=E2=80=99ve been wondering about this- how seriously to take the rtt = from rrul tests in general. When I=E2=80=99m making config changes that = change the rtt by a few milliseconds here or there, I=E2=80=99m not = convinced I=E2=80=99m not missing other externalities, which are so far = are probably only accessible by reading the tea leaves. So for example, = when sfq and fq_codel seem quite close in my half duplex rate limiting = results, at least in terms of average rtt, are they really? Aren't there = test cases where I can better demonstrate that fq_codel is =E2=80=9Cbetter= =E2=80=9D than sfq? Ultimately it will come down to how well fq_codel (or Cake) behaves in = the real world, and I=E2=80=99m not even sure how to measure that. In = FreeNet, I can get regular ICMP ping results between their routers and = look at them after a day=E2=80=99s time, for example, but will I really = have everything I need to know I=E2=80=99ve made a difference? And = furthermore, if Ubiquiti is prioritizing ICMP (I intend to find that = out), are those ICMP results even useful? I may have to measure RTT with = regular best-effort UDP packets between the routers, and even then, I = want to know when I see latency spikes whether I=E2=80=99m looking at = bufferbloat or radio related issues, so I=E2=80=99ll need other stats as = well. They have Ubiquiti=E2=80=99s airControl deployed, but my sense is = that it=E2=80=99s going to be easy to deploy fq_codel, and not very easy = to demonstrate if and how it=E2=80=99s helped!= --Apple-Mail=_054DEAC2-F116-4FAF-93D2-6F4F5B79C4C4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On Feb 8, 2017, at 5:35 PM, Dave Taht <dave.taht@gmail.com>= wrote:

On Wed, Feb 8, 2017 at 8:11 AM, Toke = H=C3=B8iland-J=C3=B8rgensen <toke@toke.dk> wrote:

It's not so much running the test *from* = the router, as it is having the
server component = (netserver) run on a router to test. To do that it'll
have = to be C, basically...

I note that I usually run tests *through* = the router rather than *to*
the router, and most of my targets for = this have evolved to be arm
(odroid c2), and x86 platforms, so I = could get past a gbit.

Me too so = far. The OM2P-HS (520 MHz MIPS 74Kc, similar to the NSM5 I=E2=80=99m = about to test), was unable to max out the link rate even with iperf3, so = I knew I had to run flent on separate hardware. Right now I have a 4 = node config:

flent-client/router =E2=80= =94 station =E2=80=94 AP =E2=80=94 router/flent-server

When I test the Alix router, which is pretty low = end, I=E2=80=99ll go to a 6 node config because I don=E2=80=99t think it = will bear running flent while routing and managing the = queue:

flent-client =E2=80=94 router = =E2=80=94 station =E2=80=94 AP =E2=80=94 router =E2=80=94 = flent-server

It seems that flent = should be run on sufficiently overpowered hardware (basically any even = semi-modern Intel hardware) relative to what=E2=80=99s being tested. = Maybe there are cases where it needs to run on embedded devices that = I=E2=80=99m not thinking of.

Of note I've always regarded exposing = d-itg and some sort of monotonic
voip-like test to the internet as too = dangerous without a solid 3 way
handshake, and leveraging existing voip = and webrtc/videoconferencing
servers (like freeswitch) way too hard to = setup and measure. (there
are plenty of tools to generate rtp-like = packets).

I haven=E2=80=99t = gotten d-itg to run so far. I haven=E2=80=99t looked into it much, but = for some reason it seems like ITGRecv only wants to listen for signaling = (port 9000) on tcp6, even with an explicit bind address:

sysadmin@mbp:~$ sudo ITGRecv -a 10.0.1.2
ITGRecv version 2.8.1 (r1023)
Compile-time = options: sctp dccp bursty multiport
Press Ctrl-C to = terminate
...
sysadmin@mbp:~$ netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address       =     Foreign Address       =   State      
tcp  =       0      0 = localhost:domain        0.0.0.0:*   =             LISTEN    =  
tcp        0    =   0 0.0.0.0:ssh           =   0.0.0.0:*             =   LISTEN     
tcp    =     0      0 mbp:ssh   =             =   10.72.0.34:60739      =   ESTABLISHED
tcp      =   0    172 mbp:ssh       =           10.72.0.34:60768    =     ESTABLISHED
tcp6     =   0      0 [::]:ssh      =           [::]:*      =             LISTEN    =  
tcp6       0    =   0 [::]:12865            =   [::]:*              =     LISTEN     
tcp6       0    =   0 [::]:9000             =   [::]:*              =     LISTEN     
udp  =       0      0 = localhost:domain        0.0.0.0:*   =                     =    
udp        0  =     0 0.0.0.0:bootpc        =   0.0.0.0:*             =              

I am unfond of the current ping and udp tests in = the rrul component
because they don't look like these traffic = types, and drawing
conclusions from their behavior as if they were = is wrong. Also, with
very low RTTs, the measurement flows tend = to skew the tests - you'll
consistently see "lower bandwidth" = measured when you cut an RTT from
100 to 1ms via active queue management, = when what is really happening
is 100x more measurement = traffic.

I=E2=80=99ve = been wondering about this- how seriously to take the rtt from rrul tests = in general. When I=E2=80=99m making config changes that change the rtt = by a few milliseconds here or there, I=E2=80=99m not convinced I=E2=80=99m= not missing other externalities, which are so far are probably only = accessible by reading the tea leaves. So for example, when sfq and = fq_codel seem quite close in my half duplex rate limiting results, at = least in terms of average rtt, are they really? Aren't there test cases = where I can better demonstrate that fq_codel is =E2=80=9Cbetter=E2=80=9D = than sfq?

Ultimately it will come = down to how well fq_codel (or Cake) behaves in the real world, and I=E2=80= =99m not even sure how to measure that. In FreeNet, I can get regular = ICMP ping results between their routers and look at them after a day=E2=80= =99s time, for example, but will I really have everything I need to know = I=E2=80=99ve made a difference? And furthermore, if Ubiquiti is = prioritizing ICMP (I intend to find that out), are those ICMP results = even useful? I may have to measure RTT with regular best-effort UDP = packets between the routers, and even then, I want to know when I see = latency spikes whether I=E2=80=99m looking at bufferbloat or radio = related issues, so I=E2=80=99ll need other stats as well. They have = Ubiquiti=E2=80=99s airControl deployed, but my sense is that it=E2=80=99s = going to be easy to deploy fq_codel, and not very easy to demonstrate if = and how it=E2=80=99s helped!
= --Apple-Mail=_054DEAC2-F116-4FAF-93D2-6F4F5B79C4C4--