From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id C3E5921F19C for ; Sat, 11 Jan 2014 14:15:29 -0800 (PST) Received: from hms-beagle-3.home.lan ([217.254.130.56]) by mail.gmx.com (mrgmx102) with ESMTPSA (Nemesis) id 0LfHs4-1VZQyV0Twu-00on7y for ; Sat, 11 Jan 2014 23:15:27 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) From: Sebastian Moeller In-Reply-To: Date: Sat, 11 Jan 2014 23:15:26 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <4720BEA3-977F-43CF-BB49-74A212FB6127@gmx.de> References: <4B0976EC-5224-4E81-B5DF-96A85424FB6E@gmail.com> <49DBDBE7-BF76-4823-B723-670935A55833@gmx.de> <10B5A5C1-A4D7-429D-A57D-21612B5169E1@gmail.com> To: Aaron Wood X-Mailer: Apple Mail (2.1510) X-Provags-ID: V03:K0:+RGQfXnOeDy+V2i9xyTAAw6e00v4O4K1+jSyBdGl4gA2gO6uUc9 3meZ1bfuU/h4Elrw026AluPSzyEbco1hD04M/64YBCDASvZoTipQ/QZS7K9ow2F2WRvX9Ap 5mkjNDG9i3ccqCb21oCPNOr7u/EhYbvqzITpqqPBSBpTUCrawYjIx1ND9W+lwKcOB6jlYgA NCoH+xzAlpQkFcf/wSVIQ== Cc: cerowrt-devel Subject: Re: [Cerowrt-devel] VDSL X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jan 2014 22:15:30 -0000 Hi Aaron, On Jan 11, 2014, at 21:23 , Aaron Wood wrote: > Rich, Sebastian, (and others), >=20 > First, a hello. I've been lurking on the bufferbloat mailing list for = a bit, and just joined here as well to better follow what's going on, = and see if there's any way I can help. >=20 > Next, I have an ADSL+ link in Paris (Free.fr), and am willing to run a = number of tests using the various LLA options and overhead estimations. = But 3 hours on a dead-quiet link could be hard to deal with. =20 Well, it takes 3 hours to collect 10000 samples for 100 = different packet sizes (creating a maximum transient traffic of < = 150kbit/s), you could collect less (your RTT will be dominated by your = uplink, so 2000 to 3000 samples per size will be plenty, reducing the = time to something more manageable). And as long as your link has 150 = kbit/sec reserved on the uplink (since it is more critical there) you = should be fine. Unless you saturate your link, additional traffic to the = probe will increase the variance of the ping times, but you might be = able to get away with, just try to measure for as long as you are = comfortable with. (I typically just run this over night=85 and there is = not much going on in our network at that time, but it is not dead-quiet = either). > I'm happy to run an hour's worth of netperf tests to a nearby server, = slowly working through the parameter space, and then comparing the = results.\ Again, I recommend to perform the ATM quantization probe against = the nearest host (already in the ISPs network, ideally the DSLAM/MSAN) = to reduce the variance in the data... >=20 > My ad-hoc comparisons last week with various modes showed that too = high of settings for the overhead (coupled with the already reduced bw = limit in the shaper) killed the bulk upload/download performance (which = I care about on a meager 18Mbps/1Mbps link). =20 Yes, as expected you pay a bandwidth price for getting the = overhead too large. The price you pay for specifying the overhead to low = is that for some packet sizes there is no difference while some other = packet sizes will drag in an additional ATM cell of 43 bytes. A larger = packet sizes typical for bulk upload this additional 43 bytes will not = really kill your through put, for small packets this can be = catastrophic. Think your specify verhead 0 no link layer adjustments and = send a 50 byte packet, the shaper thinks this takes 64 bytes, but on an = ATM link with PPPoE LLC/SNAP you actually have 40 bytes overhead + 64 = byte packet =3D 104 bytes which require 3 ATM cells of 48 byte totaling = 3*48 =3D 144 bytes on the wire, your packet more than doubled (factor = 2.25). And if you specify an overhead that is too small, you still get = some packet sizes that drag in an additional cell with the same = consequences. But note that these effects depend on the size of the packets, = which people typically do not vary during bandwidth tests... > But I found that setting the bw shaper limit to the reported line = speed (from the modem), and then adjusting the overhead parameters got = me the same bulk performances, with the same latencies (or what appeared = to be the same, I need to do more data crunching on the results, and run = again in a quieter setting). As I tried to indicate above for bulk traffic the effect of 47 = bytes in 1500 might be lost in the noise (and 47 bytes per packet is the = worst case), think roughly 3 percent worst case link capacity = overestimation (and if you shape to 95% this already fits into the = shaped margin). >=20 > It would also help if my target server was closer than it is. The = only server I know of is in Germany, 55ms away (unloaded). I think you should measure the per racket overhead where it = actually matters, on the link to the DSLAM/ISP, any congestion on shared = network segments further away from the router cannot really be = controlled well by shaping in the router, so let's not try to fix that = part of the route :) . >=20 > OTOH, I can say that any changes here over the defaults are still = gilding the lily (to an end user). =20 I think I need to come up with a worst case ATM carrier effects = test, so people can feel the pain quicker. The dang dependence on packet = size really is unfortunate as no bulk speed tester I know tries a decent = set of packet sizes... > But Free.fr's router/modem already uses codel, so it wasn't that bad = to begin with (vs. Numericable on a docsis 3 Netgear router/modem). If I understand correctly, at least the newer free boxes should = be very hard to beat, can you disable the modems AQM for testing? >=20 > =3D=3D=3D >=20 > I can also say that I found the current verbiage on that particular = page a bit "clear as mud". =20 Which page? > Even knowing what my network is (to some degree, since the Free.fr = modem can tell me), it was difficult to follow, and I quickly found = myself at about 3/4 my previous speeds, with no visible improvement in = latency (although that could have been a measurement tool issue, as I = was doing 60-second runs using the rrul netperf loads). So codel/fq_codel typically try to keep the latency increase = bound to 5ms (per leg) so the ping times should stay nicely somewhere = between 55ms and 65ms, if the rrul ping trace stays in that band you are = already golden; instead of reducing the shaped bandwidth I would start = increasing it again, until you notice the latencies increase again. Note = for RRUL I typically aim for 300 seconds (I think Dave once recommended = that) to get over all transient effects. >=20 > The main question I have is: >=20 > - Should we both limit the bandwidth well below the reported line rate = (or measured IP rate) AND use the link layer adaption settings? (85-90% = of bandwidth) > or > - rely on the LLA settings to do the overhead, and shape to just a = tiny bit under the reported line rate? (95-99% of bandwidth) I wou;d go for the second option. But note while you have full = control over your uplink, on the downlink there might be not expected = packets arriving (think you connect to your network via VPN) so the = current recommendation is to shape the downlink a bit more aggressively = than the uplink. Currently I use 95% on uplink and 90% on downlink (I = have not yet found the time to see how far up I can push these, but = latency is generally quite pleasent). Best Regards Sebastian >=20 > Thanks, >=20 > - Aaron Wood >=20 >=20 > On Sat, Jan 11, 2014 at 8:06 PM, Rich Brown = wrote: > Hi Sebastian, >=20 > >>> Well, that looks like a decent recommendation for the wiki. = The SQM configuration page still needs to expose all three values, atm, = ethernet, and none so that people can actually change things... > >> > >> So two questions really: > >> > >> 1) (=46rom my previous note) What=92s the difference between the = current =93Ethernet=94 (for VDSL) and =93None=94 link layer adaptations? > > > > Currently, "none" completely disables the link layer = adjustments, "ethernet" enables them, but will only use the overhead = (unless you specify an tcMPU, but that is truly exotic). > > > >> > >> 2) When we distinguish the Ethernet/VDSL case, I would really like = to use a different name from =93Ethernet=94 because it seems confusingly = similar to having a real Ethernet path/link (e.g., direct connection to = internet backbone without any ADSL, cable modem, etc.) > > > > On the one hand I agree, but the two options are called "ATM" = (well for tc "adsl" is a valid alias for ATM) and "ethernet" if you pass = them to tc (what we do), and I would really hate it to hide this under = fancy names. I see no chance of renaming those options in tc, so we are = sort of stuck with them and adding another layer of indirection seems = too opaque to me. This is why I put some explanation behind the option = names in the list box=85 >=20 > Now I see how it works. (I didn=92t understand that =93None=94 really = meant NONE.) The following choices in the Link Layer Adaptation would = have eased my confusion: >=20 > - ATM (almost every type of ADSL or DSL) > - Ethernet with overhead > - None (default) >=20 > Then the text can say: >=20 > =97 > You must set the Link Layer Adaptation options so that CeroWrt can = perform its best with video and audio chat, gaming, and other protocols = that rely on short packets. The general rule for selecting the Link = Layer Adaption is: >=20 > * If you use any kind of DSL/ADSL connection to the Internet (that is, = if you get your internet service through the telephone line), you should = choose the =93ATM (almost every type of ADSL or DSL)" item. Set the = Per-packet Overhead to 44. >=20 > * If you know you have a VDSL connection, you should choose =93Ethernet = with overhead" and set the Per-packet Overhead to 8. >=20 > * If you use Cable modem, Fiber, or another kind of connection to the = Internet, you should choose =93None (default)=94. All other parameters = will be ignored. >=20 > If you cannot tell what kind of link you have, first try using "None", = then run the [[Quick Test for Bufferbloat]]. If the results are good, = you=92re done. Next, try the ADSL/ATM choice, then the VDSL choice to = see which performs best. Read the **Details** (below) to learn more = about tuning the parameters for your link. > =97 >=20 > Would that be better? Thanks. >=20 > Rich > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel >=20