From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 1159121F151 for ; Mon, 6 Jan 2014 04:29:03 -0800 (PST) Received: from hms-beagle-3.home.lan ([217.254.130.56]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MTBLi-1VrwO32gic-00SAWm for ; Mon, 06 Jan 2014 13:28:53 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 6 Jan 2014 13:28:52 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <44C9514F-7E9D-4ADD-AE69-E74FBD7C2AB1@gmx.de> References: <01558084-B7D8-448A-A4ED-CE36D18AAA97@gmail.com> <9628EA9A-E5AD-4D6B-A8E3-30AFECB2FC2E@gmx.de> To: Dave Taht X-Mailer: Apple Mail (2.1510) X-Provags-ID: V03:K0:JWLUGRQxUVrwqx3bw0sro3ml10Ff7EIMlnx+3qDDqovKMa6KXK2 zgGOj/TXR0mO8pzkOtOH+Uc0Sd8j0jQ0jAw0JphDriUkIQrV+p2XVi/6iAxH+w9XXZ357tt TkssKe0nP69psVmNerFDXJXU90KuYMNuNfygu2JzXW+7vQigkJbj/xGrVAElgYR595ZsbmA msuiIg03NXRbYYtH9oNdA== Cc: cerowrt-devel Subject: Re: [Cerowrt-devel] SQM Question #5: Link Layer Adaptation Overheads X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2014 12:29:11 -0000 Hi Dave, On Jan 6, 2014, at 04:46 , Dave Taht wrote: > On Sat, Jan 4, 2014 at 12:22 PM, Sebastian Moeller = wrote: >> Hi Rich, >>=20 >> On Jan 4, 2014, at 19:16 , Rich Brown = wrote: >>=20 >>> QUESTION #5: I still don=92t have any great answers for the Link = Layer Adaptation overhead descriptions and recommendations. In an = earlier message, (see = https://lists.bufferbloat.net/pipermail/cerowrt-devel/2013-December/001914= .html and following messages), Fred Stratton described the overheads = carried by various options, and Sebastian Moeller also gave some useful = advice. >>>=20 >>> After looking at the options, I despair of giving people a clear = recommendation that would be optimal for their equipment. Consequently, = I believe the best we can do is come up with =93good enough=94 = recommendations that are not wrong, and still give decent performance. >>=20 >> Not wanting to be a spoilsport, but IMHO the issue is = complicated hence no simple recommendations. I know that my last word = was that 40bytes would be a good default overhead, but today I had the = opportunity to measure the overhead on fast ADSL connection in = Luxembourg and found that in this double-play situation (television and = internet via DSL) that an other wise invisible VLAN was further = increasing the overhead (from the 40 expected to 44 bytes). At least on = faster links these combo packets (internet, phone and potentially = telephone) are becoming more and more common, so maybe the = recommendation should be 44 (hopping that FCS are truly rare). >>=20 >>>=20 >>> In this spirit, I have changed Draft #3 of the =93Setting up SQM=94 = page to reflect this understanding. See = http://www.bufferbloat.net/projects/cerowrt/wiki/Setting_up_AQM_for_CeroWr= t_310 >>>=20 >>> ADSL/ATM link: Choose =93ADSL/ATM", and set Per Packet Overhead = to 40 >>=20 >> While I prefer ATM, I think all deployed ADSL is on ATM so = these are synonyms for our purpose. I prefer ATM since the most critical = part of the link layer adjustments is caused by the impedance mismatch = between what ATM offers and what the data transport layer requires. I = have the impression that ADSL might still evolve to a different carrier, = while ATM is basically in maintenance mode (not much new deployment if = any). >>=20 >>> VDSL2 link: Choose =93VDSL=94, and set Per Packet Overhead to 8 >>=20 >> There are several issues with this; VDSL is not the direct = predecessor of VDSL2 (rather VDSL2 is the successor of ADSL2+ with some = similarities to VDSL). Lumping VDSL with VDSL2 will require us figuring = out whether both behave the same. =46rom my cursory reading of the = standards of both I think VDSL is not unlikely to be using an ATM link = layer, VDSL2 is unlikely to do the same, both seem technically able to = use ATM. >>=20 >>=20 >>> Other kind of link (e.g., Cable, Fiber, Ethernet, other not = listed): Choose =93None (default)=94, and set Per Packet Overhead to 0 >>=20 >> This is not going to be worse than today, so sounds fine (it = would be good to know whether there is truly no overhead on these links = in practical useage). >=20 > Well, I just spent a painful hour trying to find an optimum for the > device on my mom's (cable) network, which I plan to replace with one > that is ipv6 compatible shortly. (I am settled in one place for a > while, so may setup a local dhcpv6-pd server, too - but my primary > mission is to get a test suite going) >=20 > Before: = http://snapon.lab.bufferbloat.net/~cero2/taht_mahal/taht_house_comcast_lon= g-3.svg >=20 > (note it took a long time to fully saturate the buffers as the test > box was 50ms away. I usually test with one 16ms away) >=20 > After: >=20 > = http://snapon.lab.bufferbloat.net/~cero2/taht_mahal/fq_codel_30_8_taht_hou= se_comcast-long.svg >=20 > The upload value isn't making that much sense, and I lost a lot of > throughput. Variety of other tests in that dir. >=20 >> Quick vote: anyone on this list using ceroWRT on an VDSL/VDSL2 = link or cable fiber whatnot that could do some quick testing for us? >=20 > There doesn't appear to be any need for "overhead" related settings on > cable. There might be some use on docsis 3 in changing the htb burst > value. That is what I thought before as well, nowadays I am not so = sure. Our problem is that we need to get a reliable estimate of the IP = bandwidth between modem and CTMS, so we can reliably avoiding filling = the modems buffers. The catch is that the advertised speed typically is = given in some "raw" reference frame not close enough to what a = user/router typically can handle out of the box. With cable I am = confident that there are some packet headers that make sure only one = modem picks up the data (and from the euphemistic bandwidth descriptions = on the DSL side I am quite confident that this is in-band not = out-of-band with an independent bandwidth supply.) Dave, would you be willing to collect a ping sample on your = cable link, so I could use this as control for my ATM detector code? As an example of this situation where I have a better handle = take, drum rolls, a typical DSL connection :)=20 A) The bandwidth promised by the ISP typically is inside a corridor = often quite wide, and say 6 to 16Mbit/s is not too helpful for setting = the shaper.=20 B) The modem itself reports the a number of values "Actual Data Rate" = (among others even less useful values like "Attainable Data Rate"); this = requires active intervention by the user, these values are typically = presented via a http. Alternatively advanced users can telnet/ssh into = some modems/routers and get more detailed low-level information from the = DSL modem (but I digress). C) This "Actual Data Rate" is the raw synchronization rate, but since = forward error correction (FEC) is handled semi out-of-band an = unspecified part (given enough information about the FEC parameters this = can b calculated though) of that synch rate might not available to the = user (I cynically just assume that these bits are taken from the actual = data rate without any real proof). With ADSL it seems that an integer = number of sub carrier tones are dedicated to FEC.=20 D) The next issue is the 48 in 53 ATM cell encapsulation which reduces = the useable rate down to ~ 90% (this is handled by link layer ATM quite = well). Naively one would expect that just shaping down to 90% should = have the same effect, but see E). To properly configure this the user = needs to know whether his link is using ATM to the DSLAM or not (this = information might or might not be available on the modem/router's web = interface or on the ISP's customer web interface). E) Unfortunately, the method used to wrap data packets into ATM cells = requires that each packet fills an integer number of ATM cells, so the = left over of the last cell will be padded. This especially dire as this = can almost double the wire size of small packets like ACKs, making it = really hard for the shaper to work well with small packet traffic like = ACKs and to some degree VoIP. Luckily for the user, thanks to Jesper and = Russell, the "linklayer ATM" method for tc handles this as well. But = note for this to work the kernel needs to know how large the packet = including overhead is going to be, which leads to... F) Per packet overhead, there is a number of possible methods to = "encapsulate" IP packets into an ATM carrier, which add different = amounts of overhead. This ranges from IP over ATM (IPoA over VC-MUX) = which basically wraps IP packets into ATM cell meta packets with minimal = overhead of 8 bytes to PPPoE over LLC/SNAP which can add 40bytes per = packet (this not only includes 8byte PPPoE overhead, but also a full = ethernet header worth 14 bytes). To add insult to injury it seems that = the 40+ bytes is the ISP's preferred option (where is the invisible hand = of the market if you need it :) ). Again for small packets not taking = the overhead into account makes shaping quite impossible; in addition = underestimating the overhead will create specific packet sizes for which = the shaper underestimates the wire size by one cell. G) and finally (at least this is the last thing I stumbled over = empirically there might be more out there) any additional overhead the = carrier adds to the packet. Foe example I stumbled over an combined IPTV = Internet access which added a 4 byte ethernet VLAN tag which got removed = at the ISP supplied router creating a total of 44 bytes of overhead. At = least in Europe these kinds of multi-play services using VLANs seem = quite popular. (Now, interestingly both VC-MUX and LLC/SNAP basically = are multiplexing headers that would allow to keep different streams = logically and bandwidth wise separated, but a) these are ATM specific = and b) do not fit into the current ideal of a pure IP "next generation" = network of the telcos. But users on ATM are not getting short changed = here.) Overall, there is quite a significant difference between the = "advertised" wire sped and the useable IP bandwidth on ATM links the = user actually associates with that number, and it seems non-trivial for = a typical user to easily collect all the required information to = configure SQM correctly. I do not see this changing any time soon, = all-fiber networks are not going to come in the (near) future, as much = as I want it to. >=20 >>> NB: I have changed the first menu choice to =93ADSL/ATM=94 and the = second to =93VDSL=94 in the description. >>=20 >> I am fine with changing names, just see what the consensus is = for the names. >=20 > I can live with whatever is decided here, and sebastian has commit = access. So we have "ADSL/ATM" as one, what about "VDSL/PTM", (PTM: = packet transfer mode) to keep things symmetric? (We still have the issue = with possible ATM carrier on VDSL1 but we should deal with this = independently). Or "ATM/ADSL" and "PTM/VDSL", "ethernet" Awaiting your votes... Best Regards Sebastian >=20 >>> I would ask that we change to GUI to reflect those names as well. = This makes it far easier/less confusing to talk about the options. >>=20 >> Agreed. >=20 > Go for it! >=20 >>=20 >>=20 >>> As always, I welcome help in setting out clear recommendations that = work well for the vast majority of people who try CeroWrt. Thanks. >=20 > I don't mind if cero stays too complex for mom to configure, although > I appreciate > the efforts to simplify things. >=20 >>=20 >> I guess, we need a new wiki page detailing the procedure to = figure out the link layer (and overhead if on ATM). >>=20 >> Best >> Sebastian >>=20 >>>=20 >>> Rich >>> _______________________________________________ >>> Cerowrt-devel mailing list >>> Cerowrt-devel@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>=20 >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel >=20 >=20 >=20 > --=20 > Dave T=E4ht >=20 > Fixing bufferbloat with cerowrt: = http://www.teklibre.com/cerowrt/subscribe.html