From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id DF6A721F556 for ; Sun, 27 Jul 2014 04:38:40 -0700 (PDT) Received: from hms-beagle.lan ([134.2.89.70]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MgbvP-1Wp2ZX0AwZ-00Nvoz; Sun, 27 Jul 2014 13:38:36 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: Date: Sun, 27 Jul 2014 13:38:33 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <598BFF8C-37B6-4F5B-9F0D-A0D69A1F32B7@gmx.de> To: David Lang X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:oXhkLz+7F7kKZITHtMVG0RS71KCJnsxin7S2O3qGPUxFkt9ktcK 1TMsPNefXK9AgCtbULqGzCpF3jBSn13r10m9xc1+xkNA048e0PWvs1vIoBBX5wY+YiTh3dU EwLv68Ei4w6uUc9XKNYe+UBFZXDn8GUngvlVha1FoB/6+T/zLOqHt1Is6m88SmB/5mXwz5i blPX/PrW5G653Wzko0ajg== Cc: Wes Felter , cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] Ideas on how to simplify and popularize bufferbloat control for consideration. X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jul 2014 11:38:41 -0000 On Jul 27, 2014, at 03:04 , David Lang wrote: > On Sun, 27 Jul 2014, Sebastian Moeller wrote: [...] >>=20 >>>=20 >>> Think of the router ASICs that handle the 'normal' traffic in the = ASIC in the card, but 'unusual' traffic needs to be sent to the core CPU = to be processed and is therefor MUCH slower >>=20 >> Except for my ICMP RTT measurements I still saw quantization = steps in accordance with the expected best case RTT for a packet, = showing that the slow processing at least is constant and hence easy to = get ridd of in measurements... >=20 > yeah, I have to remind myself of the "perfect is the enemy of good = enough" frequently as well. I tend to fall into that trap pretty easily, = as this discussion has shown :-) >=20 > ping is easy to test. As a thought, is the response time of NTP = queries any more or less stable? No idea? How would you test this (any command line to try). The = good thingg with the ping is that often even the DSLAM responds keeping = external sources (i.e. hops further away in the network) of variability = out of the measurement... >=20 >>>>>>> One thought I have is to require a high TTL on the packets for = the services to respond to them. That way any abuse of the service would = have to take place from very close on the network. >>>>>>>=20 >>>>>>> Ideally these services would only respond to senders that are = directly connected, but until these services are deployed and enabled by = default, there is going to be a need to be the ability to 'jump over' = old equipment. This need will probably never go away completely. >>>>>>=20 >>>>>> But if we need to modify DSLAMs and CMTSs it would be much nicer = if we could just ask nicely what the current negotiated bandwidths are = ;) >>>>>=20 >>>>> negotiated bandwith and effective bandwidth are not the same >>>>>=20 >>>>> what if you can't talk to the devices directly connected to the = DSL line, but only to a router one hop on either side? >>>>=20 >>>> In my limited experience the typical bottleneck is the DSL line, = so if we shape for that we are fine=85 Assume for a moment the DSLAM = uplink is so congested because of oversubscription of the DSLAM, that = now this constitutes the bottleneck. Now the available bandwidth for = each user depends on the combined traffic of all users, not a situation = we can reasonable shape for anyway (I would hope that ISPs monitor this = situation and would remedy it by adding uplink capacity, so this = hopefully is just a transient event). >>>=20 >>> for DSL you are correct, it's a point-to-point connection (star = network topology), but we have other technologies used in homes that are = shared-media bus topology networks. This includes cablemodems and = wireless links. >>=20 >> Well, yes I understand, but you again would assume that the = cable ISP tries to provision the system so that most users are happy, so = congestion is not the rule? Even then I think cable guarantees some = minimum rates per user, no? With wireless it is worse in that RF events = outside of the ISP and end users control can ruin the day. >=20 > guarantee is too strong a word. It depends on how much competition = there is. >=20 > 15 years or so ago I moved from a 3Mb cablemodem to a 128K IDSL line = and saw my performance increase significantly. I used to think exactly the same, but currently I tend to think = that the difference is about how well managed a node is not so much the = access technology, with DSL the shared medium is the link connecting the = DSLAM to the backbone, if this is congested it is similar to a busy = cable node. In both cases the ISP needs to make sure the shared segments = congestion is well managed. I might be that DSLAMs are typically better = manages as TELCO=92s always dealt with interactive (bi-directional) = traffic while cable traditionally was a one-directional transport. So I = assume both have different traditions about provisioning. I could be off = my rocker here ;) >=20 >>>>> for example, I can't buy (at least not for anything close to a = reasonable price) a router to run at home that has a DSL port on it, so = I will always have some device between me and the DSL. >>>>=20 >>>> http://wiki.openwrt.org/toh/tp-link/td-w8970 or >>>=20 >>> no 5GHz wireless? >>=20 >> Could be, but definitely reasonable priced, probany cheap enough = to use as smart de-bloated DSL modem, so your main router does not need = HTB traffic shaping on uplink anymore. I might actually go that route = since I really dislike my ISP primary router, but I digress... >>=20 >>>=20 >>>> http://www.traverse.com.au/products ? >>>=20 >>> I couldn't figure out where to buy one through their site. >>=20 >> Maybe they only sell in AU, I guess I just wanted to be helpful, >>=20 >>>=20 >>>> If you had the DSL modem in the router under cerowrts control you = would not need to use a traffic shaper for your uplink, as you could = apply the BQL ideas to the ADSL driver. >>>>=20 >>>>>=20 >>>>> If you have a shared media (cable, wireless, etc), the negotiated = speed is meaningless. >>>>=20 >>>> Not exactly meaningless, if gives you an upper bound... >>>=20 >>> true, but is an upper bound good enough? How close does the estimate = need to be? >>=20 >> If we end up recommending people using say binary search to find = the best tradeoff (maximizing throughput while keeping the maximum = latency under load increase bounded to say 10ms) we should have an idea = where to start, so bit to large is fine as a starting point. = Traditionally the recommendation was around 85% of link rates, but that = never came with a decent justification or data. >=20 > well, if we are doing a binary search, having the initial estimate off = by a lot isn't actually going to hurt much, we'll still converge very = quickly on the right value Yes, but we still need to solve the question what infrastructure = to test against ;) >=20 >>>=20 >>> and does it matter if both sides are doign fq_codel or is this still = in the mode of trying to control the far side indirectly? >>=20 >> Yes, this is only relevant as long as both sides of the = bottleneck link are not de-bloated. But it does not look like = DSLAMs/CMTs will change any time soon from the old ways... >=20 > yep, I had been forgetting this. >=20 >>>=20 >>>>> In my other location, I have a wireless link that is ethernet to = the dish on the roof, I expect the other end is a similar setup, so I = can never see the link speed directly (not to mention the fact that rain = can degrade the effective link speed) >>>>=20 >>>> One more case for measuring the link speed continuously! >>>=20 >>> at what point does the measuring process interfere with the use of = the link? or cause other upstream issues. >>=20 >> If my measuring by sparse stream idea works out the answer to = both questions is not much ;) >>=20 >>>=20 >>>>>>> Other requirements or restrictions? >>>>>>=20 >>>>>> I think the measurement should be fast and continuous=85 >>>>>=20 >>>>> Fast yes, because we want to impact the network as little as = possible >>>>>=20 >>>>> continuous?? I'm not so sure. Do conditions really change that = much? >>>>=20 >>>> You just gave an example above for changing link conditions, by = shared media... >>>=20 >>> but can you really measure fast enough to handle shared media? at = some point you need to give up measuring because by the time you have = your measurement it's obsolete. >>=20 >> So this is not going to work well a wifi wlan with wildly = fluctuating rates (see Dave=92s upcoming project make-wifi-fast) but for = typical cable node where congestion changes over the day as a function = of people being at home it might be fast enough. >>=20 >>>=20 >>> If you look at networking with a tight enough timeframe, it's either = idle or 100% utilized depending on if a bit is being sent at that = instant, however a plot at that precision is worthless :-) >>=20 >> Yes I think a moving average over some time would be required. >>=20 >>>=20 >>>>> And as I ask in the other thread, how much does it hurt if your = estimates are wrong? >>>>=20 >>>> I think I sent a plot to that regard. >>>=20 >>> yep, our mails are crossing >>>=20 >>>>>=20 >>>>> for wireless links the conditions are much more variable, but we = don't really know what is going to work well there. >>>>=20 >>>> Wireless as in point 2 point links or in wifi? >>>=20 >>> both, point-to-point is variable based on weather, trees blowing in = the wind, interference, etc. Wifi has a lot more congestion, so = interference dominates everything else. >>=20 >> So maybe that is a diffent kettle of fish then. >=20 > I think we need to get a simple, repeatable test together and then = have people start using it and reporting what they find and the type of = connection they are on, otherwise we are speculating from far too little = data. So Rich Brown=92s betterspeedtest.sh is a simple test, at least = for the crowd of people involved in the buffer bloat discussion right = now. I always love to see more data, especially I would be interested to = see data from VDSL1 lines and GPON fiber lines=85 Best Regards Sebastian >=20 > David Lang