From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-31-ewr.dyndns.com (mxout-100-ewr.mailhop.org [216.146.33.100]) by lists.bufferbloat.net (Postfix) with ESMTP id 877DB2E0077 for ; Fri, 11 Feb 2011 09:55:30 -0800 (PST) Received: from scan-31-ewr.mailhop.org (scan-31-ewr.local [10.0.141.237]) by mail-31-ewr.dyndns.com (Postfix) with ESMTP id 0D8746F7CBE for ; Fri, 11 Feb 2011 17:55:29 +0000 (UTC) X-Spam-Score: -4.0 (----) X-Mail-Handler: MailHop by DynDNS X-Originating-IP: 192.6.19.124 Received: from madara.hpl.hp.com (madara.hpl.hp.com [192.6.19.124]) by mail-31-ewr.dyndns.com (Postfix) with ESMTP id 4D0446F6A22 for ; Fri, 11 Feb 2011 17:55:25 +0000 (UTC) Received: from mailhub-pa1.hpl.hp.com (mailhub-pa1.hpl.hp.com [15.25.115.25]) by madara.hpl.hp.com (8.14.3/8.14.3/HPL-PA Relay) with ESMTP id p1BHtJ5G008995 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 11 Feb 2011 09:55:19 -0800 Received: from bougret.hpl.hp.com (bougret.hpl.hp.com [15.9.72.130]) by mailhub-pa1.hpl.hp.com (8.14.3/8.14.3/HPL-PA Hub) with ESMTP id p1BHtIxG026774 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Fri, 11 Feb 2011 09:55:18 -0800 Received: from jt by bougret.hpl.hp.com with local (Exim 4.69) (envelope-from ) id 1PnxDC-0008PC-P3; Fri, 11 Feb 2011 09:55:18 -0800 Date: Fri, 11 Feb 2011 09:55:18 -0800 From: Jean Tourrilhes To: Dave =?iso-8859-1?Q?T=E4ht?= , bloat@lists.bufferbloat.net Message-ID: <20110211175518.GA32302@bougret.hpl.hp.com> References: <871v3ey4rk.fsf@cruithne.co.teklibre.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <871v3ey4rk.fsf@cruithne.co.teklibre.org> Organisation: HP Labs Palo Alto Address: HP Labs, 1U-17, 1501 Page Mill road, Palo Alto, CA 94304, USA. E-mail: jt@hpl.hp.com User-Agent: Mutt/1.5.18 (2008-05-17) X-Scanned-By: MIMEDefang 2.69 on 15.0.152.124 Cc: Juliusz Chroboczek , Stuart Cheshire Subject: Re: [Bloat] Analyzing wireless multiqueue behavior & bufferbloat X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list Reply-To: jt@hpl.hp.com List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 17:55:31 -0000 Hi all, Dave Täht asked me to give my two cents about interraction between Wireless and TCP/IP. My first reaction is "here we go, again". This subject has been beaten to death by the experts in the past, for example the IETF had a PILC subgroup, it generated a couple of intertesting documents on the subject, and the mailing list has interesting discussion. The PILC workgroup : http://datatracker.ietf.org/wg/pilc/charter/ http://www.isi.edu/pilc/ http://tools.ietf.org/html/draft-ietf-pilc-error-06 My humble contribution on the mailing list : http://www.ietf.org/mail-archive/web/pilc/current/msg00211.html I did not go deep in the Bloat discussion, I don't have time to read the mailing list, so I don't know if there is something truly new apart from a cute marketing name. My personal view on the subject of TCP/IP over Wireless is that the characteristics of the physical layer will dictate what is possible, and without extensive knowledge and characterisation of said physical layer, there is no point having a discussion. The Wireless layer is very peculiar, and to use it efficiently you need to be very intimate with it. I don't think it's a good idea to make TCP/IP aware of things like RSSI and phy speed on every hop, so the MAC has to encapsulate all that (the wireless hop may be in the middle of a multi-hop link). Obviously, we would like the MAC layer to help TCP/IP rather than get in the way. The main way to achieve that is for the MAC timing to be way below the TCP/IP timings. Also, UDP and TCP have different characteristics and both need to be supported. In my mind, the two main point is how should TCP/IP deal with fading and bursty interferers. Forget about the other sideshow. A gross approximation of fading is that every 100ms the characteristic of the link change drastically. Sometimes it is faster, sometimes slower, sometime no packet can go through. If you link is very poor for 100ms due to fading, what should TCP/IP do ? The 2.4 GHz and 5 GHz bands are unlicensed, which mean that there can be quite a few potential interferers. I personally looked at micorowave ovens and Bluetooth interference on 802.11, but you can also have cordless phones, baby monitors, ZigBee and maybe wireless USB. Very often, those interferers will result in bursty losses, that the MAC cover by retrying various packet for a long time, what should TCP/IP do ? Dave Täht wrote : > > I've put some thought into the history of my use of proxies over > wireless - re-analyzing something that I'd incorporated into my gut back > in 1996. > > The "split TCP" smoothing effect on the last mile I'd noticed *then* was > minor; the time savings were dominated by the latencies in the modem of > 100ms or so. [2] > > The mosquitonet research[3] had clearly shown the disastrous effects on > un-error-corrected wireless on TCP. There are many papers dealing with the subject, see at the end... > Even then, we regarded 1-3% packet loss as acceptable. Is that loss above Phy, or loss above the MAC layer ? If it's above the Phy, that would be what I expect, but I'd also say the Phy and the MAC not very agressive in trying to use the medium. If it's above the MAC, then I would consider the MAC broken, unless the link conditions are really bad. > 1) Wireless Packet aggregation is now becoming more common, leading > to bursty behavior for short packets. What sorts of packets are > being aggregated? How much buffering is in place? I worry. There is many way to do Packet aggregation. Some add minimal delay in presence of errors, some take time to detect errors. > 2) I think I can theorize that starting 8 TCP connections in > a browser, rather than 2, also "smooths" out bursty packet loss on > the wireless last hop. I sincerly hope that your 802.11 link does not loose packets and only adds trivial delay with a good ARQ mechanism. > browsers still look for wpad Do they ? I always found support pretty lacking. > I imagine, if, for example, apple would use these techniques on their > tightly integrated wireless gateways and apple tv - that it would > provide a competitive advantage. > > The same goes for other manufacturers. The problem is that there is no money to be made selling $50 wireless gateways and $15 Mini-PCI NIC, so I don't see who would be driving innovation in Wireless LAN space. Note that one of the problem of those gateways is that they are software devices, and therefore need buffers to handle the inherent latencies between the NIC and the CPU. Real Time handling of incomming network packet would be needed to decrease those buffers. On Fri, Feb 11, 2011 at 08:23:11AM -0700, Dave Täht wrote: > There also seems to be an impedance mismatch between the > availability of various theoretical QoS mechanisms in the 802.11 > standards and their actual implementations in the networking > stacks. We're treating wireless too much like ethernet to the > detriment of the entire internet. Last time I checked (which is a long time ago), the Intel guys (iwlwifi) were the most advances in term of QoS support in Wireless. > High on my list is making TX_RETRY a tunable. Has always been available in Wireless Extensions since day one, for obvious reasons. Not many driver support it. Note that the optimal retry mechanism is not based on number of retry, but on elapsed time. Obviously, no driver that I know implement that :-( > And I'm deeply concerned about possible side-effects of packet > aggregation in 802.11n. There is many type of packet aggregation in 802.11 and proprietary schemes, and they should behave differently. > I've also been unable to dig up the original mosquitonet paper - or > something more current - that determined the need for link layer error > correction/retransmits in the first place. The reference I find truly useful, I've added to the papers I published on the subject : http://www.hpl.hp.com/personal/Jean_Tourrilhes/Papers/papers.html Those are old papers, but the references should be still relevant. A paper that I don't cite, but was a big hit in its time was the Berkely Snoop Protocol : http://nms.lcs.mit.edu/~hari/papers/snoop.html Ironically, this is the Snoop protocol that convinced me that the right way was to design the appropriate ARQ mechanisms in the link layer, rather than using Split-TCP approaches, as long as the ARQ was immediate so that its timing would not be perceived by TCP/IP. If you look at Snoop and understand what it does, your gut reaction should be, yeah, we can do much simpler by doing that at the MAC level. Personally, I think you really should read the following paper which I consider one of the definitive answer on the subject : [3] Hari Balakrishnan, Venkata N. Padmanabhan, Srinivasan Seshan and Randy H. Katz. A comparison of mechanisms for improving TCP performance over wireless links. Proc. ACM SIGCOMM Conference, Stanford, CA, USA, Aug 1996. Remember that their timeout for LL are very coarse compared to what a typical MAC would do, which explain why they have to play with TCP acks. And I think they deal pretty adequately with all the Proxy approaches ;-) Have fun... Jean