From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from bifrost.lang.hm (mail.lang.hm [64.81.33.126]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 0021521F517 for ; Fri, 10 Oct 2014 20:15:56 -0700 (PDT) Received: from asgard.lang.hm (asgard.lang.hm [10.0.0.100]) by bifrost.lang.hm (8.13.4/8.13.4/Debian-3) with ESMTP id s9B3FrMX002581; Fri, 10 Oct 2014 20:15:53 -0700 Date: Fri, 10 Oct 2014 20:15:53 -0700 (PDT) From: David Lang X-X-Sender: dlang@asgard.lang.hm To: dpreed@reed.com In-Reply-To: <1412988767.10122173@apps.rackspace.com> Message-ID: References: <1412988767.10122173@apps.rackspace.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="680960-1586881214-1412997353=:23992" Cc: "cerowrt-devel@lists.bufferbloat.net" , Jesper Dangaard Brouer Subject: Re: [Cerowrt-devel] bulk packet transmission X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Oct 2014 03:16:25 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --680960-1586881214-1412997353=:23992 Content-Type: TEXT/PLAIN; format=flowed; charset=ISO-8859-15 Content-Transfer-Encoding: 8BIT I've been watching Linux kernel development for a long time and they add locks only when benchmarks show that a lock is causing a bottleneck. They don't just add them because they can. They do also spend a lot of time working to avoid locks. One thing that you are missing is that you are thinking of the TCP/IP system as a single thread of execution, but there's far more going on than that, especially when you have multiple NICs and cores and have lots of interrupts going on. Each TCP/IP stream is not a separate queue of packets in the kernel, instead the details of what threads exist is just a table of information. The packets are all put in a small number of queues to be sent out, and the low-level driver picks the next packet to send from these queues without caring about what TCP/IP stream it's from. David Lang On Fri, 10 Oct 2014, dpreed@reed.com wrote: > The best approach to dealing with "locking overhead" is to stop thinking that > if locks are good, more locking (finer grained locking) is better. OS > designers (and Linux designers in particular) are still putting in way too > much locking. I deal with this in my day job (we support systems with very > large numbers of cpus and because of the "fine grained" locking obsession, the > parallelized capacity is limited). If you do a thoughtful design of your > network code, you don't need lots of locking - because TCP/IP streams don't > have to interact much - they are quite independent. But instead OS designers > spend all their time thinking about doing "one thing at a time". > > There are some really good ideas out there (e.g. RCU) but you have to think > about the big picture of networking to understand how to use them. I'm not > impressed with the folks who do the Linux networking stacks. > > > On Thursday, October 9, 2014 3:48pm, "Dave Taht" said: > > > >> I have some hope that the skb->xmit_more API could be used to make >> aggregating packets in wifi on an AP saner. (my vision for it was that >> the overlying qdisc would set xmit_more while it still had packets >> queued up for a given station and then stop and switch to the next. >> But the rest of the infrastructure ended up pretty closely tied to >> BQL....) >> >> Jesper just wrote a nice piece about it also. >> http://netoptimizer.blogspot.com/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html >> >> It was nice to fool around at 10GigE for a while! And netperf-wrapper >> scales to this speed also! :wow: >> >> I do worry that once sch_fq and fq_codel support is added that there >> will be side effects. I would really like - now that there are al >> these people profiling things at this level to see profiles including >> those qdiscs. >> >> /me goes grumbling back to thinking about wifi. >> >> On Thu, Oct 9, 2014 at 12:40 PM, David Lang wrote: >> > lwn.net has an article about a set of new patches that avoid some locking >> > overhead by transmitting multiple packets at once. >> > >> > It doesn't work for things with multiple queues (like fq_codel) in it's >> > current iteration, but it sounds like something that should be looked at and >> > watched for latency related issues. >> > >> > http://lwn.net/Articles/615238/ >> > >> > David Lang >> > _______________________________________________ >> > Cerowrt-devel mailing list >> > Cerowrt-devel@lists.bufferbloat.net >> > https://lists.bufferbloat.net/listinfo/cerowrt-devel >> >> >> >> -- >> Dave Täht >> >> https://www.bufferbloat.net/projects/make-wifi-fast >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel >> --680960-1586881214-1412997353=:23992--