[Cerowrt-devel] bulk packet transmission
david at lang.hm
Fri Oct 10 23:15:53 EDT 2014
I've been watching Linux kernel development for a long time and they add locks
only when benchmarks show that a lock is causing a bottleneck. They don't just
add them because they can.
They do also spend a lot of time working to avoid locks.
One thing that you are missing is that you are thinking of the TCP/IP system as
a single thread of execution, but there's far more going on than that,
especially when you have multiple NICs and cores and have lots of interrupts
Each TCP/IP stream is not a separate queue of packets in the kernel, instead
the details of what threads exist is just a table of information. The packets
are all put in a small number of queues to be sent out, and the low-level driver
picks the next packet to send from these queues without caring about what TCP/IP
stream it's from.
On Fri, 10 Oct 2014, dpreed at reed.com wrote:
> The best approach to dealing with "locking overhead" is to stop thinking that
> if locks are good, more locking (finer grained locking) is better. OS
> designers (and Linux designers in particular) are still putting in way too
> much locking. I deal with this in my day job (we support systems with very
> large numbers of cpus and because of the "fine grained" locking obsession, the
> parallelized capacity is limited). If you do a thoughtful design of your
> network code, you don't need lots of locking - because TCP/IP streams don't
> have to interact much - they are quite independent. But instead OS designers
> spend all their time thinking about doing "one thing at a time".
> There are some really good ideas out there (e.g. RCU) but you have to think
> about the big picture of networking to understand how to use them. I'm not
> impressed with the folks who do the Linux networking stacks.
> On Thursday, October 9, 2014 3:48pm, "Dave Taht" <dave.taht at gmail.com> said:
>> I have some hope that the skb->xmit_more API could be used to make
>> aggregating packets in wifi on an AP saner. (my vision for it was that
>> the overlying qdisc would set xmit_more while it still had packets
>> queued up for a given station and then stop and switch to the next.
>> But the rest of the infrastructure ended up pretty closely tied to
>> Jesper just wrote a nice piece about it also.
>> It was nice to fool around at 10GigE for a while! And netperf-wrapper
>> scales to this speed also! :wow:
>> I do worry that once sch_fq and fq_codel support is added that there
>> will be side effects. I would really like - now that there are al
>> these people profiling things at this level to see profiles including
>> those qdiscs.
>> /me goes grumbling back to thinking about wifi.
>> On Thu, Oct 9, 2014 at 12:40 PM, David Lang <david at lang.hm> wrote:
>> > lwn.net has an article about a set of new patches that avoid some locking
>> > overhead by transmitting multiple packets at once.
>> > It doesn't work for things with multiple queues (like fq_codel) in it's
>> > current iteration, but it sounds like something that should be looked at and
>> > watched for latency related issues.
>> > http://lwn.net/Articles/615238/
>> > David Lang
>> > _______________________________________________
>> > Cerowrt-devel mailing list
>> > Cerowrt-devel at lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> Dave Täht
>> Cerowrt-devel mailing list
>> Cerowrt-devel at lists.bufferbloat.net
More information about the Cerowrt-devel