Development issues regarding the cerowrt test router project
 help / color / mirror / Atom feed
From: David Lang <david@lang.hm>
To: dpreed@reed.com
Cc: "cerowrt-devel@lists.bufferbloat.net"
	<cerowrt-devel@lists.bufferbloat.net>,
	Jesper Dangaard Brouer <brouer@redhat.com>
Subject: Re: [Cerowrt-devel] bulk packet transmission
Date: Fri, 10 Oct 2014 20:15:53 -0700 (PDT)	[thread overview]
Message-ID: <alpine.DEB.2.02.1410102010100.23992@nftneq.ynat.uz> (raw)
In-Reply-To: <1412988767.10122173@apps.rackspace.com>

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3735 bytes --]

I've been watching Linux kernel development for a long time and they add locks 
only when benchmarks show that a lock is causing a bottleneck. They don't just 
add them because they can.

They do also spend a lot of time working to avoid locks.

One thing that you are missing is that you are thinking of the TCP/IP system as 
a single thread of execution, but there's far more going on than that, 
especially when you have multiple NICs and cores and have lots of interrupts 
going on.

Each TCP/IP stream is not a separate queue of packets in the kernel, instead 
the details of what threads exist is just a table of information. The packets 
are all put in a small number of queues to be sent out, and the low-level driver 
picks the next packet to send from these queues without caring about what TCP/IP 
stream it's from.

David Lang

On Fri, 10 Oct 2014, dpreed@reed.com wrote:

> The best approach to dealing with "locking overhead" is to stop thinking that 
> if locks are good, more locking (finer grained locking) is better.  OS 
> designers (and Linux designers in particular) are still putting in way too 
> much locking.  I deal with this in my day job (we support systems with very 
> large numbers of cpus and because of the "fine grained" locking obsession, the 
> parallelized capacity is limited).  If you do a thoughtful design of your 
> network code, you don't need lots of locking - because TCP/IP streams don't 
> have to interact much - they are quite independent.  But instead OS designers 
> spend all their time thinking about doing "one thing at a time".
> 
> There are some really good ideas out there (e.g. RCU) but you have to think 
> about the big picture of networking to understand how to use them.  I'm not 
> impressed with the folks who do the Linux networking stacks.
>
>
> On Thursday, October 9, 2014 3:48pm, "Dave Taht" <dave.taht@gmail.com> said:
>
>
>
>> I have some hope that the skb->xmit_more API could be used to make
>> aggregating packets in wifi on an AP saner. (my vision for it was that
>> the overlying qdisc would set xmit_more while it still had packets
>> queued up for a given station and then stop and switch to the next.
>> But the rest of the infrastructure ended up pretty closely tied to
>> BQL....)
>> 
>> Jesper just wrote a nice piece about it also.
>> http://netoptimizer.blogspot.com/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html
>> 
>> It was nice to fool around at 10GigE for a while! And netperf-wrapper
>> scales to this speed also! :wow:
>> 
>> I do worry that once sch_fq and fq_codel support is added that there
>> will be side effects. I would really like - now that there are al
>> these people profiling things at this level to see profiles including
>> those qdiscs.
>> 
>> /me goes grumbling back to thinking about wifi.
>> 
>> On Thu, Oct 9, 2014 at 12:40 PM, David Lang <david@lang.hm> wrote:
>> > lwn.net has an article about a set of new patches that avoid some locking
>> > overhead by transmitting multiple packets at once.
>> >
>> > It doesn't work for things with multiple queues (like fq_codel) in it's
>> > current iteration, but it sounds like something that should be looked at and
>> > watched for latency related issues.
>> >
>> > http://lwn.net/Articles/615238/
>> >
>> > David Lang
>> > _______________________________________________
>> > Cerowrt-devel mailing list
>> > Cerowrt-devel@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> 
>> 
>> 
>> --
>> Dave Täht
>> 
>> https://www.bufferbloat.net/projects/make-wifi-fast
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>

  reply	other threads:[~2014-10-11  3:15 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-10-09 19:40 David Lang
2014-10-09 19:48 ` Dave Taht
2014-10-11  0:52   ` dpreed
2014-10-11  3:15     ` David Lang [this message]
2014-10-11  4:20       ` David P. Reed
2014-10-13 22:11     ` Dave Taht
2014-10-15 19:49     ` Wes Felter
2014-10-15 22:41       ` dpreed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/cerowrt-devel.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.02.1410102010100.23992@nftneq.ynat.uz \
    --to=david@lang.hm \
    --cc=brouer@redhat.com \
    --cc=cerowrt-devel@lists.bufferbloat.net \
    --cc=dpreed@reed.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox