From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-bw0-f43.google.com (mail-bw0-f43.google.com [209.85.214.43]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 3A03720216B for ; Tue, 30 Aug 2011 18:45:58 -0700 (PDT) Received: by bkbzv15 with SMTP id zv15so385358bkb.16 for ; Tue, 30 Aug 2011 18:45:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=CWTNfp7zaUH0tOST138BR6qBaOl2xnaUuFy8FwvpG58=; b=x1EuhA3tMxA3/1ToI/ETKGXMF603fZM+xabJwJ0NwIR3zhJISkpj+jxvHAo+oAI9uM w/3YYh2Jdg8DGNr3yOd2/FevWoq9ggYFfw+DATB+/m/OMP0d8+Kx72P0lyDeJlkqNnN8 U5ysrv5GtNHwcPFSkV3Lct9vffNyaOUjADaR0= MIME-Version: 1.0 Received: by 10.223.51.195 with SMTP id e3mr267246fag.118.1314755155986; Tue, 30 Aug 2011 18:45:55 -0700 (PDT) Received: by 10.152.40.194 with HTTP; Tue, 30 Aug 2011 18:45:55 -0700 (PDT) In-Reply-To: <4E5D87DD.7040705@hp.com> References: <4E5D87DD.7040705@hp.com> Date: Tue, 30 Aug 2011 18:45:55 -0700 Message-ID: Subject: Re: oprofiling is much saner looking now with rc6-smoketest From: Dave Taht To: Rick Jones Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat-devel X-BeenThere: bloat-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Developers working on AQM, device drivers, and networking stacks" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Aug 2011 01:45:58 -0000 On Tue, Aug 30, 2011 at 6:01 PM, Rick Jones wrote: > On 08/30/2011 05:32 PM, Dave Taht wrote: >> It bugs me that iptables and conntrack eat so much cpu for what >> is an internal-only connection, e.g. one that >> doesn't need conntracking. > > The csum_partial is a bit surprising - I thought every NIC and its dog > offered CKO these days - or is that something happening with > ip_tables/contrack? If this chipset supports it, so far as I know, it isn't documented or implemented. > I also thought that Linux used an integrated > copy/checksum in at least one direction, or did that go away when CKO bec= ame > prevalent? Don't know. > > If this is inbound, and there is just plain checksumming and not anything > funny from conntrack, I would have expected checksum to be much larger th= an > copy. =A0Checksum (in the inbound direction) will take the cache misses a= nd > the copy would not. =A0Unless... the data cache of the processor is getti= ng > completely trashed - say from the netserver running on the router not > keeping up with the inbound data fully and so the copy gets "far away" fr= om > the checksum verification. 220Mbit isn't good enough for ya? Previous tests ran at about 140Mbit, but = due to some major optimizations by felix to fix a bunch of mis-alignment issues. Through the router, I've seen 260Mbit - which is perilously close to the speed that I can drive it at from the test boxes. > > Does perf/perf_events (whatever the followon to perfmon2 is called) have > support for the CPU used in the device? =A0(Assuming it even has a PMU to= be > queried in the first place) Yes. Don't think it's enabled. It is running flat out, according to top. > >> That said, I understand that people like their statistics, and me, >> I'm trying to make split-tcp work better, ultimately, one day.... >> >> I'm going to rerun this without the fw rules next. > > It would be interesting to see if the csum time goes away. =A0Long ago an= d far > away when I was beating on a 32-core system with aggregate netperf TCP_RR > and enabling or not FW rules, conntrack had a non-trivial effect indeed o= n > performance. Stays about the same. iptables time drops. How to disable conntrack? Don't you only really need it for nat? > > http://markmail.org/message/exjtzel7vq2ugt66#query:netdev%20conntrack%20r= ick%20jones%2032%20netperf+page:1+mid:s5v5kylvmlfrpb7a+state:results > > I think will get to the start of that thread. =A0The subject is '32 core > net-next stack/netfilter "scaling"' > > rick jones > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://the-edge.blogspot.com