From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iw0-f171.google.com (mail-iw0-f171.google.com [209.85.214.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 1CECC20016F for ; Fri, 3 Jun 2011 20:31:18 -0700 (PDT) Received: by iwn8 with SMTP id 8so2917045iwn.16 for ; Fri, 03 Jun 2011 20:49:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=ZN5v82raQAGNaWbsQ/ZYNnQN1mJDv5JmsgOX9+EtZX0=; b=Nk5lo0LvfnVJoGwBGJ4/M++g7hmDTQq2QyDgVwMF6And+T4JiYRE04xN5wu2603fQp L1MdzhAiJjyE4muTnaY5hdG5AD9WFkG+JWJZYd4krKXe2+eWni2sWeQON8bPgcBmZ2JV 38j1q5XnwQ9II5MyTcNwiDstfWb0WjEC075ic= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=T9vReZMoTYxi4cnb0r5r7hR5wrX2DfBqrxvIiMB/8jb5nB+IuRJjeyI0/VlNiIlDaV zSwXrUEPc7yvh8oGgefiSt5+zdyx0M8cdk+mHBegBY+QxP6zjst66SuO4YuN6XaOThxz ziggOL199KPpdZM6EDu7565GlvrFr/DdKKlRQ= MIME-Version: 1.0 Received: by 10.231.68.202 with SMTP id w10mr3766583ibi.63.1307159381131; Fri, 03 Jun 2011 20:49:41 -0700 (PDT) Received: by 10.231.13.76 with HTTP; Fri, 3 Jun 2011 20:49:41 -0700 (PDT) In-Reply-To: References: <7i39jr3x3a.fsf@lanthane.pps.jussieu.fr> Date: Fri, 3 Jun 2011 21:49:41 -0600 Message-ID: Subject: Re: Hacking on the rtl8366S From: Dave Taht To: Juliusz Chroboczek Content-Type: multipart/alternative; boundary=0015177407d093c11004a4dac2a2 Cc: Gabor Juhos , bloat-devel@lists.bufferbloat.net X-BeenThere: bloat-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Developers working on AQM, device drivers, and networking stacks" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Jun 2011 03:31:18 -0000 --0015177407d093c11004a4dac2a2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable So I built a fresh build of openwrt from git head, patched in my preliminar= y patch to the switch, compiled it with oprofile support, and turned on -O2 rather than -Os, in my usual approach to all-up testing that makes it hard to track down what's really going on without backtracking. The net result with iperf was better than before, with iptables and qos turned entirely off, no nat, etc. pings stayed very flat and well below 8ms, down from 100ms. elara: [ 3] 0.0-60.0 sec 435 MBytes 60.7 Mbits/sec io: [ 3] 0.0-60.0 sec 495 MBytes 69.2 Mbits/sec leda: [ 3] 0.0-60.9 sec 226 MBytes 31.1 Mbits/sec thebe: [ 3] 0.0-60.0 sec 105 MBytes 14.7 Mbits/sec (how all this stuff is routed is beyond the scope of this, but only partially explains the discrepancies above) Oprofiling a later run, we're spending a lot of time in an unaligned trap, and I have no idea why iptables and conntrack even register. I am not sure at the moment how to track down the source of the alignment fault, I assume I can look at the performance counter somehow, and backtrac= k that to a given function or functions. It's late... samples cum. samples % cum. % app name symbol name 14992 14992 22.7158 22.7158 vmlinux do_ade 10054 25046 15.2338 37.9496 ip_tables /ip_tables 7257 32303 10.9958 48.9454 nf_conntrack /nf_conntrack 3398 35701 5.1486 54.0941 vmlinux handle_adel_int 2613 38314 3.9592 58.0533 vmlinux ip_rcv 2356 40670 3.5698 61.6231 vmlinux nf_iterate 1869 42539 2.8319 64.4550 iptable_nat /iptable_nat 1626 44165 2.4637 66.9187 vmlinux r4k_dma_cache_inv 1544 45709 2.3395 69.2582 nf_conntrack_ipv4 /nf_conntrack_ipv4 1331 47040 2.0167 71.2749 vmlinux ag71xx_poll 1011 48051 1.5319 72.8068 vmlinux ip_route_input_common 988 49039 1.4970 74.3038 vmlinux ret_from_exception asmlinkage void do_ade(struct pt_regs *regs) { unsigned int __user *pc; mm_segment_t seg; perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, 0, regs, regs->cp0_badvaddr); /* * Did we catch a fault trying to load an instruction? * Or are we running in MIPS16 mode? */ if ((regs->cp0_badvaddr =3D=3D regs->cp0_epc) || (regs->cp0_epc & 0= x1)) goto sigbus; pc =3D (unsigned int __user *) exception_epc(regs); if (user_mode(regs) && !test_thread_flag(TIF_FIXADE)) goto sigbus; if (unaligned_action =3D=3D UNALIGNED_ACTION_SIGNAL) goto sigbus; else if (unaligned_action =3D=3D UNALIGNED_ACTION_SHOW) show_registers(regs); /* * Do branch emulation only if we didn't forward the exception. * This is all so but ugly ... */ seg =3D get_fs(); if (!user_mode(regs)) set_fs(KERNEL_DS); emulate_load_store_insn(regs, (void __user *)regs->cp0_badvaddr, pc); set_fs(seg); return; --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://the-edge.blogspot.com --0015177407d093c11004a4dac2a2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: base64 U28gSSBidWlsdCBhIGZyZXNoIGJ1aWxkIG9mIG9wZW53cnQgZnJvbSBnaXQgaGVhZCwgcGF0Y2hl ZCBpbiBteSBwcmVsaW1pbmFyeSBwYXRjaCB0byB0aGUgc3dpdGNoLCBjb21waWxlZCBpdCB3aXRo IG9wcm9maWxlIHN1cHBvcnQsIGFuZCB0dXJuZWQgb24gLU8yIHJhdGhlciB0aGFuIC1PcywgaW4g bXkgdXN1YWwgYXBwcm9hY2ggdG8gYWxsLXVwIHRlc3RpbmcgdGhhdCBtYWtlcyBpdCBoYXJkIHRv IHRyYWNrIGRvd24gd2hhdCYjMzk7cyByZWFsbHkgZ29pbmcgb24gd2l0aG91dCBiYWNrdHJhY2tp bmcuPGJyPgo8YnI+VGhlIG5ldCByZXN1bHQgd2l0aCBpcGVyZiB3YXMgYmV0dGVyIHRoYW4gYmVm b3JlLDxicj53aXRoIGlwdGFibGVzIGFuZCBxb3MgdHVybmVkIGVudGlyZWx5IG9mZiwgbm8gbmF0 LCBldGMuIDxicj5waW5ncyBzdGF5ZWQgdmVyeSBmbGF0IGFuZCB3ZWxsIGJlbG93IDhtcywgZG93 biBmcm9tIDEwMG1zLjxicj48YnI+ZWxhcmE6IFugIDNdoCAwLjAtNjAuMCBzZWOgoCA0MzUgTUJ5 dGVzoCA2MC43IE1iaXRzL3NlYzxicj4KaW86IFugIDNdoCAwLjAtNjAuMCBzZWOgoCA0OTUgTUJ5 dGVzoCA2OS4yIE1iaXRzL3NlYzxicj5sZWRhOiBboCAzXaAgMC4wLTYwLjkgc2VjoKAgMjI2IE1C eXRlc6AgMzEuMSBNYml0cy9zZWM8YnI+dGhlYmU6IFugIDNdoCAwLjAtNjAuMCBzZWOgoCAxMDUg TUJ5dGVzoCAxNC43IE1iaXRzL3NlYzxiciBjbGVhcj0iYWxsIj48YnI+KGhvdyBhbGwgdGhpcyBz dHVmZiBpcyByb3V0ZWQgaXMgYmV5b25kIHRoZSBzY29wZSBvZiB0aGlzLCBidXQgb25seSBwYXJ0 aWFsbHkgZXhwbGFpbnMgdGhlIGRpc2NyZXBhbmNpZXMgYWJvdmUpPGJyPgo8YnI+T3Byb2ZpbGlu ZyBhIGxhdGVyIHJ1biwgd2UmIzM5O3JlIHNwZW5kaW5nIGEgbG90IG9mIHRpbWUgaW4gYW4gdW5h bGlnbmVkIHRyYXAsIGFuZCBJIGhhdmUgbm8gaWRlYSB3aHkgaXB0YWJsZXMgYW5kIGNvbm50cmFj ayBldmVuIHJlZ2lzdGVyLiA8YnI+PGJyPkkgYW0gbm90IHN1cmUgYXQgdGhlIG1vbWVudCBob3cg dG8gdHJhY2sgZG93biB0aGUgc291cmNlIG9mIHRoZSBhbGlnbm1lbnQgZmF1bHQsIEkgYXNzdW1l IEkgY2FuIGxvb2sgYXQgdGhlIHBlcmZvcm1hbmNlIGNvdW50ZXIgc29tZWhvdywgYW5kIGJhY2t0 cmFjayB0aGF0IHRvIGEgZ2l2ZW4gZnVuY3Rpb24gb3IgZnVuY3Rpb25zLiBJdCYjMzk7cyBsYXRl Li4uPGJyPgoKPGJyPnNhbXBsZXOgIGN1bS4gc2FtcGxlc6AgJaCgoKCgoKAgY3VtLiAloKCgoCBh cHAgbmFtZaCgoKCgoKCgoKCgoKCgoKAgc3ltYm9sIG5hbWU8YnI+MTQ5OTKgoKAgMTQ5OTKgoKCg oKCgoCAyMi43MTU4oCAyMi43MTU4oKCgIHZtbGludXigoKCgoKCgoKCgoKCgoKCgoCBkb19hZGU8 YnI+MTAwNTSgoKAgMjUwNDagoKCgoKCgoCAxNS4yMzM4oCAzNy45NDk2oKCgIGlwX3RhYmxlc6Cg oKCgoKCgoKCgoKCgoCAvaXBfdGFibGVzPGJyPgo3MjU3oKCgoCAzMjMwM6CgoKCgoKCgIDEwLjk5 NTigIDQ4Ljk0NTSgoKAgbmZfY29ubnRyYWNroKCgoKCgoKCgoKCgIC9uZl9jb25udHJhY2s8YnI+ MzM5OKCgoKAgMzU3MDGgoKCgoKCgoKAgNS4xNDg2oCA1NC4wOTQxoKCgIHZtbGludXigoKCgoKCg oKCgoKCgoKCgoCBoYW5kbGVfYWRlbF9pbnQ8YnI+MjYxM6CgoKAgMzgzMTSgoKCgoKCgoKAgMy45 NTkyoCA1OC4wNTMzoKCgIHZtbGludXigoKCgoKCgoKCgoKCgoKCgoCBpcF9yY3Y8YnI+CjIzNTag oKCgIDQwNjcwoKCgoKCgoKCgIDMuNTY5OKAgNjEuNjIzMaCgoCB2bWxpbnV4oKCgoKCgoKCgoKCg oKCgoKAgbmZfaXRlcmF0ZTxicj4xODY5oKCgoCA0MjUzOaCgoKCgoKCgoCAyLjgzMTmgIDY0LjQ1 NTCgoKAgaXB0YWJsZV9uYXSgoKCgoKCgoKCgoKCgIC9pcHRhYmxlX25hdDxicj4xNjI2oKCgoCA0 NDE2NaCgoKCgoKCgoCAyLjQ2MzegIDY2LjkxODegoKAgdm1saW51eKCgoKCgoKCgoKCgoKCgoKCg IHI0a19kbWFfY2FjaGVfaW52PGJyPgoxNTQ0oKCgoCA0NTcwOaCgoKCgoKCgoCAyLjMzOTWgIDY5 LjI1ODKgoKAgbmZfY29ubnRyYWNrX2lwdjSgoKCgoKCgIC9uZl9jb25udHJhY2tfaXB2NDxicj4x MzMxoKCgoCA0NzA0MKCgoKCgoKCgoCAyLjAxNjegIDcxLjI3NDmgoKAgdm1saW51eKCgoKCgoKCg oKCgoKCgoKCgIGFnNzF4eF9wb2xsPGJyPjEwMTGgoKCgIDQ4MDUxoKCgoKCgoKCgIDEuNTMxOaAg NzIuODA2OKCgoCB2bWxpbnV4oKCgoKCgoKCgoKCgoKCgoKAgaXBfcm91dGVfaW5wdXRfY29tbW9u PGJyPgo5ODigoKCgoCA0OTAzOaCgoKCgoKCgoCAxLjQ5NzCgIDc0LjMwMzigoKAgdm1saW51eKCg oKCgoKCgoKCgoKCgoKCgIHJldF9mcm9tX2V4Y2VwdGlvbjxicj48YnI+PGJyPjxicj5hc21saW5r YWdlIHZvaWQgZG9fYWRlKHN0cnVjdCBwdF9yZWdzICpyZWdzKTxicj57PGJyPqCgoKCgoKAgdW5z aWduZWQgaW50IF9fdXNlciAqcGM7PGJyPqCgoKCgoKAgbW1fc2VnbWVudF90IHNlZzs8YnI+PGJy PgqgoKCgoKCgIHBlcmZfc3dfZXZlbnQoUEVSRl9DT1VOVF9TV19BTElHTk1FTlRfRkFVTFRTLDxi cj6goKCgoKCgoKCgoKCgoKCgoKCgoKCgoCAxLCAwLCByZWdzLCByZWdzLSZndDtjcDBfYmFkdmFk ZHIpOzxicj6goKCgoKCgIC8qPGJyPqCgoKCgoKCgICogRGlkIHdlIGNhdGNoIGEgZmF1bHQgdHJ5 aW5nIHRvIGxvYWQgYW4gaW5zdHJ1Y3Rpb24/PGJyPqCgoKCgoKCgICogT3IgYXJlIHdlIHJ1bm5p bmcgaW4gTUlQUzE2IG1vZGU/PGJyPgqgoKCgoKCgoCAqLzxicj6goKCgoKCgIGlmICgocmVncy0m Z3Q7Y3AwX2JhZHZhZGRyID09IHJlZ3MtJmd0O2NwMF9lcGMpIHx8IChyZWdzLSZndDtjcDBfZXBj ICZhbXA7IDB4MSkpPGJyPqCgoKCgoKCgoKCgoKCgoCBnb3RvIHNpZ2J1czs8YnI+PGJyPqCgoKCg oKAgcGMgPSAodW5zaWduZWQgaW50IF9fdXNlciAqKSBleGNlcHRpb25fZXBjKHJlZ3MpOzxicj6g oKCgoKCgIGlmICh1c2VyX21vZGUocmVncykgJmFtcDsmYW1wOyAhdGVzdF90aHJlYWRfZmxhZyhU SUZfRklYQURFKSk8YnI+CqCgoKCgoKCgoKCgoKCgoCBnb3RvIHNpZ2J1czs8YnI+oKCgoKCgoCBp ZiAodW5hbGlnbmVkX2FjdGlvbiA9PSBVTkFMSUdORURfQUNUSU9OX1NJR05BTCk8YnI+oKCgoKCg oKCgoKCgoKCgIGdvdG8gc2lnYnVzOzxicj6goKCgoKCgIGVsc2UgaWYgKHVuYWxpZ25lZF9hY3Rp b24gPT0gVU5BTElHTkVEX0FDVElPTl9TSE9XKTxicj6goKCgoKCgoKCgoKCgoKAgc2hvd19yZWdp c3RlcnMocmVncyk7PGJyPgo8YnI+oKCgoKCgoCAvKjxicj6goKCgoKCgoCAqIERvIGJyYW5jaCBl bXVsYXRpb24gb25seSBpZiB3ZSBkaWRuJiMzOTt0IGZvcndhcmQgdGhlIGV4Y2VwdGlvbi48YnI+ oKCgoKCgoKAgKiBUaGlzIGlzIGFsbCBzbyBidXQgdWdseSAuLi48YnI+oKCgoKCgoKAgKi88YnI+ oKCgoKCgoCBzZWcgPSBnZXRfZnMoKTs8YnI+oKCgoKCgoCBpZiAoIXVzZXJfbW9kZShyZWdzKSk8 YnI+oKCgoKCgoKCgoKCgoKCgIHNldF9mcyhLRVJORUxfRFMpOzxicj4KoKCgoKCgoCBlbXVsYXRl X2xvYWRfc3RvcmVfaW5zbihyZWdzLCAodm9pZCBfX3VzZXIgKilyZWdzLSZndDtjcDBfYmFkdmFk ZHIsIHBjKTs8YnI+oKCgoKCgoCBzZXRfZnMoc2VnKTs8YnI+PGJyPqCgoKCgoKAgcmV0dXJuOzxi cj48YnI+PGJyPi0tIDxicj5EYXZlIFTkaHQ8YnI+U0tZUEU6IGRhdmV0YWh0PGJyPlVTIFRlbDog MS0yMzktODI5LTU2MDg8YnI+PGEgaHJlZj0iaHR0cDovL3RoZS1lZGdlLmJsb2dzcG90LmNvbSIg dGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly90aGUtZWRnZS5ibG9nc3BvdC5jb208L2E+IDxicj4KCg== --0015177407d093c11004a4dac2a2--