From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 5BA5121FC4E for ; Mon, 29 Jun 2015 09:44:32 -0700 (PDT) Received: by obzt3 with SMTP id t3so17494193obz.3 for ; Mon, 29 Jun 2015 09:44:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MkLus3M03Ou2rT61wz8vMj1xih+cXiEs9lODkO3u1cQ=; b=neYuvlDQgt01r2lbTygTxZgXFfBjOr3Cx5W1RLXboswzJdYn9BE/u+Rtt7zGo6xGpl dlpxxS2lux396lFlBeZEhlDqIzTFScObpa9eVOrp9MhCsf3wu+MzjNxwUFcmvBj2sZqq +yyIIs01IJwwMDgRObiMZjqa17fvfHmQL/NlUq1EG0dWHkFDtUAOzknTs6xGDos3Rlyj uVQgwS6eqG75Mh5SXAO6kKntPwIyS1pHciLISguaUu11aeSdB9tRTflDu1XDargVIwWQ 3NOXeII6Xy/9dAC9Nr/wZ/ceAn1Cwp53E2xuQ9iAkpt+RMfQBjC2RfdYxIITjKwObR6T o2jg== MIME-Version: 1.0 X-Received: by 10.182.246.9 with SMTP id xs9mr12343204obc.45.1435596272051; Mon, 29 Jun 2015 09:44:32 -0700 (PDT) Received: by 10.202.107.9 with HTTP; Mon, 29 Jun 2015 09:44:31 -0700 (PDT) In-Reply-To: <92199704-0522-447A-887A-1EE0E6AE4421@gmx.de> References: <8B853F1C-DE5D-4F3D-88CC-CB8DA2D3E8B1@gmx.de> <04331509-F163-4184-90B4-8589073AFD62@gmx.de> <09BA156C-460D-4794-A082-33E805F3D6FD@gmx.de> <5436B48C-0803-46DA-B355-14E917A5BB37@gmx.de> <4E002218-174D-44F9-91A0-C7F34B9E83C7@gmx.de> <87pp4eomfx.fsf@alrua-karlstad.karlstad.toke.dk> <92199704-0522-447A-887A-1EE0E6AE4421@gmx.de> Date: Mon, 29 Jun 2015 09:44:31 -0700 Message-ID: From: Dave Taht To: Sebastian Moeller Content-Type: multipart/mixed; boundary=001a11c306eed6a30e0519aacd28 Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; "cake overhead 40" didn't) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 16:45:01 -0000 --001a11c306eed6a30e0519aacd28 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon, Jun 29, 2015 at 6:42 AM, Sebastian Moeller wrote: > HI Mikael,, hi Jonathan, > >> [...] >> >> These are the results from 50M and 500M, also including 50up and 50down = that I added to my test suite script. >> >> http://swm.pp.se/aqm/rrul_150629-cake-4.tar >> > > Now both ingress and egress are up to roughly 455Mbps from roughly 360 wi= th cake just playing leaf qdisc for HTB. This looks even better than before= =E2=80=A6 350 *usec* induced delay on the 50mbit rrul_be test. w00t! Most of the tests compare well with the reference rangeley data now. I would like a 900mbit soft shaped result. 1.2ms at 500mbit. Less of a w00t. Possible it is coming from elsewhere on that path (fq or fq_codel on the server and client?) cake currently peels at 1ms / flows (more or less)... NAPI is an issue... hw mq an issue... There are a half dozen things in the mvneta driver I would try to reduce it's latency more. The easy ones: reduce this to 16: netif_napi_add(dev, &pp->napi, mvneta_poll, NAPI_POLL_WEIGHT); Reduce this to 24: (this will also reduce the max outstanding stuff in the driver by a LOT, but is still not BQL!) /* Max number of allowed TCP segments for software TSO */ #define MVNETA_MAX_TSO_SEGS 100 Both of the will improve read side latency at the cost of more sirqs. I do not know what reducing these will do, and would test both of the above separately. /* Coalescing */ #define MVNETA_TXDONE_COAL_PKTS 1 #define MVNETA_RX_COAL_PKTS 32 #define MVNETA_RX_COAL_USEC 100 As for cake itself, eric dumazet told us we dont need atomic ops in it, and peeling at at even lower threshold has some appeal (to me, anyway) attached is a patch for that, put it in your feeds/cero/kmod_sched_cake/pat= ches directory, rebuild (make package/kmod-sched-cake/{clean,compile,install}) (bump up the makefile rel number also, if you want) > Best Regards > Sebastian > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel --=20 Dave T=C3=A4ht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast --001a11c306eed6a30e0519aacd28 Content-Type: text/x-patch; charset=US-ASCII; name="0001-Rid-unneeded-atomic-ops-and-reduce-peeling-threshold.patch" Content-Disposition: attachment; filename="0001-Rid-unneeded-atomic-ops-and-reduce-peeling-threshold.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ibi4t2qk0 RnJvbSA0NmJlNjA5ZTk1NDc0ZTlkYjg1NmI1ZTEyNzU2ZDRhNzU2OGFkZjQyIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBEYXZlIFRhaHQgPGRhdmUudGFodEBidWZmZXJibG9hdC5uZXQ+ CkRhdGU6IE1vbiwgMjkgSnVuIDIwMTUgMDk6Mzg6MDAgLTA3MDAKU3ViamVjdDogW1BBVENIXSBS aWQgdW5uZWVkZWQgYXRvbWljIG9wcyBhbmQgcmVkdWNlIHBlZWxpbmcgdGhyZXNob2xkCgotLS0K IHNjaF9jYWtlLmMgfCAxMiArKysrKystLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCA2IGluc2VydGlv bnMoKyksIDYgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvc2NoX2Nha2UuYyBiL3NjaF9jYWtl LmMKaW5kZXggODBlMWNiMi4uOWEzNThiOSAxMDA2NDQKLS0tIGEvc2NoX2Nha2UuYworKysgYi9z Y2hfY2FrZS5jCkBAIC0xMjEsNyArMTIxLDcgQEAgc3RydWN0IGNha2VfZnFjZF9zY2hlZF9kYXRh IHsKIAogICAgIHN0cnVjdCBjb2RlbF9wYXJhbXMgY3BhcmFtczsKICAgICB1MzIgICAgICBkcm9w X292ZXJsaW1pdDsKLSAgICBhdG9taWNfdCBmbG93X2NvdW50OworICAgIHUzMiBmbG93X2NvdW50 OwogCiAgICAgc3RydWN0IGxpc3RfaGVhZCBuZXdfZmxvd3M7IC8qIGxpc3Qgb2YgbmV3IGZsb3dz ICovCiAgICAgc3RydWN0IGxpc3RfaGVhZCBvbGRfZmxvd3M7IC8qIGxpc3Qgb2Ygb2xkIGZsb3dz ICovCkBAIC00MjcsNyArNDI3LDcgQEAgc3RhdGljIGludCBjYWtlX2VucXVldWUoc3RydWN0IHNr X2J1ZmYgKnNrYiwgc3RydWN0IFFkaXNjICpzY2gpCiAJICogU3BsaXQgR1NPIGFnZ3JlZ2F0ZXMg aWYgdGhleSdyZSBsaWtlbHkgdG8gaW1wYWlyIGZsb3cgaXNvbGF0aW9uCiAJICogb3IgaWYgd2Ug bmVlZCB0byBrbm93IGluZGl2aWR1YWwgcGFja2V0IHNpemVzIGZvciBmcmFtaW5nIG92ZXJoZWFk LgogCSAqLwotCWlmKHVubGlrZWx5KChsZW4gKiBtYXgoYXRvbWljX3JlYWQoJmZxY2QtPmZsb3df Y291bnQpLCAxKSkgPiBxLT5wZWVsX3RocmVzaG9sZCAmJiBza2JfaXNfZ3NvKHNrYikpKQorCWlm KHVubGlrZWx5KChsZW4gKiBtYXgoJmZxY2QtPmZsb3dfY291bnQsIDEpKSA+IHEtPnBlZWxfdGhy ZXNob2xkICYmIHNrYl9pc19nc28oc2tiKSkpCiAJewogCQlzdHJ1Y3Qgc2tfYnVmZiAqc2Vncywg Km5za2I7CiAJCW5ldGRldl9mZWF0dXJlc190IGZlYXR1cmVzID0gbmV0aWZfc2tiX2ZlYXR1cmVz KHNrYik7CkBAIC00NzcsNyArNDc3LDcgQEAgc3RhdGljIGludCBjYWtlX2VucXVldWUoc3RydWN0 IHNrX2J1ZmYgKnNrYiwgc3RydWN0IFFkaXNjICpzY2gpCiAJLyogZmxvd2NoYWluICovCiAJaWYo bGlzdF9lbXB0eSgmZmxvdy0+Zmxvd2NoYWluKSkgewogCQlsaXN0X2FkZF90YWlsKCZmbG93LT5m bG93Y2hhaW4sICZmcWNkLT5uZXdfZmxvd3MpOwotCQlhdG9taWNfaW5jKCZmcWNkLT5mbG93X2Nv dW50KTsKKwkJZnFjZC0+Zmxvd19jb3VudCs9MTsKIAkJZmxvdy0+ZGVmaWNpdCA9IGZxY2QtPnF1 YW50dW07CiAJCWZsb3ctPmRyb3BwZWQgPSAwOwogCX0KQEAgLTYxNSw3ICs2MTUsNyBAQCByZXRy eToKIAkJCWxpc3RfbW92ZV90YWlsKCZmbG93LT5mbG93Y2hhaW4sICZmcWNkLT5vbGRfZmxvd3Mp OwogCQl9IGVsc2UgewogCQkJbGlzdF9kZWxfaW5pdCgmZmxvdy0+Zmxvd2NoYWluKTsKLQkJCWF0 b21pY19kZWMoJmZxY2QtPmZsb3dfY291bnQpOworCQkJZnFjZC0+Zmxvd19jb3VudC09MTsKIAkJ fQogCQlnb3RvIGJlZ2luOwogCX0KQEAgLTk2Niw3ICs5NjYsNyBAQCBzdGF0aWMgdm9pZCBjYWtl X3JlY29uZmlndXJlKHN0cnVjdCBRZGlzYyAqc2NoKQogCQlpZihxLT5idWZmZXJfbGltaXQgPCA2 NTUzNikKIAkJCXEtPmJ1ZmZlcl9saW1pdCA9IDY1NTM2OwogCi0JCXEtPnBlZWxfdGhyZXNob2xk ID0gKHEtPnJhdGVfZmxhZ3MgJiBDQUtFX0ZMQUdfQVRNKSA/IDAgOiBtaW4oNjU1MzVVLCBxLT5y YXRlX2JwcyA+PiAxMCk7CisJCXEtPnBlZWxfdGhyZXNob2xkID0gKHEtPnJhdGVfZmxhZ3MgJiBD QUtFX0ZMQUdfQVRNKSA/IDAgOiBtaW4oNjU1MzVVLCBxLT5yYXRlX2JwcyA+PiAxMik7CiAJfSBl bHNlIHsKIAkJcS0+YnVmZmVyX2xpbWl0ID0gMSA8PCAyMDsKIAkJcS0+cGVlbF90aHJlc2hvbGQg PSAwOwpAQCAtMTA4Myw3ICsxMDgzLDcgQEAgc3RhdGljIGludCBjYWtlX2luaXQoc3RydWN0IFFk aXNjICpzY2gsIHN0cnVjdCBubGF0dHIgKm9wdCkKIAkJZnFjZC0+cGVydHVyYmF0aW9uID0gcHJh bmRvbV91MzIoKTsKIAkJSU5JVF9MSVNUX0hFQUQoJmZxY2QtPm5ld19mbG93cyk7CiAJCUlOSVRf TElTVF9IRUFEKCZmcWNkLT5vbGRfZmxvd3MpOwotCQlhdG9taWNfc2V0KCZmcWNkLT5mbG93X2Nv dW50LCAwKTsKKwkJZnFjZC0+Zmxvd19jb3VudCA9IDA7CiAJCS8qIGNvZGVsX3BhcmFtc19pbml0 KCZmcWNkLT5jcGFyYW1zKTsgKi8KIAogCQlmcWNkLT5mbG93cyAgICA9IGNha2VfemFsbG9jKGZx Y2QtPmZsb3dzX2NudCAqIHNpemVvZihzdHJ1Y3QgY2FrZV9mcWNkX2Zsb3cpKTsKLS0gCjEuOS4x Cgo= --001a11c306eed6a30e0519aacd28--