From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x244.google.com (mail-qt0-x244.google.com [IPv6:2607:f8b0:400d:c0d::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 823503BA8E for ; Wed, 25 Jul 2018 17:03:49 -0400 (EDT) Received: by mail-qt0-x244.google.com with SMTP id m13-v6so9133810qth.1 for ; Wed, 25 Jul 2018 14:03:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=vfMcBzgTKMdOc2t28iH7cQPZatlW3LHXB5Fz3vSf2Fo=; b=TIwjyg9WTJYPyNXbkH0tC3zOHs2Oiidibzm8+nypZw/sCJLp/mC4jckSP4CMmitMim d4AidnsXFbuJ60VUlFZNwpdRJo+ZeKxxvA1fgfGmFoImbBonuwSL2N4yuf+Nb1Tf85K3 52BUyY5lTQo1v0n8oxEujKquRQGRUYIkvPOtM6JYDOB9xf0E94o6pqTakWeSC8keN1/7 1gR63CdNb95mWCIactv/A4pkS6/dVJn4CMnjulCPEPvqJ95fqW40p6E1AQaB6AsOrltM GCZqS1bgd9Ub+75ykG23/XtSKVuRXYyTiMVsxoOaESc1jMLJNGKTBJp/LazwfyoujQrr 4cyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=vfMcBzgTKMdOc2t28iH7cQPZatlW3LHXB5Fz3vSf2Fo=; b=RHf/v6CNDrLS/ouwqS4RsLurExOI+neOnBWqCMCEGbm3qKQqZYYGfscp3T2gH5Q4yx KTH8IEAyN0EJgD0zCKn0VF89gFArZfvoUT6uJ08xizS62dJyyNI8FzVXXOKCMpMTNeaa sQve8txLYTNqBKkZqt64aZj2eQ2VnNfIBwfVAvpBPMGjVUkomdfK0QoIKpJpuotLoooF gvjDGDf9K2IwzamfH4I6n9sx5Rd32ZPYlZ5zl2WR+n/wqCy9WJs9u6la23nsIHVYaUrt i/9e6eKFw1gArrF+eGCzT+uhlban70X17ez+cR6ejjBCkgFtvAKjEAx72LgR/Bqfa43P ibew== X-Gm-Message-State: AOUpUlHhFe8spo/r2ZmTPOdolm8s0D6lu9ZDt8lUJ2AZZM+A07oLtCyS TYGnR4mv72Vynucq92oapMHas/pIoBfJdmJG1q34ew== X-Google-Smtp-Source: AAOMgpdpg0yMtq98nZQXqV63dLvB5ROhL4gGrBxObnWkhmFnCzWt+qdxnELnJMTDyks9O9GOSHzBzh8+UjvG4ftIlgc= X-Received: by 2002:ac8:266b:: with SMTP id v40-v6mr22715824qtv.104.1532552628690; Wed, 25 Jul 2018 14:03:48 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Wed, 25 Jul 2018 14:03:37 -0700 Message-ID: To: Cake List Content-Type: multipart/mixed; boundary="0000000000000882410571d93721" Subject: Re: [Cake] quick patch for split_gso X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2018 21:03:49 -0000 --0000000000000882410571d93721 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable so, I went and tested that patch. Turned out len was signed... I'd put it in the wrong place... and the very act of doing that test at all permutes the result. cake unshaped at 1gig, always splitting, ends up with a bql value of about 42,000, full throughput, e.g. if (skb_is_gso(skb)) { ... avg median # data pts Ping (ms) ICMP : 0.80 0.77 ms 341 TCP upload avg : 235.40 235.39 Mbits/s 301 TCP upload sum : 941.61 941.55 Mbits/s 301 TCP upload::1 : 235.36 235.39 Mbits/s 290 TCP upload::2 : 235.46 235.39 Mbits/s 291 TCP upload::3 : 235.38 235.45 Mbits/s 275 TCP upload::4 : 235.41 235.44 Mbits/s 278 conditionally splitting, either never or always, gives me bql of: 140852 if (skb_is_gso(skb) && len >=3D q->split_gso) { avg median # data pts Ping (ms) ICMP : 1.64 1.71 ms 347 TCP upload avg : 235.40 235.38 Mbits/s 301 TCP upload sum : 941.59 941.52 Mbits/s 301 TCP upload::1 : 235.35 235.44 Mbits/s 280 TCP upload::2 : 235.40 235.39 Mbits/s 276 TCP upload::3 : 235.47 235.43 Mbits/s 276 TCP upload::4 : 235.37 235.45 Mbits/s 282 the cache line miss can't possibly have cost that much. (?) Guess I have to go check the range of len. I WANT MY 800usec back! PS For giggles, I accidentally ended up using pfifo_fast for one run. I thought I'd broke something. Summary of tcp_nup test run 'split_gso_test_again_len_512000_bql_auto' (at 2018-07-25 20:51:10.533339): avg median # data pts Ping (ms) ICMP : 3.32 3.59 ms 347 TCP upload avg : 235.38 235.38 Mbits/s 301 TCP upload sum : 941.52 941.54 Mbits/s 301 TCP upload::1 : 152.48 150.93 Mbits/s 281 TCP upload::2 : 152.54 150.35 Mbits/s 286 TCP upload::3 : 483.96 491.12 Mbits/s 284 TCP upload::4 : 152.54 150.49 Mbits/s 284 On Tue, Jul 24, 2018 at 10:01 PM Dave Taht wrote: > > Ah... 1gbit, BQL at a sane size, 750 usec RTT. I guess I have to go > compile this patch now..... > > > -- > > Dave T=C3=A4ht > CEO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-669-226-2619 --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619 --0000000000000882410571d93721 Content-Type: application/x-patch; name="cake_split_gso.patch" Content-Disposition: attachment; filename="cake_split_gso.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_jk1m7se50 ZGlmZiAtLWdpdCBhL25ldC9zY2hlZC9zY2hfY2FrZS5jIGIvbmV0L3NjaGVkL3NjaF9jYWtlLmMK aW5kZXggNTM5Yzk0OS4uNDg4Y2ZjNiAxMDA2NDQKLS0tIGEvbmV0L3NjaGVkL3NjaF9jYWtlLmMK KysrIGIvbmV0L3NjaGVkL3NjaF9jYWtlLmMKQEAgLTgwLDcgKzgwLDYgQEAKICNkZWZpbmUgQ0FL RV9RVUVVRVMgKDEwMjQpCiAjZGVmaW5lIENBS0VfRkxPV19NQVNLIDYzCiAjZGVmaW5lIENBS0Vf RkxPV19OQVRfRkxBRyA2NAotI2RlZmluZSBDQUtFX1NQTElUX0dTT19USFJFU0hPTEQgKDEyNTAw MDAwMCkgLyogMUdicHMgKi8KIAogLyogc3RydWN0IGNvYmFsdF9wYXJhbXMgLSBjb250YWlucyBj b2RlbCBhbmQgYmx1ZSBwYXJhbWV0ZXJzCiAgKiBAaW50ZXJ2YWw6CWNvZGVsIGluaXRpYWwgZHJv cCByYXRlCkBAIC0xNjQsNiArMTYzLDcgQEAgc3RydWN0IGNha2VfdGluX2RhdGEgewogCXUxNgl1 bnJlc3BvbnNpdmVfZmxvd19jb3VudDsKIAogCXUzMgltYXhfc2tibGVuOworCXUzMglzcGxpdF9n c287CiAKIAlzdHJ1Y3QgbGlzdF9oZWFkIG5ld19mbG93czsKIAlzdHJ1Y3QgbGlzdF9oZWFkIG9s ZF9mbG93czsKQEAgLTI2MCw3ICsyNjAsNiBAQCBlbnVtIHsKIAlDQUtFX0ZMQUdfQVVUT1JBVEVf SU5HUkVTUyA9IEJJVCgxKSwKIAlDQUtFX0ZMQUdfSU5HUkVTUwkgICA9IEJJVCgyKSwKIAlDQUtF X0ZMQUdfV0FTSAkJICAgPSBCSVQoMyksCi0JQ0FLRV9GTEFHX1NQTElUX0dTTwkgICA9IEJJVCg0 KQogfTsKIAogLyogQ09CQUxUIG9wZXJhdGVzIHRoZSBDb2RlbCBhbmQgQkxVRSBhbGdvcml0aG1z IGluIHBhcmFsbGVsLCBpbiBvcmRlciB0bwpAQCAtMTY1MSw3ICsxNjUwLDcgQEAgc3RhdGljIHMz MiBjYWtlX2VucXVldWUoc3RydWN0IHNrX2J1ZmYgKnNrYiwgc3RydWN0IFFkaXNjICpzY2gsCiAJ aWYgKHVubGlrZWx5KGxlbiA+IGItPm1heF9za2JsZW4pKQogCQliLT5tYXhfc2tibGVuID0gbGVu OwogCi0JaWYgKHNrYl9pc19nc28oc2tiKSAmJiBxLT5yYXRlX2ZsYWdzICYgQ0FLRV9GTEFHX1NQ TElUX0dTTykgeworCWlmIChza2JfaXNfZ3NvKHNrYikgJiYgcS0+c3BsaXRfZ3NvIDwgbGVuICkg ewogCQlzdHJ1Y3Qgc2tfYnVmZiAqc2VncywgKm5za2I7CiAJCW5ldGRldl9mZWF0dXJlc190IGZl YXR1cmVzID0gbmV0aWZfc2tiX2ZlYXR1cmVzKHNrYik7CiAJCXVuc2lnbmVkIGludCBzbGVuID0g MDsKQEAgLTI1NjksMTAgKzI1NjgsOCBAQCBzdGF0aWMgaW50IGNha2VfY2hhbmdlKHN0cnVjdCBR ZGlzYyAqc2NoLCBzdHJ1Y3QgbmxhdHRyICpvcHQsCiAJaWYgKHRiW1RDQV9DQUtFX01FTU9SWV0p CiAJCXEtPmJ1ZmZlcl9jb25maWdfbGltaXQgPSBubGFfZ2V0X3UzMih0YltUQ0FfQ0FLRV9NRU1P UlldKTsKIAotCWlmIChxLT5yYXRlX2JwcyAmJiBxLT5yYXRlX2JwcyA8PSBDQUtFX1NQTElUX0dT T19USFJFU0hPTEQpCi0JCXEtPnJhdGVfZmxhZ3MgfD0gQ0FLRV9GTEFHX1NQTElUX0dTTzsKLQll bHNlCi0JCXEtPnJhdGVfZmxhZ3MgJj0gfkNBS0VfRkxBR19TUExJVF9HU087CisJaWYgKHRiW1RD QV9DQUtFX1NQTElUX0dTT10pCisJCXEtPnNwbGl0X2dzbyA9IG5sYV9nZXRfdTMyKHRiW1RDQV9D QUtFX1NQTElUX0dTT10pOwogCiAJaWYgKHEtPnRpbnMpIHsKIAkJc2NoX3RyZWVfbG9jayhzY2gp OwpAQCAtMjYxMSw3ICsyNjA4LDcgQEAgc3RhdGljIGludCBjYWtlX2luaXQoc3RydWN0IFFkaXNj ICpzY2gsIHN0cnVjdCBubGF0dHIgKm9wdCwKIAogCXEtPmN1cl90aW4gPSAwOwogCXEtPmN1cl9m bG93ICA9IDA7Ci0KKwlxLT5zcGxpdF9nc28gPSB+MDsgLyogYWx3YXlzIHNwbGl0IGdzbyBwYWNr ZXRzICovCiAJcWRpc2Nfd2F0Y2hkb2dfaW5pdCgmcS0+d2F0Y2hkb2csIHNjaCk7CiAKIAlpZiAo b3B0KSB7CkBAIC0yNzI5LDggKzI3MjYsNyBAQCBzdGF0aWMgaW50IGNha2VfZHVtcChzdHJ1Y3Qg UWRpc2MgKnNjaCwgc3RydWN0IHNrX2J1ZmYgKnNrYikKIAlpZiAobmxhX3B1dF91MzIoc2tiLCBU Q0FfQ0FLRV9NUFUsIHEtPnJhdGVfbXB1KSkKIAkJZ290byBubGFfcHV0X2ZhaWx1cmU7CiAKLQlp ZiAobmxhX3B1dF91MzIoc2tiLCBUQ0FfQ0FLRV9TUExJVF9HU08sCi0JCQkhIShxLT5yYXRlX2Zs YWdzICYgQ0FLRV9GTEFHX1NQTElUX0dTTykpKQorCWlmIChubGFfcHV0X3UzMihza2IsIFRDQV9D QUtFX1NQTElUX0dTTywgcS0+c3BsaXRfZ3NvKSkKIAkJZ290byBubGFfcHV0X2ZhaWx1cmU7CiAK IAlyZXR1cm4gbmxhX25lc3RfZW5kKHNrYiwgb3B0cyk7Cg== --0000000000000882410571d93721--