From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-x231.google.com (mail-ob0-x231.google.com [IPv6:2607:f8b0:4003:c01::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DBA763B260 for ; Thu, 5 May 2016 13:27:36 -0400 (EDT) Received: by mail-ob0-x231.google.com with SMTP id aq1so34130400obc.3 for ; Thu, 05 May 2016 10:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-transfer-encoding; bh=eA0kc4V8wYME+OqNDJJiXylhzhW4LLNMMuIiB/3FPR8=; b=siSEQQnzAcmIuvcApg+mMB6TezqLLIApRTdPfUhkFxpcAK/sKpjCCATWFA5P29EmvV NyQjuuAe2nPas95X0moq/mHuaWYDyDmHvvQFvYa9WgZhCa2/tFxNtZhSL5s3wGOqc5eR BgirJdLEdjXW9L8l8Ukyni8CiHSuJFfNpEi+7aY5LdlqJXEl/geUS22ToIku0ZSR+hiy CU39P3FHtupzetqd7UoNs1DFAsF6TAhdTvrYoeKQKwKc1X8OutVRl9zTMvNkj6ajCcT5 dZYVNR4M2osqLVxoB0qQkKxcKflxq77QJ3s9bmh5oJ0wyNqUj6Ow86oktgV4zGuGjn1W HAcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-transfer-encoding; bh=eA0kc4V8wYME+OqNDJJiXylhzhW4LLNMMuIiB/3FPR8=; b=LwXaKV+5326XgklYRlq6iebFMtcZ3ldHxtCO6hIjNl8L6WFGEtTf+wQA89KMJlhtJ/ 3+tjBoD27XAybiCuROLMhgc7B9xswOLaLfjdP4td7tNc6OPHUXeUNHzups+IWPzS/EMn 7Kkdhe60JGTjqMCu1K700xce2MD86YZU3nsJSqNN1fB5XKs8U15yQpoP50gbAbnDr23o kpn5F/KLn1Xt0bsxpEYZdt30rTsnR47EYiV5f6HdXSK9wm+ZFuHDO4JAgv1eodctJpme bSEiVAYMkESqp6TqeHZqohXTusUe/a0hzm2Q2CRoobkkXWbshhDOUWt+7nIBobU4fYvS Lb/w== X-Gm-Message-State: AOPr4FUX+BDsNeQG8WZuJ5l6n9RmQtLLbfM/bAMaI1go1pDEHDQ6Mo6MdF0PZc2yUOkc1HsutRK1dadHDo4AHw== MIME-Version: 1.0 X-Received: by 10.60.52.177 with SMTP id u17mr5957568oeo.61.1462469256213; Thu, 05 May 2016 10:27:36 -0700 (PDT) Received: by 10.202.81.76 with HTTP; Thu, 5 May 2016 10:27:36 -0700 (PDT) In-Reply-To: References: Date: Thu, 5 May 2016 10:27:36 -0700 Message-ID: From: Dave Taht To: Aaron Wood Cc: make-wifi-fast@lists.bufferbloat.net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Make-wifi-fast] hacking on the candelatech and qca ath10k firmware X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 May 2016 17:27:37 -0000 On Thu, May 5, 2016 at 10:05 AM, Aaron Wood wrote: > I think you might be mis-reading the box-plots as error-bars (since their > quartile plots). I'll need to crunch the numbers, but I'm pretty sure th= at > the fq results are going to show a higher median throughput (and lower > median latency), with a fair bit of significance. I'll see if I can figu= re > out how to calculate the SD of the mean (and other quartiles) from the fl= ent > output (I have scripts that can do this for iperf3's json output). Thanks in advance!!! I hate box plots honestly. They often lie. I'd rather look at a detailed time series first, and the box plot *only* after I verified that that was sane. And I'm not good at reading box plots right! Tthat said, what I meant by error bars was that I mentally disregard any eyeball comparison variance of ~10% as a possible artifact of the usually single or dual test, and rely on doing extensive, repeated and/or long term tests to get that down to significance. Eventually. After all the bugs are out. Toke uses 30 tests in a row to get somewhere, which takes weeks, so I fly by the seat of my pants in this way for as long as I can. I've had so many cases where I'd look at a box plot and not understand what was going on. The one that sticks in my memory best (never got around to writing it up though) were the ones where we were dealing with the unaligned access bugs in tcp on cerowrt. We'd see overall throughput drop by like 20% for ipv6 vs ipv4 in the box plots. We'd see periodic total losses in throughput on the detailed time based ones. Here's another case where box plots lie, showing the impact of "something" every 2 minutes: see second plot on: http://blog.cerowrt.org/post/cs5_lockout/ I'd rather look at a https://en.wikipedia.org/wiki/Seven-number_summary in terms of box plot. A howto or lecture on how to better interpret various flent tests would be nice to do up, I don't think it's clear to many people how the width of the sawtooths on most of the flent tests relative to the direct latency measurement still show the effectiveness of the underlying AQM even with fq in place, because the relate better in a single queue aqm - I've seen so many of the aqm alone vs fq+aqm plots - and also the backlog plots which I more rarely collect and publish - that I just filter them into a mental something that works. People using box plots exclusively to analyze tcp throughput are on drugs. I have consciously focused on doing plots rather than reporting single number results like "Got 110Mbits throughput! Ship it", 'cause that doesn't show the sawtooth. This message brought to you by the "Society to Save the Sawtooth" > > -Aaron > > On Thu, May 5, 2016 at 12:09 AM, Dave Taht wrote: >> >> see: http://blog.cerowrt.org/post/ath10_ath9k_1/ >> >> the regular qca firmware survived the rrul better, and seemed to do >> wmm better. (CS6 for example, was fine) Aside from that it was slower >> and more jittery than the candelatech firmware. some pics there. Am >> too tired to write it up right now. >> >> https://github.com/dtaht/blog-cerowrt/tree/master/content/flent/qca-10.2 >> >> I guess I gotta go boot into baseline kernels now and pray I haven't >> been deluding myself at these speeds. For all I know everything is >> actually better with those than all these patches. >> >> night >> >> -- >> Dave T=C3=A4ht >> Let's go make home routers and wifi faster! With better software! >> http://blog.cerowrt.org >> _______________________________________________ >> Make-wifi-fast mailing list >> Make-wifi-fast@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/make-wifi-fast > > --=20 Dave T=C3=A4ht Let's go make home routers and wifi faster! With better software! http://blog.cerowrt.org