From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yh0-x22a.google.com (mail-yh0-x22a.google.com [IPv6:2607:f8b0:4002:c01::22a]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 2AF2C21F1CF; Thu, 2 Apr 2015 12:03:13 -0700 (PDT) Received: by yhfw71 with SMTP id w71so23178516yhf.2; Thu, 02 Apr 2015 12:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XzCtEcHLnB7oz+fPbKuMk6QsMh6c/cWtZF7K2I9XN1g=; b=bake5YWjIfp2b74Wqlt8J4uFj1ntHkzZv6SdQ34+k31IA34blkOcB5hlSx+oCb7IZZ MgmTW6o4QZg3hLtZtIKHFs2zbDSL+PWUxr/cuTtmiY1QiL2z+GXrs0vad0EiaQunSFtX 2SUYl1Ey5C0Zlw3ZumdQ/UYnMlecRcr9q987bdtZmaoUpcvklYXMSjA8zFFaCCId1GGf Jsi+Lt0NqjmjxnUWLcuHuGGcrr1q2qoxepIZko/ZwSHUmQ7gKtcDuuf1TW13COHa0e5F zyRxm84mQzTfaShsBEpSO+0n5IETe5duKL8ZGBUT8trDIhd+m8ZoRn1mIgEvBNdSZeOT l1bQ== MIME-Version: 1.0 X-Received: by 10.53.7.72 with SMTP id da8mr31239163vdd.58.1428001392586; Thu, 02 Apr 2015 12:03:12 -0700 (PDT) Received: by 10.52.240.145 with HTTP; Thu, 2 Apr 2015 12:03:12 -0700 (PDT) Received: by 10.52.240.145 with HTTP; Thu, 2 Apr 2015 12:03:12 -0700 (PDT) In-Reply-To: References: Date: Thu, 2 Apr 2015 22:03:12 +0300 Message-ID: From: Jonathan Morton To: Dave Taht Content-Type: multipart/alternative; boundary=001a1134c9fcbe75ad0512c27bcd Cc: codel@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] =?utf-8?q?cake3_vs_sqm+fq=5Fcodel_at_115/12_mbit_?= =?utf-8?q?=28basically_comcast=C2=B4s_blast_service=29?= X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 19:03:42 -0000 --001a1134c9fcbe75ad0512c27bcd Content-Type: text/plain; charset=UTF-8 Awesome. Oddly enough, cake3 actually gets slightly less throughput than htb+fq_codel on the Pentium-MMX. However that's with the simplest possible htb configuration (since I'm manually typing it in), and no firewall rules or NAT going on (just a bridge between two Ethernet ports). A couple of notes on the statistics that are now reported: The rate for each class is now a threshold rather than a limit. The class is permitted to use more than that bandwidth (up to the global limit), but will yield to lower priority classes in that condition. This is consistent with both user expectations and standard PHB specs, and means that traffic benefits from high priority markings only if it's appropriately sparse. On that note, I expect roughly the filtering uses of each class: 0 - background bulk traffic, CS1 marked, ie. BitTorrent. Use as many parallel connections as you like, without worrying about ordinary traffic. 1 - best effort, the great majority of ordinary traffic - web pages, software updates, whatever. If in doubt, leave it here (default CS0 lands here). 2 - elevated priority, bandwidth sensitive traffic, such as streaming video or a vlan. 3 - low volume, latency sensitive traffic such as VoIP, online games, NTP, etc. EF traffic lands here. A minor frustration for me here - firewall rules on ingress are processed only after the traffic has already passed through ifb. This means I can't custom mark my inbound traffic. Three delay statistics are now reported, all of which are based on EWMAs of packet sojourn times at dequeue. Pk is biased heavily to high delays (so should usually report on fat flows), Sp to low delays (so should capture sparse flows), and Av keeps a true average. The concept of a biased EWMA is borrowed from ReplayGain and the whole "loudness war" problem that it aims to solve; some broadcast studios (including the BBC) use audio meters which work this way. The new set-associative hash function also generates extra statistics. The same 1024 queues are now divided into 128 sets of 8 "ways", and a tag on each queue tracks which flow is presently using it. This allows hash collisions to be resolved in most cases, with limited worst case overhead, greatly improving flow isolation under severely stressed conditions. (It's difficult to provoke this on a home network, but offices may well appreciate this feature.) The "way miss" counter is incremented whenever an empty queue's tag is changed to assign it to a new flow, signalling a departure from the fast path for that packet. Expect to see a small percentage of these with normal traffic. The "way indirect hit" counter tracks the situations where a hash collision would have occurred with a plain hash function, but was resolved by the set associativity. This is also a departure from the fast path. The "way collision" counter indicates when even set associative hashing is insufficient - there are more than 8 distinct flows attempting to occupy queues in the same set. In such a case, the search for an empty queue is terminated and the packet is placed in the queue matching the plain hash. NB: so far this code path is completely untested to my knowledge! - Jonathan Morton --001a1134c9fcbe75ad0512c27bcd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Awesome.

Oddly enough, cake3 actually gets slightly less throughput t= han htb+fq_codel on the Pentium-MMX. However that's with the simplest p= ossible htb configuration (since I'm manually typing it in), and no fir= ewall rules or NAT going on (just a bridge between two Ethernet ports).

A couple of notes on the statistics that are now reported:

The rate for each class is now a threshold rather than a lim= it. The class is permitted to use more than that bandwidth (up to the globa= l limit), but will yield to lower priority classes in that condition. This = is consistent with both user expectations and standard PHB specs, and means= that traffic benefits from high priority markings only if it's appropr= iately sparse.

On that note, I expect roughly the filtering uses of each cl= ass:

0 - background bulk traffic, CS1 marked, ie. BitTorrent. Use= as many parallel connections as you like, without worrying about ordinary = traffic.

1 - best effort, the great majority of ordinary traffic - we= b pages, software updates, whatever. If in doubt, leave it here (default CS= 0 lands here).

2 - elevated priority, bandwidth sensitive traffic, such as = streaming video or a vlan.

3 - low volume, latency sensitive traffic such as VoIP, onli= ne games, NTP, etc. EF traffic lands here.

A minor frustration for me here - firewall rules on ingress = are processed only after the traffic has already passed through ifb. This m= eans I can't custom mark my inbound traffic.

Three delay statistics are now reported, all of which are ba= sed on EWMAs of packet sojourn times at dequeue. Pk is biased heavily to hi= gh delays (so should usually report on fat flows), Sp to low delays (so sho= uld capture sparse flows), and Av keeps a true average. The concept of a bi= ased EWMA is borrowed from ReplayGain and the whole "loudness war"= ; problem that it aims to solve; some broadcast studios (including the BBC)= use audio meters which work this way.

The new set-associative hash function also generates extra s= tatistics. The same 1024 queues are now divided into 128 sets of 8 "wa= ys", and a tag on each queue tracks which flow is presently using it. = This allows hash collisions to be resolved in most cases, with limited wors= t case overhead, greatly improving flow isolation under severely stressed c= onditions. (It's difficult to provoke this on a home network, but offic= es may well appreciate this feature.)

The "way miss" counter is incremented whenever an = empty queue's tag is changed to assign it to a new flow, signalling a d= eparture from the fast path for that packet. Expect to see a small percenta= ge of these with normal traffic.

The "way indirect hit" counter tracks the situatio= ns where a hash collision would have occurred with a plain hash function, b= ut was resolved by the set associativity. This is also a departure from the= fast path.

The "way collision" counter indicates when even se= t associative hashing is insufficient - there are more than 8 distinct flow= s attempting to occupy queues in the same set. In such a case, the search f= or an empty queue is terminated and the packet is placed in the queue match= ing the plain hash. NB: so far this code path is completely untested to my = knowledge!

- Jonathan Morton

--001a1134c9fcbe75ad0512c27bcd--