[Bloat] What is fairness, anyway? was: Re: finally... winning on wired!

[Dave Taht]

On Fri, Jan 13, 2012 at 10:45 PM, Dan Siemon <dan at coverfire.com> wrote:
> On Sun, 2012-01-08 at 01:40 +0100, Dave Taht wrote:
>> On Thu, Jan 5, 2012 at 6:52 PM, Bob Briscoe <bob.briscoe at bt.com> wrote:
>> >
>> > In a nutshell, bit-rate equality, where each of N active users gets 1/N of
>> > the bit-rate, was found to be extremely _unfair_ when the activity of
>> > different users is widely different. For example:
>> > * 5 light users all active 1% of the time get close to 100% of a shared link
>> > whenever they need it.
>> > * However, if instead 2 of these users are active 100% of the time, FQ gives
>> > the other three light users only 33% of the link whenever they are active.
>> > * That's pretty rubbish for a solution that claims to isolate each user from
>> > the excesses of others.
>>
>> Without AQM or FQ, we have a situation where one stream from one user
>> at a site, can eat more than 100% of the bandwidth.
>>
>> 1/u would be a substantial improvement!
>
> Indeed I've found this to be the case. I've been using a Linux tc
> configuration in both the upstream and downstream which is designed to
> protect each host's bandwidth share and within that provide three
> traffic classes with flow fairness (script link below). With this
> configuration I no longer have to worry about other network traffic
> interfering with a decent web experience or VoIP call.
>
> http://git.coverfire.com/?p=linux-qos-scripts.git;a=blob;f=src-3tos.sh;hb=HEAD

In looking over your script I see there will be several improvements
in linux 3.3 that will apply. I'd be very interested in A/B results on
this script between current and next linux.

SFQ used to permute it's hash and 'scramble' a stream by default every
10 seconds. A 10 second periodicity of tcp weirdness was often visible
on a tcpdump/tcptrace/xplot.org when multiple streams were in play.
Now SFQ permutes the hash in the background, not permuting the hash
until after all the packets that applied to it have been delivered.

SFQ used to add new streams to the tail of the queue, now it adds them
to the head.

There are a zillion more options to SFQ now, deeper buckets, more
flows, etc - and the hybrid combination of REDSFQ. The former stuff
can lower or eliminate the need for perturbation (and or make
perturbation very costly), and handle higher and more diverse
workloads, the latter, like, does it all. I don't even know on where
to start on describing it.

Lastly - with the new stuff SFQ or QFQ - I have very rarely had a need
to explicitly increase the priority of packets using the TOS IMM field
on my workloads, however you are running at rates below what I've been
working with, so that's interesting.

(basically the more 'sparse' a flow is, the faster it leaps to the
front of the queues anyway. Interactive ssh flows fit in that category
well)

So for me the only gateway'd high priority packets are marked EF
(voip) and sometimes dns... and most of the time, none...

Secondly I can imagine decreasing the priority for bulk packets in the
way you are would help, particularly with torrents.

>
> --
> Key ID: 133F6C3E
> Key Fingerprint: 72B3 AF04 EFFE 65E4 46FF  7E5B 9297 18BA 133F 6C3E
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net