[Bloat] AQM creeping into L2 equipment

[Dave Taht]

On Fri, Mar 21, 2014 at 5:51 PM, Eric Dumazet <eric.dumazet at gmail.com> wrote:
> On Fri, 2014-03-21 at 16:53 +0100, renaud sallantin wrote:
>> For our tests,  we needed to adjust the "tcp_initial_quantum" in the
>> FQ,
>> but as you said, it is just a FQ parameter.
>>
>
> Yep, default ones are a compromise between performance and pacing
> accuracy. At 40Gbps speeds, it is a bit challenging.

At sub 10Mbit speeds it is also challenging.

> The consensus is that IW10 is adopted, meaning that we can send 10 MSS
> at whatever speed we want without knowing anything of the network
> conditions.

And I do wish that research and decision included measurements of the
effects on DSL (sub 1mbit) and cable modem (sub 10mbit) uplinks and
uploads, and tcp using
traffic like bittorrent, ssh, rsync, cifs, etc.

>
> If people want to play with other values, they have to change the route
> settings of their linux box, and fq parameters if they want.
>
>
> ip ro change default via 192.168.1.254 dev eth0 initcwnd 20

This is likely to have brutal effects on slow uplinks and uploads
without pacing enabled.

to get results semi comparable to (for example) OSX, a initcwnd 3
should be used.

I DO have very high hopes for pacing in these cases at various iw settings
in the case of slow uplinks and uploads, I am concerned about the effects
on wifi and other burst-prefering macs.

I look forward to more benchmarks. I'm not in a position to do much with
kernels later than 3.10 at the moment...

> (As a matter of fact it seems some providers use higher values than
> IW10)
>
>> The "patch" we added, and once again, it was just a  few lines,
>> enabled to set, via a sysctl parameter, the initial pacing value,
>> regardless of the RTT.
>> This can be valuable for different reasons:
>>      o  In case of long RTT, not set the pacing value is going to
>> introduce an un-necessary delay
>> (we aims to use this mechanism for satcom, so the delay could be
>> greater than 500ms)
>
> If you have a 500ms rtt, then you also want a bigger IW. Sending 10 MSS
> in the first RTT is going to be slow, no matter how you pace them.
> The first ACK wont come before 500 ms.

In the case of a satellite link (>800ms RTT) with competing traffic,
it is my hope
that (n)fq_codel will further mitigate the large iws common today. A
fairly common
satellite uplink/downlink is 500kbit/8mbit.

>>      o  In case of a wrong RTT measurement, i.e. an RTT measurement
>> that is higher that the real RTT (because of congestion for example),
>>  you are going to have a wrong pacing evaluation...
>
> Well, if you have big rtt because of congestion, you exactly want to
> reduce the rate...
>
> rate = cwnd * mss / srtt

A nice thing about fq_codel inbetween on the congested link is that
your first RTT on a new flow is generally very close to your actual
physical RTT between links even when congested.

> And fq/pacing uses srtt, not rtt, so a single wrong rtt doesn't have a
> big impact (unless it is the first sample, as it will serve as the ewma
> initial value)
>
> You can not predict the network conditions just by studying the
> SYN/SYNACK/ACK initial messages. It gives a guess, but it is hard to
> send everything you want in a single RTT at 'optimal speed'
>
> Thats why it was so hard to decide the IW if you want an universal
> value.
> It depends on the state of the Internet, and it changes every day or
> so...

I kind of wish there was a way to propagate saner IW settings throughout
a home network - where you can use a big IW internally, but downgrade to
lower values on exit to the real world.

Again, if pacing actually works maybe indeed larger iws are truly feasible...

(If I sound grumpy - I just wish I had time to play with this new stuff
instead of what I'm doing right now - it's promising as heck)

>
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html