[Bloat] SFB tuning

Sun May 29 20:52:44 EDT 2011

On Sun, May 29, 2011 at 05:29:10PM +0200, Juliusz Chroboczek wrote:
> > Internet (100Mbps ethernet capped to 70Mbps by the ISP):
> >
> > tc qdisc add dev eth4 parent 1:3  handle 13:  sfb hash-type source limit 100 target 10 max 15 penalty_rate 60
> >
> > And on the internal interfaces (1Gbps ethernet) to clients like this:
> >
> > tc qdisc add dev ${DEV} parent 50:20 handle 52: sfb hash-type dest limit 100 target 10 max 15 penalty_rate 100
> 
> Looks good to me.

Thank you!

> You may want to increase limit and penalty -- remember that these values
> are shared between all clients.  (Are you seeing any queuedrop?)

Ok will increase that, when I discover bufferbloat in my interfaces
(all of them with pfifo_fast and txqueuelen 4000 plus max in tx ring)
I reduced to min in tx ring and to 10 in txqueuelen with very good
results but now with proper AQM I think I could increase all buffers
for good?

> You may also want to experiment with increasing increment/decrement --
> increment should be roughly 5 times larger than decrement, and the
> values should be as large as you can make them without seeing
> oscillations.  (Larger values yield faster convergence, but may cause
> overshoot.)

Ok, will do that.

> You may also want to encourage your clients to enable ECN.

Servers have it enabled, but I discover that I clear in my routers
iptables the DSCP field as you acurately mention it in another post,
for clients it's very hard as it's a very large network in my
University.

> May I see the output of ``tc -s qdisc show''?

Sure, just note that my current 'QoS' scheme involves some classes for
realtime, others for shaping and lastly SFB for fairness.

In advace thank you for taking a look to all this data that my
brain can't cope :)

eth4 (Internet 70Mbps):
qdisc prio 1: root refcnt 2 bands 3 priomap  2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 Sent 11135566140550 bytes 1231458943 pkt (dropped 7284219, overlimits 0 requeues 2683938) 
 rate 0bit 0pps backlog 0b 0p requeues 2683938 
qdisc sfb 13: parent 1:3 hash source limit 100 max 15 target 10
  increment 0.00050 decrement 0.00005 penalty rate 60 burst 20 (600s 60s 6x32)
 Sent 10466924374664 bytes 3212112668 pkt (dropped 5891604, overlimits
3098159 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
  earlydrop 2793445 penaltydrop 0 bucketdrop 3098159 queuedrop 0 marked 555
  maxqlen 0 maxprob 0.00000
qdisc sfq 131: parent 13: limit 127p quantum 1514b flows 127/1024 perturb 10sec 
 Sent 10466924365580 bytes 3212112662 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 

eth2 (wired LAN 1Gbps)
qdisc prio 1: root refcnt 2 bands 5 priomap  4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
 Sent 2483998367057 bytes 2866500587 pkt (dropped 6910353, overlimits 0 requeues 75576) 
 rate 0bit 0pps backlog 0b 0p requeues 75576 
qdisc htb 50: parent 1:5 r2q 25 default 30 direct_packets_stat 0 ver 3.17
 Sent 680549233437 bytes 496435880 pkt (dropped 6832111, overlimits 506216214 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc sfb 52: parent 50:20 hash dest limit 100 max 15 target 10
  increment 0.00050 decrement 0.00005 penalty rate 100 burst 20 (600s 60s 6x32)
 Sent 476083684999 bytes 335250357 pkt (dropped 5167392, overlimits 1190208 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
  earlydrop 3977184 penaltydrop 0 bucketdrop 1190208 queuedrop 0 marked 416
  maxqlen 0 maxprob 0.00000
qdisc sfq 521: parent 52: limit 127p quantum 9014b flows 127/1024 perturb 10sec 
 Sent 476083684999 bytes 335250357 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc sfb 53: parent 50:30 hash dest limit 100 max 15 target 10
  increment 0.00050 decrement 0.00005 penalty rate 100 burst 20 (600s 60s 6x32)
 Sent 165013253442 bytes 125931567 pkt (dropped 1615538, overlimits 229717 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
  earlydrop 1385821 penaltydrop 0 bucketdrop 229717 queuedrop 0 marked 0
  maxqlen 0 maxprob 0.00000
qdisc sfq 531: parent 53: limit 127p quantum 9014b flows 127/1024 perturb 10sec 
 Sent 165013253182 bytes 125931564 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 

eth6 (wireless 802.11bgn)
qdisc prio 1: root refcnt 2 bands 5 priomap  4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
 Sent 6677917818544 bytes 1754047115 pkt (dropped 63616539, overlimits 0 requeues 2993) 
 rate 0bit 0pps backlog 0b 0p requeues 2993 
qdisc htb 50: parent 1:5 r2q 25 default 30 direct_packets_stat 0 ver 3.17
 Sent 5139577338863 bytes 3714353622 pkt (dropped 63614459, overlimits 967897712 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc sfb 52: parent 50:20 hash dest limit 100 max 15 target 10
  increment 0.00050 decrement 0.00005 penalty rate 100 burst 20 (600s 60s 6x32)
 Sent 3036033504418 bytes 2069667963 pkt (dropped 54627811, overlimits 11816962 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
  earlydrop 42810849 penaltydrop 0 bucketdrop 11795286 queuedrop 0 marked 4918
  maxqlen 0 maxprob 0.00000
qdisc sfq 521: parent 52: limit 127p quantum 1514b flows 127/1024 perturb 10sec 
 Sent 3036033504418 bytes 2069667963 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
qdisc sfb 53: parent 50:30 hash dest limit 100 max 15 target 10
  increment 0.00050 decrement 0.00005 penalty rate 100 burst 20 (600s 60s 6x32)
 Sent 973169683987 bytes 746581178 pkt (dropped 5858989, overlimits 1292169 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
  earlydrop 4566820 penaltydrop 0 bucketdrop 1287799 queuedrop 0 marked 18884
  maxqlen 0 maxprob 0.00000
qdisc sfq 531: parent 53: limit 127p quantum 1514b flows 127/1024 perturb 10sec 
 Sent 973169683987 bytes 746581178 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 

Thank you!
-
 Otto