I ran a prodigous amount of benchmarks today...

they aint all dead yet. I plan to run the thc toolkit shortly.

<pre>


[12963.671875] Process swapper (pid: 0, threadinfo=8032a000, task=8032f030,
tls=00000000)
[12963.679687] Stack : 80095f54 00000000 8033b4d0 00000029 00000001
00000001 00000080 80340000
[12963.679687]         00000006 8381a800 00000000 8321bc00 8381a800
00000000 8381a800 00000001
[12963.679687]         00000004 00000000 83245f98 82aba07c 00000005
00000001 803a0000 803a0000
[12963.679687]         00200000 800ab560 803a5428 8032bca4 8321bc00
8032dfd0 00000000 8022ea74
[12963.679687]         8032bcd8 00000000 00000000 832b84ec 83245f98
800aac3c 82abae60 00000002
[12963.679687]         ...
[12963.714843] Call Trace:
[12963.718750] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c [nf_defrag_ipv6]
[12963.726562] [<82aba07c>] ipv6_defrag+0x74/0xc8 [nf_defrag_ipv6]
[12963.730468] [<8022ea74>] nf_iterate+0x98/0x104
[12963.734375] [<8022eb78>] nf_hook_slow+0x98/0x170
[12963.742187] [<8324643c>] ipv6_rcv+0x3fc/0x4a0 [ipv6]
[12963.746093] [<80209410>] __netif_receive_skb+0x63c/0x6c0
[12963.750000] [<8298f384>] ri_tasklet+0x104/0x2b0 [ifb]
[12963.757812] [<80079434>] tasklet_action+0x78/0xc8
[12963.761718] [<80078c48>] __do_softirq+0xb0/0x184
[12963.765625] [<80078dcc>] do_softirq+0x48/0x68
[12963.769531] [<80078fe8>] irq_exit+0x4c/0xb8
[12963.773437] [<8006316c>] ret_from_irq+0x0/0x4
[12963.777343] [<80063380>] r4k_wait+0x20/0x40
[12963.781250] [<80064e68>] cpu_idle+0x38/0x70
[12963.785156] [<803488fc>] start_kernel+0x374/0x394
[12963.792968]
[12963.792968]
[12963.792968] Code: 00003021  0c01e3a4  26160008 <8eb40004> 8fa20054
26130018  26f0b0a0  afa20014  afb60018
[13027.007812] Cpu 0
[13027.007812] $ 0   : 00000000 00000000 803a0000 00000100
[13027.007812] $ 4   : 80342e38 82abb0a0 00000000 01820d57
[13027.007812] $ 8   : 00000000 00000000 00000000 00000000
[13027.027343] $12   : 00060000 0014002b 00000000 00000000
[13027.031250] $16   : 8087283e 82007240 821fb240 000004d8
[13027.035156] $20   : 0000002c 80872866 80872846 82ac0000
[13027.042968] $24   : 00000000 82a00b20
[13027.046875] $28   : 8032a000 8032bbb8 80340000 82aba334
[13027.050781] Hi    : 0000001b
[13027.054687] Lo    : 00006c64
[13027.058593] epc   : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.066406]     Tainted: G           O
[13027.070312] ra    : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.078125] Status: 1000fc03    KERNEL EXL IE
[13027.082031] Cause : 00800010
[13027.082031] BadVA : 8087286a
[13027.085937] PrId  : 00019374 (MIPS 24Kc)
[13027.089843] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq
sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred
sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic
act_police act_ipt act_connmark act_skbedit act_mirred em_u32 cls_u32
cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_fq_codel sch_codel
sch_ingress ath79_wdt xt_hashlimit xt_set(O) ip_set_list_set(O)
ip_set_hash_netport(O) ip_set_hash_netiface(O) ip_set_hash_net(O)
ip_set_hash_ipportnet(O) ip_set_hash_ipportip(O) ip_set_hash_ipport(O)
ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O)
ip_set_bitmap_ip(O) ip_set(O) ip6t_REJECT ip6t_rt ip6t_hbh ip6t_mh
ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6table_mangle
ip6table_filter ip6_tables nf_conntrack_ipv6 nf_defrag_ipv6 nfnetlink
nf_nat_irc nf_nat_ftp nf_conntrack_irc nf_conntrack_ftp xt_policy xt_esp
ipt_ah xt_HL xt_hl xt_ecn ipt_ECN xt_CLASSIFY xt_time xt_tcpmss
xt_statistic xt_mark xt_length xt_DSCP xt_dscp xt_string xt_layer7 xt_quota
xt_pkttype xt_physdev xt_owner xt_addrtype ipt_MASQUERADE iptable_nat
xt_nat nf_nat_ipv4 nf_nat xt_recent xt_helper xt_connmark xt_connbytes pptp
pppoe xt_conntrack xt_CT iptable_raw xt_state nf_conntrack_ipv4
nf_defrag_ipv4 nf_conntrack pppox ipt_REJECT xt_TCPMSS xt_LOG xt_comment
xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables
xt_tcpudp x_tables ip_gre gre ifb sit ipcomp xfrm4_tunnel xfrm4_mode_tunnel
xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel4 tun tcp_ledbat(O)
ppp_async ppp_generic slhc af_key xfrm_user xfrm_ipcomp xfrm_algo vfat fat
autofs4 button_hotplug(O) ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O)
nls_utf8 nls_iso8859_2 nls_iso8859_15 nls_iso8859_13 nls_iso8859_1
nls_cp437 mac80211(O) ts_fsm ts_bm ts_kmp crc_ccitt ipv6 input_polldev
cfg80211(O) compat(O) input_core sha1_generic md5 hmac des_generic deflate
cbc authenc arc4 usb_storage ohci_hcd ehci_hcd sd_mod ext4 jbd2 mbcache
usbcore usb_common scsi_mod nls_base crc16 zlib_inflate zlib_deflate
ledtrig_timer ledtrig_default_on leds_gpio gpio_button_hotplug(O)
[13027.273437] Process swapper (pid: 0, threadinfo=8032a000, task=8032f030,
tls=00000000)
[13027.281250] Stack : 8381a800 00000000 000086dd 80340000 80343870
800ae424 00000001 82b54e78
[13027.281250]         00000006 8381a800 00000000 821fb240 8381a800
00000000 8381a800 00000001
[13027.281250]         00000004 00000000 83245f98 82aba07c 8032be18
00000001 831f0400 80340000
[13027.281250]         8381a800 8006316c 80233f00 8032bca4 821fb240
8032dfd0 00000000 8022ea74
[13027.281250]         00000000 00000000 81084078 00400000 83245f98
8381a800 82abae60 8381a800
[13027.281250]         ...
[13027.320312] Call Trace:
[13027.324218] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c [nf_defrag_ipv6]
[13027.328125] [<82aba07c>] ipv6_defrag+0x74/0xc8 [nf_defrag_ipv6]
[13027.335937] [<8022ea74>] nf_iterate+0x98/0x104
[13027.339843] [<8022eb78>] nf_hook_slow+0x98/0x170
[13027.343750] [<8324643c>] ipv6_rcv+0x3fc/0x4a0 [ipv6]
[13027.347656] [<80209410>] __netif_receive_skb+0x63c/0x6c0
[13027.355468] [<8298f384>] ri_tasklet+0x104/0x2b0 [ifb]
[13027.359375] [<80079434>] tasklet_action+0x78/0xc8
[13027.363281] [<80078c48>] __do_softirq+0xb0/0x184
[13027.371093] [<80078dcc>] do_softirq+0x48/0x68
[13027.375000] [<80078fe8>] irq_exit+0x4c/0xb8
[13027.378906] [<8006316c>] ret_from_irq+0x0/0x4
[13027.382812] [<80063380>] r4k_wait+0x20/0x40
[13027.386718] [<80064e68>] cpu_idle+0x38/0x70
[13027.390625] [<803488fc>] start_kernel+0x374/0x394
[13027.394531]
[13027.398437]
[13027.398437] Code: 00003021  0c01e3a4  26160008 <8eb40004> 8fa20054
26130018  26f0b0a0  afa20014  afb60018
[13027.410156] Cpu 0
[13027.410156] $ 0   : 00000000 00000001 803a0000 00000100
[13027.417968] $ 4   : 80342e38 82abb0a0 00000000 01820ea9
[13027.421875] $ 8   : 00000000 00000000 00000000 00000000
[13027.425781] $12   : 00060000 1403f2b2 00000000 00000000
[13027.433593] $16   : 8087083e 8083e540 83217480 000000e8
[13027.437500] $20   : 0000002c 80870866 80870846 82ac0000
[13027.441406] $24   : 00000000 83361334
[13027.449218] $28   : 8032a000 8032bbb8 80340000 82aba334
[13027.453125] Hi    : 00000031
[13027.457031] Lo    : 00000871
[13027.457031] epc   : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.464843]     Tainted: G           O
[13027.468750] ra    : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.476562] Status: 1000fc03    KERNEL EXL IE
[13027.480468] Cause : 00800010
[13027.484375] BadVA : 8087086a
[13027.488281] PrId  : 00019374 (MIPS 24Kc)
[13027.492187] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq
sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred
sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic
act_police act_ipt act_connmark act_skbedit act_mirred em_u32 cls_u32
cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_fq_codel sch_codel
sch_ingress ath79_wdt xt_hashlimit xt_set(O) ip_set_list_set(O)
ip_set_hash_netport(O) ip_set_hash_netiface(O) ip_set_hash_net(O)
ip_set_hash_ipportnet(O) ip_set_hash_ipportip(O) ip_set_hash_ipport(O)
ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O)
ip_set_bitmap_ip(O) ip_set(O) ip6t_REJECT ip6t_rt ip6t_hbh ip6t_mh
ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6table_mangle
ip6table_filter ip6_tables nf_conntrack_ipv6 nf_defrag_ipv6 nfnetlink
nf_nat_irc nf_nat_ftp nf_conntrack_irc nf_conntrack_ftp xt_policy xt_esp
ipt_ah xt_HL xt_hl xt_ecn ipt_ECN xt_CLASSIFY xt_time xt_tcpmss
xt_statistic xt_mark xt_length xt_DSCP xt_dscp xt_string xt_layer7 xt_quota
xt_pkttype xt_physdev xt_owner xt_addrtype ipt_MASQUERADE iptable_nat
xt_nat nf_nat_ipv4 nf_nat xt_recent xt_helper xt_connmark xt_connbytes pptp
pppoe xt_conntrack xt_CT iptable_raw xt_state nf_conntrack_ipv4
nf_defrag_ipv4 nf_conntrack pppox ipt_REJECT xt_TCPMSS xt_LOG xt_comment
xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables
xt_tcpudp x_tables ip_gre gre ifb sit ipcomp xfrm4_tunnel xfrm4_mode_tunnel
xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel4 tun tcp_ledbat(O)
ppp_async ppp_generic slhc af_key xfrm_user xfrm_ipcomp xfrm_algo vfat fat
autofs4 button_hotplug(O) ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O)
nls_utf8 nls_iso8859_2 nls_iso8859_15 nls_iso8859_13 nls_iso8859_1
nls_cp437 mac80211(O) ts_fsm ts_bm ts_kmp crc_ccitt ipv6 input_polldev
cfg80211(O) compat(O) input_core sha1_generic md5 hmac des_generic deflate
cbc authenc arc4 usb_storage ohci_hcd ehci_hcd sd_mod ext4 jbd2 mbcache
usbcore usb_common scsi_mod nls_base crc16 zlib_inflate zlib_deflate
ledtrig_timer ledtrig_default_on leds_gpio gpio_button_hotplug(O)
[13027.675781] Process swapper (pid: 0, threadinfo=8032a000, task=8032f030,
tls=00000000)
[13027.683593] Stack : 8381a000 8381a800 c196c5e8 829cc77c c1c3d570
829cc77c 00000005 80064b68
[13027.683593]         00000006 8381a800 00000000 83217480 8381a800
00000000 8381a800 00000001
[13027.683593]         00000004 00000000 83245f98 82aba07c a20046c0
00000001 8381a000 82a643e0
[13027.683593]         8032bcac 823040c0 803a5428 8032bca4 83217480
8032dfd0 00000000 8022ea74
[13027.683593]         00000000 82304840 00000000 187ac42a 83245f98
80881d48 82abae60 83217480
[13027.683593]         ...
[13027.718750] Call Trace:
[13027.722656] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c [nf_defrag_ipv6]
[12596.171875]         000008a0 00000000 00000000 3596c669 83245f98
80881d48 82a
[12596.171875]
...
[12596.207031] Call
Trace:
[12596.210937] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[12596.218750] [<82aba07c>] ipv6_defrag+0x74/0xc8
[nf_defrag_ipv6]
[12596.222656] [<8022ea74>]
nf_iterate+0x98/0x104
[12596.226562] [<8022eb78>]
nf_hook_slow+0x98/0x170
[12596.234375] [<8324643c>] ipv6_rcv+0x3fc/0x4a0
[ipv6]
[12596.238281] [<80209410>]
__netif_receive_skb+0x63c/0x6c0
[12596.242187] [<8298f384>] ri_tasklet+0x104/0x2b0
[ifb]
[12596.250000] [<80079434>]
tasklet_action+0x78/0xc8
[12596.253906] [<80078c48>]
__do_softirq+0xb0/0x184
[12596.257812] [<80078dcc>]
do_softirq+0x48/0x68
[12596.261718] [<80078fe8>]
irq_exit+0x4c/0xb8
[12596.265625] [<8006316c>]
ret_from_irq+0x0/0x4
[12596.269531] [<80063380>]
r4k_wait+0x20/0x40
[12596.273437] [<80064e68>]
cpu_idle+0x38/0x70
[12596.277343] [<803488fc>]
start_kernel+0x374/0x394
[12596.285156]
</pre>
-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html