[Cerowrt-devel] possible ingress shaping bug?

[Richard Edmands]

Greetings Everyone!
As part of research into a related part of the kernel we ran across an
oddity with nfct on ifb devices.
To give weight to the existence of the oddity, a Debian Jessie (so 3.16.3
based kernel) system was setup with wan and lan ports.
A modified version of the gentoo fq_codel script was used (Wan is eth0 and
associated IFB is ifb0). Including script.
The kernel had 2 printk's inserted (see included change to cls_flows.c) to
print out the ip addresses which were being returned to the flows process
for nfct-src and nfct-dst.
The nfct-src attached to eth0 printk was heard from, it returned the right
answers and some nonsense as expected.
The nfct-dst attached to ifb0 printk didn't return a thing, not even
nonsense.
------------------------------------------------------------------------------------------------------
(Wild speculation)
Maybe the conntrack lookup only works correctly from the right device? so
the lookup from the packet on ifb0 is being looked up in conntrack as if it
should relate to a connection on ifb0 instead of a connection on eth0?

---------------------------------------------------------------------------
#!/bin/sh



## Paths and definitions
tc=/sbin/tc
IPT=iptables

#The interface name of the internet link
ext=eth0

#Ignore this
ext_ingress=ifb0



# Set these as per your adsl sync rate, at 90% to start with, modify up and
down until the tests work.
ext_up=40000kbit
ext_down=40000kbit


ethtool -K eth0 tso off gro off gso off
ethtool -K eth1 tso off gro off gso off

modprobe ifb
modprobe sch_fq_codel
modprobe act_mirred
modprobe cls_flow

# Clear old queuing disciplines (qdisc) on the interfaces
$tc qdisc del dev $ext root
$tc qdisc del dev $ext ingress
$tc qdisc del dev $ext_ingress root
$tc qdisc del dev $ext_ingress ingress



#########
# INGRESS
#########

# Create ingress on external interface
$tc qdisc add dev $ext handle ffff: ingress
ifconfig $ext_ingress up # if the interace is not up bad things happen

# Forward all ingress traffic to the IFB device
$tc filter add dev $ext parent ffff: protocol all u32 match u32 0 0 action
mirred egress redirect dev $ext_ingress

# Create an EGRESS filter on the IFB device
#You can play with the r2q value, higher for faster links, lower for
slower. FIXME? Totally need some maths and tables for the correct values in
various situations. Is being too high safer than being too low?
$tc qdisc add dev $ext_ingress root handle 1: htb default 12 r2q 5

# Add root class HTB with rate limiting
$tc class add dev $ext_ingress parent 1: classid 1:1 htb rate $ext_down
overhead 40 mtu 1492 mpu 53 linklayer atm
$tc class add dev $ext_ingress parent 1:1 classid 1:11 htb rate $ext_down
prio 0 overhead 40 mtu 1492 mpu 53 linklayer atm
$tc class add dev $ext_ingress parent 1:1 classid 1:12 htb rate $ext_down
prio 0 overhead 40 mtu 1492 mpu 53 linklayer atm


#This is where fq_codel is installed for downlink traffic
$tc qdisc add dev $ext_ingress parent 1:11 fq_codel noecn target 50ms
interval 300ms quantum 512 flows 1024 limit 3000
$tc qdisc add dev $ext_ingress parent 1:12 fq_codel noecn target 50ms
interval 300ms quantum 512 flows 1024 limit 3000


#FIXME? Really need to find/have a discussion and determine good values for
this
$tc filter add dev $ext_ingress protocol ip parent 1: handle 11 flow hash
keys nfct-dst divisor 1024 baseclass 1:11
$tc filter add dev $ext_ingress protocol ip parent 1: handle 12 flow hash
keys nfct-dst divisor 1024 baseclass 1:12

# Yes I KNOW this does not work.
$tc filter add dev $ext_ingress protocol ip parent 1: prio 2 u32 match ip
dst 172.30.200.2 flowid 1:11


#########
# EGRESS
#########

# Add FQ_CODEL to EGRESS on external interface
#You can play with the r2q value, higher for faster links, lower for
slower. FIXME? Totally need some maths and tables for the correct values in
various situations. Is being too high safer than being too low?
$tc qdisc add dev $ext root handle 1: htb default 12 r2q 1

# Add root class HTB with rate limiting
$tc class add dev $ext parent 1: classid 1:1 htb rate $ext_up overhead 40
mtu 1492 mpu 53 linklayer atm
$tc class add dev $ext parent 1:1 classid 1:11 htb rate 70kbit ceil $ext_up
overhead 40 mtu 1492 mpu 53 linklayer atm
$tc class add dev $ext parent 1:1 classid 1:12 htb rate 30kbit ceil $ext_up
overhead 40 mtu 1492 mpu 53 linklayer atm


#This is where fq_codel is installed for uplink traffic
$tc qdisc add dev $ext parent 1:11 handle 11: fq_codel noecn target 100ms
interval 300ms quantum 300 flows 1024 limit 3000
$tc qdisc add dev $ext parent 1:12 handle 12: fq_codel noecn target 100ms
interval 300ms quantum 300 flows 1024 limit 3000





$tc filter add dev $ext protocol ip parent 1: handle 11 flow hash keys
nfct-src divisor 1024 baseclass 1:11
$tc filter add dev $ext protocol ip parent 1: handle 12 flow hash keys
nfct-src divisor 1024 baseclass 1:12

$tc filter add dev $ext protocol ip parent 1: prio 2 u32 match ip src
172.30.200.2 flowid 1:11
----------------------------------------------------------------------------------------------------------------------------

--- linux-3.16.3.orig/net/sched/cls_flow.c 2014-09-18 03:22:16.000000000
+1000
+++ linux-3.16.3/net/sched/cls_flow.c 2014-10-04 13:01:56.000000000 +1000
@@ -143,6 +143,7 @@
 {
  switch (skb->protocol) {
  case htons(ETH_P_IP):
+printk(KERN_WARNING "nfct src %d \n", (ntohl(CTTUPLE(skb, src.u3.ip))) );
  return ntohl(CTTUPLE(skb, src.u3.ip));
  case htons(ETH_P_IPV6):
  return ntohl(CTTUPLE(skb, src.u3.ip6[3]));
@@ -155,6 +156,7 @@
 {
  switch (skb->protocol) {
  case htons(ETH_P_IP):
+printk(KERN_WARNING "nfct dst %d \n", (ntohl(CTTUPLE(skb, dst.u3.ip))) );
  return ntohl(CTTUPLE(skb, dst.u3.ip));
  case htons(ETH_P_IPV6):
  return ntohl(CTTUPLE(skb, dst.u3.ip6[3]));
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cerowrt-devel/attachments/20141006/f0256e77/attachment-0002.html>