[Cerowrt-devel] possible ingress shaping bug?

Wed Oct 8 22:48:57 EDT 2014

On Mon, Oct 6, 2014 at 3:07 AM, Richard Edmands <thesirdmz at gmail.com> wrote:
> Greetings Everyone!
> As part of research into a related part of the kernel we ran across an
> oddity with nfct on ifb devices.
> To give weight to the existence of the oddity, a Debian Jessie (so 3.16.3
> based kernel) system was setup with wan and lan ports.
> A modified version of the gentoo fq_codel script was used (Wan is eth0 and
> associated IFB is ifb0). Including script.
> The kernel had 2 printk's inserted (see included change to cls_flows.c) to
> print out the ip addresses which were being returned to the flows process
> for nfct-src and nfct-dst.
> The nfct-src attached to eth0 printk was heard from, it returned the right
> answers and some nonsense as expected.
> The nfct-dst attached to ifb0 printk didn't return a thing, not even
> nonsense.
> ------------------------------------------------------------------------------------------------------
> (Wild speculation)
> Maybe the conntrack lookup only works correctly from the right device? so
> the lookup from the packet on ifb0 is being looked up in conntrack as if it
> should relate to a connection on ifb0 instead of a connection on eth0?

does straight src or dst work in ifb?

The conntrack stuff is long gone by the time you get to ifb.

> ---------------------------------------------------------------------------
> #!/bin/sh
>
>
>
> ## Paths and definitions
> tc=/sbin/tc
> IPT=iptables
>
> #The interface name of the internet link
> ext=eth0
>
> #Ignore this
> ext_ingress=ifb0
>
>
>
> # Set these as per your adsl sync rate, at 90% to start with, modify up and
> down until the tests work.
> ext_up=40000kbit
> ext_down=40000kbit
>
>
> ethtool -K eth0 tso off gro off gso off
> ethtool -K eth1 tso off gro off gso off
>
> modprobe ifb
> modprobe sch_fq_codel
> modprobe act_mirred
> modprobe cls_flow
>
> # Clear old queuing disciplines (qdisc) on the interfaces
> $tc qdisc del dev $ext root
> $tc qdisc del dev $ext ingress
> $tc qdisc del dev $ext_ingress root
> $tc qdisc del dev $ext_ingress ingress
>
>
>
> #########
> # INGRESS
> #########
>
> # Create ingress on external interface
> $tc qdisc add dev $ext handle ffff: ingress
> ifconfig $ext_ingress up # if the interace is not up bad things happen
>
> # Forward all ingress traffic to the IFB device
> $tc filter add dev $ext parent ffff: protocol all u32 match u32 0 0 action
> mirred egress redirect dev $ext_ingress
>
> # Create an EGRESS filter on the IFB device
> #You can play with the r2q value, higher for faster links, lower for slower.
> FIXME? Totally need some maths and tables for the correct values in various
> situations. Is being too high safer than being too low?

It is best to be correct. See the sqm source code and recent threads on this
list. *I* don't grok all that is involved on getting dsl or PPPoe right,
all the systems I have access to are cable or fiber.

> $tc qdisc add dev $ext_ingress root handle 1: htb default 12 r2q 5
>
> # Add root class HTB with rate limiting
> $tc class add dev $ext_ingress parent 1: classid 1:1 htb rate $ext_down
> overhead 40 mtu 1492 mpu 53 linklayer atm
> $tc class add dev $ext_ingress parent 1:1 classid 1:11 htb rate $ext_down
> prio 0 overhead 40 mtu 1492 mpu 53 linklayer atm
> $tc class add dev $ext_ingress parent 1:1 classid 1:12 htb rate $ext_down
> prio 0 overhead 40 mtu 1492 mpu 53 linklayer atm
>
>
> #This is where fq_codel is installed for downlink traffic
> $tc qdisc add dev $ext_ingress parent 1:11 fq_codel noecn target 50ms
> interval 300ms quantum 512 flows 1024 limit 3000
> $tc qdisc add dev $ext_ingress parent 1:12 fq_codel noecn target 50ms
> interval 300ms quantum 512 flows 1024 limit 3000
>
>
> #FIXME? Really need to find/have a discussion and determine good values for
> this
> $tc filter add dev $ext_ingress protocol ip parent 1: handle 11 flow hash
> keys nfct-dst divisor 1024 baseclass 1:11
> $tc filter add dev $ext_ingress protocol ip parent 1: handle 12 flow hash
> keys nfct-dst divisor 1024 baseclass 1:12

dst is the correct thing here.

> # Yes I KNOW this does not work.
> $tc filter add dev $ext_ingress protocol ip parent 1: prio 2 u32 match ip
> dst 172.30.200.2 flowid 1:11

Well, actually no, this should work. Unless the parent should be something else.

>
>
> #########
> # EGRESS
> #########
>
> # Add FQ_CODEL to EGRESS on external interface
> #You can play with the r2q value, higher for faster links, lower for slower.
> FIXME? Totally need some maths and tables for the correct values in various
> situations. Is being too high safer than being too low?
> $tc qdisc add dev $ext root handle 1: htb default 12 r2q 1
>
> # Add root class HTB with rate limiting
> $tc class add dev $ext parent 1: classid 1:1 htb rate $ext_up overhead 40
> mtu 1492 mpu 53 linklayer atm
> $tc class add dev $ext parent 1:1 classid 1:11 htb rate 70kbit ceil $ext_up
> overhead 40 mtu 1492 mpu 53 linklayer atm
> $tc class add dev $ext parent 1:1 classid 1:12 htb rate 30kbit ceil $ext_up
> overhead 40 mtu 1492 mpu 53 linklayer atm
>
>
> #This is where fq_codel is installed for uplink traffic
> $tc qdisc add dev $ext parent 1:11 handle 11: fq_codel noecn target 100ms
> interval 300ms quantum 300 flows 1024 limit 3000
> $tc qdisc add dev $ext parent 1:12 handle 12: fq_codel noecn target 100ms
> interval 300ms quantum 300 flows 1024 limit 3000
>
>
>
>
>
> $tc filter add dev $ext protocol ip parent 1: handle 11 flow hash keys
> nfct-src divisor 1024 baseclass 1:11
> $tc filter add dev $ext protocol ip parent 1: handle 12 flow hash keys
> nfct-src divisor 1024 baseclass 1:12
>
> $tc filter add dev $ext protocol ip parent 1: prio 2 u32 match ip src
> 172.30.200.2 flowid 1:11
> ----------------------------------------------------------------------------------------------------------------------------
>
> --- linux-3.16.3.orig/net/sched/cls_flow.c 2014-09-18 03:22:16.000000000
> +1000
> +++ linux-3.16.3/net/sched/cls_flow.c 2014-10-04 13:01:56.000000000 +1000
> @@ -143,6 +143,7 @@
>  {
>   switch (skb->protocol) {
>   case htons(ETH_P_IP):
> +printk(KERN_WARNING "nfct src %d \n", (ntohl(CTTUPLE(skb, src.u3.ip))) );
>   return ntohl(CTTUPLE(skb, src.u3.ip));
>   case htons(ETH_P_IPV6):
>   return ntohl(CTTUPLE(skb, src.u3.ip6[3]));
> @@ -155,6 +156,7 @@
>  {
>   switch (skb->protocol) {
>   case htons(ETH_P_IP):
> +printk(KERN_WARNING "nfct dst %d \n", (ntohl(CTTUPLE(skb, dst.u3.ip))) );
>   return ntohl(CTTUPLE(skb, dst.u3.ip));
>   case htons(ETH_P_IPV6):
>   return ntohl(CTTUPLE(skb, dst.u3.ip6[3]));
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

-- 
Dave Täht

https://www.bufferbloat.net/projects/make-wifi-fast