[Cake] act_connmark + dscp

[Toke Høiland-Jørgensen]

Kevin Darbyshire-Bryant <kevin at darbyshire-bryant.me.uk> writes:

> OK, what I am trying to do is classify incoming connections into
> relevant cake tins to impose some bandwidth fairness.  e.g. classify
> bittorrent & things that are downloads into the Bulk tin, and
> prioritise stuff that is streaming video into the Video tin. Incoming
> DSCP has a) been washed and b) is unreliable anyway so is unhelpful in
> this case.  iptables runs too late, so having rules to classify
> incoming stuff is pointless.

Right, I see.

[... snip .. ]

> Then I recently discovered act_connmark
> (http://linux-ip.net/gl/tc-filters/tc-filters-node2.html) - the
> thinking being I could use iptables on egress to set fwmarks to
> classify a connection and have the ingress packets magically follow.
> This worked but still required 3 tc filter actions to cope with 4
> tins:
>
> $TC filter add dev $IFACE parent $MAJOR protocol ip handle 0x01 fw action skbedit priority ${MAJOR}1
> $TC filter add dev $IFACE parent $MAJOR protocol ip handle 0x03 fw action skbedit priority ${MAJOR}3
> $TC filter add dev $IFACE parent $MAJOR protocol ip handle 0x04 fw action skbedit priority ${MAJOR}4

Right, so this can be replaced with the fwmark action we already added
(and that I just pushed an update to so it supports masking the value
before selecting a tin).

> The overriding (if required) of DSCP could be done in iptables and to
> avoid going through the iptables DSCP decision/mangling for every
> packet I could use a flag within the fwmark to indicate the decision
> had previously been made and stored for this connection.

[ ... ]

> I’m doing 2 things.
>
> 1) Classifying traffic into tins on ingress based on the egress DSCP
> values contained in fwmarks.
>
> 2) Basing the fwmark contained DSCP on the initial packet of the
> connection, possibly after being modified once by iptables rules.

So I tried prototyping what it would actually look like to do all this
in iptables. The result is below (in iptables-restore format). I haven't
tested it, but I believe something along the lines of this will work,
when used along with the CAKE fwmark support (setting a mask of 0xFF
when configuring CAKE).

Now, the obvious eyesore on this is the need to replicate CAKEs diffserv
mappings in iptables rules (21 rules in this case, for the diffserv4
mapping). As long as this only runs once per connection I don't actually
think it's much of a performance issue for normal use, but obviously
there could be pathological cases, and it's also annoying to have to do
that.

So, first question becomes: Do you agree that the firewall rules below
would solve your use case (ignoring the ugliness of having to replicate
the diffserv parsing in iptables)? Or am I missing something?

-Toke



*mangle
:PREROUTING ACCEPT [0:0]
:APPLY-MARKS - [0:0]
:MARK-DSCP - [0:0]
:MARK-POLICY - [0:0]

# Run on inside iface - eth0 in this case
-A PREROUTING -i eth0 -J APPLY-MARKS

# Make sure we have the marks from conntrack
-A APPLY-MARKS -J CONNMARK --restore-mark --nfmask 0xFF --ctmask 0xFF

# Abort if our "already set" bit is set
-A APPLY-MARKS -m mark --mark 0x80/0x80 -j RETURN

# If a DSCP value is set, use DSCP-based marking
-A APPLY-MARKS -m dscp ! --dscp 0 -J MARK-DSCP

# Otherwise, set our own policy
-A APPLY-MARKS -m dscp --dscp 0 -J MARK-POLICY

# Set our "already set" bit, and store things back into conntrack
-A APPLY-MARKS -J MARK --set-mark 0x80/0x80
-A APPLY-MARKS -J CONNMARK --save-mark --nfmask 0xFF --ctmask 0xFF

# DSCP-based fwmark setting
# add whatever DSCP coverage you want here; the below is CAKE's diffserv4
# fwmark offsets into CAKE's tin_order, and are 1-indexed, so tin selection
# is offset by 1, and bulk and besteffort are swapped compared to the array
# in sch_cake.c
# We assume CAKE is configured with an fwmark mask of 0x7F

-A MARK-DSCP -m dscp --dscp 1 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 4 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 8 -J MARK --set-mark 0x1/0x7F
-A MARK-DSCP -m dscp --dscp 16 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 18 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 20 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 22 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 24 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 26 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 28 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 30 -J MARK --set-mark 0x3/0x7F
-A MARK-DSCP -m dscp --dscp 34 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 36 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 38 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 32 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 40 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 44 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 46 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 48 -J MARK --set-mark 0x4/0x7F
-A MARK-DSCP -m dscp --dscp 56 -J MARK --set-mark 0x4/0x7F
# default best effort
-A MARK-DSCP -m mark --mark 0x0/0x7F -J --set-mark 0x2/0x7F


# Policy-based fwmark setting
-A MARK-POLICY -p tcp -s 192.168.219.5 -m comment --comment "Skybox Bulk" -j MARK --set-mark 0x1/0x7F
# add more policy rules here; anything not marked will use the DSCP bits
# of each packet; optional catch-all to avoid that and make everything BE:
#-A MARK-DSCP -m mark --mark 0x0/0x7F -J --set-mark 0x2/0x7F