[Cake] Putting cake under dsmark on ingress ifb

[Cake] Putting cake under dsmark on ingress ifb

[Andy Furniss]

I am well rusty with linux qos and have never tried dsmark before.

I am likely doing something stupid here :-)

So the test: I want to set dsmark on ingress traffic so I can control
which cake tin it goes to - test just marking icmp as ef.

ingress qdisc is added to ppp0 and redirected to ifb0, on which I do -

tc qdisc add dev ifb0 handle 1:0 root dsmark indices 64

tc filter add dev ifb0 parent 1:0 protocol ip prio 1 u32 match ip 
protocol 1 0xff classid 1:1

tc class change dev ifb0 classid 1:1 dsmark mask 0x3 value 0xb8

This works in the sense that I can now get tcpdump on a PC on the
lan to show incoming icmp with tos 0xb8.

So I try to add cake under dsmark ini the hope that it sees icmp as ef
but it doesn't, icmp goes to best effort.

tc qdisc add dev ifb0 handle 2:0 parent 1:0 cake bandwidth 60mbit raw 
overhead 34 diffserv4

tc -s -d qdisc  ls dev ifb0
qdisc dsmark 1: root refcnt 2 indices 0x0040
  Sent 60259205 bytes 71562 pkt (dropped 8, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
qdisc cake 2: parent 1: bandwidth 60Mbit diffserv4 triple-isolate rtt 
100.0ms noatm overhead 48 via-ethernet
  Sent 60254313 bytes 71495 pkt (dropped 8, overlimits 67924 requeues 0)
  backlog 0b 0p requeues 0
  memory used: 160448b of 4Mb
  capacity estimate: 60Mbit
                  Bulk   Best Effort      Video       Voice
   thresh      3750Kbit      60Mbit      30Mbit      15Mbit
   target         5.0ms       5.0ms       5.0ms       5.0ms
   interval     100.0ms     100.0ms     100.0ms     100.0ms
   pk_delay         4us       311us        87us         7us
   av_delay         0us       214us         4us         0us
   sp_delay         0us         6us         4us         0us
   pkts               3       71367          95          38
   bytes            124    60246264       18265        1554
   way_inds           0         182           0           0
   way_miss           3        7043          77           7
   way_cols           0           0           0           0
   drops              0           8           0           0
   marks              0           0           0           0
   sp_flows           0           1           0           0
   bk_flows           0           0           0           0
   un_flows           0           0           0           0
   max_len           44        1500        1151          78

The packets showing as voice are AFAIKT ntp. pinging from lan
still shows 0xb8 but the counters show that they are going through
Best Effort.

Anyone see what's wrong?

TIA.

Re: [Cake] Putting cake under dsmark on ingress ifb

[Jonathan Morton]

> On 26 Feb, 2017, at 15:41, Andy Furniss <adf.lists@gmail.com> wrote:
> 
> This works in the sense that I can now get tcpdump on a PC on the
> lan to show incoming icmp with tos 0xb8.
> 
> So I try to add cake under dsmark ini the hope that it sees icmp as ef
> but it doesn't, icmp goes to best effort.

I suspect packets are queuing in cake first and only then going through dsmark.  What happens if you reverse the nesting?

 - Jonathan Morton

Re: [Cake] Putting cake under dsmark on ingress ifb

[Andy Furniss]

Jonathan Morton wrote:
>
>> On 26 Feb, 2017, at 15:41, Andy Furniss <adf.lists@gmail.com> wrote:
>>
>> This works in the sense that I can now get tcpdump on a PC on the
>> lan to show incoming icmp with tos 0xb8.
>>
>> So I try to add cake under dsmark ini the hope that it sees icmp as ef
>> but it doesn't, icmp goes to best effort.
>
> I suspect packets are queuing in cake first and only then going through dsmark.  What happens if you reverse the nesting?

tc qdisc add dev ifb0 handle 1:0 root cake bandwidth 60mbit raw overhead 
34 diffserv4

tc qdisc add dev ifb0 handle 2:0 parent 1:0 dsmark indices 64
RTNETLINK answers: Operation not supported

Going to be AFK till this evening - will try more random stuff later.

Re: [Cake] Putting cake under dsmark on ingress ifb

[Andy Furniss]

Andy Furniss wrote:
> Jonathan Morton wrote:
>>
>>> On 26 Feb, 2017, at 15:41, Andy Furniss <adf.lists@gmail.com> wrote:
>>>
>>> This works in the sense that I can now get tcpdump on a PC on the
>>> lan to show incoming icmp with tos 0xb8.
>>>
>>> So I try to add cake under dsmark ini the hope that it sees icmp as ef
>>> but it doesn't, icmp goes to best effort.
>>
>> I suspect packets are queuing in cake first and only then going
>> through dsmark.  What happens if you reverse the nesting?
>
> tc qdisc add dev ifb0 handle 1:0 root cake bandwidth 60mbit raw overhead
> 34 diffserv4
>
> tc qdisc add dev ifb0 handle 2:0 parent 1:0 dsmark indices 64
> RTNETLINK answers: Operation not supported
>
> Going to be AFK till this evening - will try more random stuff later.

I couldn't get anything to work with dsmark so went for plan B =

Use tc pedit which has a selector for dscp.  Turned out to be useless
though, it looked like it overwrote the whole tos byte, but worse I
ended up with packets that failed csum so got dropped.

Plan C worked :-) = use action ipt, which is now action xt (ugh).

tc filter add dev ppp0 parent ffff: prio 1 protocol ip u32 match ip 
protocol 1 0xff action xt -j DSCP --set-dscp-class ef action mirred 
egress redirect dev ifb0

...other filters ...

TODO see if it's possible with actions to set dscp when using connmark to
choose the flow.

Re: [Cake] Putting cake under dsmark on ingress ifb

[Jonathan Morton]

> On 28 Feb, 2017, at 16:11, Andy Furniss <adf.lists@gmail.com> wrote:
> 
> tc filter add dev ppp0 parent ffff: prio 1 protocol ip u32 match ip protocol 1 0xff action xt -j DSCP --set-dscp-class ef action mirred egress redirect dev ifb0

Okay, that’s potentially useful information - that a qdisc filter action can set DSCPs (albeit in this case by circuitous means).  An action module which does that more directly would undoubtedly be easier to use and more efficient, but one doesn’t seem to exist yet.

 - Jonathan Morton

Re: [Cake] Putting cake under dsmark on ingress ifb

[Andy Furniss]

Andy Furniss wrote:

> TODO see if it's possible with actions to set dscp when using connmark to
> choose the flow.

So I had a go at this and got a working solution - with some
interesting fails on the way.

My ISP doesn't do ipv6 so that is untested and wouldn't work for anyone
using any examples I post here in the sense that I use protocol ip.

So a working test = use iptables to set a connmark 1 for this test on 
something then -

tc qdisc add dev ppp0 ingress
tc filter add dev ppp0 parent ffff: prio 1 protocol ip u32 match u32 0 0 
action connmark continue
tc filter add dev ppp0 parent ffff: prio 2 protocol ip handle 1 fw 
action xt -j DSCP --set-dscp-class cs1 action mirred egress redirect dev 
ifb0
tc filter add dev ppp0 parent ffff: prio 3 protocol ip u32 match u32 0 0 
  action mirred egress redirect dev ifb0

Minor nit with this method = directly matching unmarked like

.... handle 0 fw .... doesn't work, so you need to allow for that or
test ematch which IIRC can do nfmark - untested by me.


The fail = try to use something like

tc filter add dev ppp0 parent ffff: prio 1 protocol ip u32 match u32 0 0 
action xt -j CONNMARK --restore-mark action continue

Testing on my router box with old iptables this was accepted OK but
didn't actually work.

I updated iptables and rebuilt cake-iproute against the new version
to see if it made any difference - tc segfaulted.

Moving testing to my desktop which has a newer kernel I got the same
result.

Next test = use vanilla git iproute2, even worse = Oops. So may be
best to avoid that one for now :-).

Re: [Cake] Putting cake under dsmark on ingress ifb

[Andy Furniss]

Andy Furniss wrote:

> Next test = use vanilla git iproute2, even worse = Oops. So may be
> best to avoid that one for now :-).

I manages to avoid the Oops by udating iptables from 1.6.0 to 1.6.1
which is handy, though now it fails with an error from iptables - but
at least it fails without taking my PC down.

-j DSCP now fails with this setup (git iproute2, git kernel, iptables 1.6.1)
but it works with cake iproute2 - so in summary getting action xt to
work is rather hit and miss.

It would, as you said, be far more preferable if there was an action
to change dscp (and unlike tc-pedit actually recalculate the checksum).