* Re: [Cerowrt-devel] rsync over ssh
2014-01-29 21:27 [Cerowrt-devel] rsync over ssh Michael Richardson
@ 2014-01-29 22:01 ` Sebastian Moeller
2014-01-29 22:18 ` Dave Taht
1 sibling, 0 replies; 3+ messages in thread
From: Sebastian Moeller @ 2014-01-29 22:01 UTC (permalink / raw)
To: Michael Richardson; +Cc: cerowrt-devel
Hi Michael,
On Jan 29, 2014, at 22:27 , Michael Richardson <mcr@sandelman.ca> wrote:
>
> I often run rsync over ssh to transfer files.
> I suspect that they are being classified wrong, and are competing with my
> interactive ssh sessions. I experience classic bufferbloat issues with a
> congested uplink when typing. (But, DNS and web is unaffected, it seems)
>
> I notice in ssh_config(5) man page that it says:
>
> IPQoS Specifies the IPv4 type-of-service or DSCP class for connections. Accepted values are ``af11'',
> ``af12'', ``af13'', ``af21'', ``af22'', ``af23'', ``af31'', ``af32'', ``af33'', ``af41'', ``af42'',
> ``af43'', ``cs0'', ``cs1'', ``cs2'', ``cs3'', ``cs4'', ``cs5'', ``cs6'', ``cs7'', ``ef'', ``lowdelay'',
> ``throughput'', ``reliability'', or a numeric value. This option may take one or two arguments, sepa-
> rated by whitespace. If one argument is specified, it is used as the packet class unconditionally. If
> two values are specified, the first is automatically selected for interactive sessions and the second
> for non-interactive sessions. The default is ``lowdelay'' fo.r interactive sessions and ``throughput''
> for non-interactive sessions.
>
>
You could try to add "-o IPQoS ef" to your interactive sessions? But this will only have a chance of working, if you use simple.qos, as smiliest.qos does not have any priority tiers… (I am sure you know that already). From simple.qos:
ipt_setup() {
ipt -t mangle -N QOS_MARK_${IFACE}
ipt -t mangle -A QOS_MARK_${IFACE} -j MARK --set-mark 0x2
# You can go further with classification but...
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class CS1 -j MARK --set-mark 0x3
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class CS6 -j MARK --set-mark 0x1
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class EF -j MARK --set-mark 0x1
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class AF42 -j MARK --set-mark 0x1
ipt -t mangle -A QOS_MARK_${IFACE} -m tos --tos Minimize-Delay -j MARK --set-mark 0x1
# and it might be a good idea to do it for udp tunnels too
# Turn it on. Preserve classification if already performed
ipt -t mangle -A POSTROUTING -o $DEV -m mark --mark 0x00 -g QOS_MARK_${IFACE}
ipt -t mangle -A POSTROUTING -o $IFACE -m mark --mark 0x00 -g QOS_MARK_${IFACE}
# The Syn optimization was nice but fq_codel does it for us
# ipt -t mangle -A PREROUTING -i s+ -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x01
# Not sure if this will work. Encapsulation is a problem period
ipt -t mangle -A PREROUTING -i vtun+ -p tcp -j MARK --set-mark 0x2 # tcp tunnels need ordering
# Emanating from router, do a little more optimization
# but don't bother with it too much.
ipt -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP --set-dscp-class AF42
#Not clear if the second line is needed
Looks to my untrained eye as if CS1 ends up in the scavenger background queue, EF in contrast should end up in the highest priority queue. (And if the rsync and interactive sessions terminate at the same end points they need to to be separated into different tiers/fq_codel instances, otherwise they end up in the same fq_codel bin and will affect each other; as far as I understand).
> I can see no evidence that my IPv6 packets are being changed in anyway.
> On IPv4, I see:
>
> 16:25:50.744290 IP (tos 0x8, ttl 64, id 62723, offset 0, flags [DF], proto TCP (6), length 52)
> 209.87.252.250.51417 > 97.107.133.15.22: Flags [F.], cksum 0x5735 (correct), seq 623906, ack 3314, win 110, options [nop,nop,TS val 1178738167 ecr 2623900673], length 0
>
> I'm not sure that "tos 0x8" is particularly useful. It's DSCP 1, and I think
> it's actually being marked TOS-like.
>
> Will any value here have any affect on default 3.10 qos.sh script?
My bet is on EF for the interact sessions…
best
Sebastian
>
> --
> ] Never tell me the odds! | ipv6 mesh networks [
> ] Michael Richardson, Sandelman Software Works | network architect [
> ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on rails [
>
>
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [Cerowrt-devel] rsync over ssh
2014-01-29 21:27 [Cerowrt-devel] rsync over ssh Michael Richardson
2014-01-29 22:01 ` Sebastian Moeller
@ 2014-01-29 22:18 ` Dave Taht
1 sibling, 0 replies; 3+ messages in thread
From: Dave Taht @ 2014-01-29 22:18 UTC (permalink / raw)
To: Michael Richardson; +Cc: cerowrt-devel
On Wed, Jan 29, 2014 at 1:27 PM, Michael Richardson <mcr@sandelman.ca> wrote:
>
> I often run rsync over ssh to transfer files.
> I suspect that they are being classified wrong, and are competing with my
> interactive ssh sessions. I experience classic bufferbloat issues with a
> congested uplink when typing. (But, DNS and web is unaffected, it seems)
0) you should measure your delay and post...
1) I switched to mosh for most of my interactive sessions that I used
to use ssh for. Helps.
2) there are patches to rsync to use a different cc algo and diffserv
marking, but those
only work over native tcp.
https://git.samba.org/?p=rsync-patches.git;a=blob_plain;f=congestion.diff;hb=master
>
> I notice in ssh_config(5) man page that it says:
>
> IPQoS Specifies the IPv4 type-of-service or DSCP class for connections. Accepted values are ``af11'',
> ``af12'', ``af13'', ``af21'', ``af22'', ``af23'', ``af31'', ``af32'', ``af33'', ``af41'', ``af42'',
> ``af43'', ``cs0'', ``cs1'', ``cs2'', ``cs3'', ``cs4'', ``cs5'', ``cs6'', ``cs7'', ``ef'', ``lowdelay'',
> ``throughput'', ``reliability'', or a numeric value. This option may take one or two arguments, sepa-
> rated by whitespace. If one argument is specified, it is used as the packet class unconditionally. If
> two values are specified, the first is automatically selected for interactive sessions and the second
> for non-interactive sessions. The default is ``lowdelay'' for interactive sessions and ``throughput''
> for non-interactive sessions.
3) I don't think rsync currently tries to set anything when using ssh
as a transport. Perhaps
those patches can be improved to pass it down to the ssh call?
>
>
> I can see no evidence that my IPv6 packets are being changed in anyway.
A lot of systems don't have the correct IPV6_TCLASS call.
> On IPv4, I see:
>
> 16:25:50.744290 IP (tos 0x8, ttl 64, id 62723, offset 0, flags [DF], proto TCP (6), length 52)
> 209.87.252.250.51417 > 97.107.133.15.22: Flags [F.], cksum 0x5735 (correct), seq 623906, ack 3314, win 110, options [nop,nop,TS val 1178738167 ecr 2623900673], length 0
>
> I'm not sure that "tos 0x8" is particularly useful. It's DSCP 1, and I think
> it's actually being marked TOS-like.
Your rsync over ssh should get marked background (CS1) somehow, and
ssh interactive, interactive.
openssh has different behavior than dropbear. Dropbear only recently
added some support
for saner classification.
> Will any value here have any affect on default 3.10 qos.sh script?
> --
> ] Never tell me the odds! | ipv6 mesh networks [
> ] Michael Richardson, Sandelman Software Works | network architect [
> ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on rails [
>
>
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
--
Dave Täht
Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
^ permalink raw reply [flat|nested] 3+ messages in thread