From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8A35D3B2A3 for ; Tue, 25 Apr 2017 17:16:53 -0400 (EDT) Received: by mail-wm0-x22b.google.com with SMTP id u65so33714943wmu.1 for ; Tue, 25 Apr 2017 14:16:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=k/1uKDvuDO0dWA0J//6wIT3hB4ZAF2fCDQtcIjUTmb4=; b=o125peIB2GhQ+RFXG0BR1p2I2TUicN45pLCKBFj9NWzgsEsQIYxdfzdmodApn2aA4A 1JSAX8LwKUF2N16NGKRi8ujYJ+jsrENdQbbwJuSSmg3Yc3pCbNh68s/HbmG/h7CWevgV ELqW6ZcDPRJaEyQIlWwkJTlPGaSMdQ9k+hzIg2KfZTvoK60voqgLYTjT8F4kKarOVkMR 2GmHvV1I5PyvPBWlC8Pf74VaFPGRDxfD6FL9YEmGk9/ql+CzrPXTmzwja98xQtzzC1o9 zAPK2tMPOeFOi2el63E3rzvEtIUs+UHeg9a1aFWFPtcXBHWW7blNtXzB3rSqhke8cHn7 V8uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=k/1uKDvuDO0dWA0J//6wIT3hB4ZAF2fCDQtcIjUTmb4=; b=dlyzlu2KxFORYPl6bWI7k85nXhf7b0Kr7MOZKb14D+bRgoqs5EmZAavQSe2h9ENlWd 0vNpN8Y2qb1VTishdf+/u7JI9pSM3VukNfhG8WM+U0NxXQaxHhC1OInJlvy23y+6OU/Y DC18kiPXQsnAcH1UhyBN+hQbKR8DE71yHxWob/h5LncccQBEzNdbeBpicXYaOTZZTWkX wS4L60ZDSoFNz1t5yTP9dEDp4R1hi2knw/lmkO7BEZ37Q2DDpbAd7x94H40s7Yd2yIQX b0cRPsccIopbSz4JIRDFqkhRSLxvIJiebTOA1Yy8JNgcUWmSGBhES5SkWSB/VDgN8unF 7dyw== X-Gm-Message-State: AN3rC/6s+n6+Ekc8ie7YnSFDUO3q6bNdhdn5FyOddANaEeunaEJc/BBX BhCHIFp6JShRzmtXqU0TPZBdKiUcDQ== X-Received: by 10.28.29.72 with SMTP id d69mr3335336wmd.25.1493155012408; Tue, 25 Apr 2017 14:16:52 -0700 (PDT) MIME-Version: 1.0 References: <22E633CF-5EE0-4B0F-89A8-B790E730FB6C@gmx.de> <0BA3EE91-C5BC-4155-9D5D-D15D34490A1A@gmx.de> <00DDAA0B-7D99-489B-BA2D-1F20289409B3@gmx.de> <2FFBF256-2932-4FC7-AD1F-0D7CEE111809@gmx.de> <3fbfd0ee-7b41-0f83-8b44-ce7eed6a0562@gmail.com> <09DB0D8E-F63C-4126-8608-9EACDE99D2F1@gmail.com> <7ac3ea6a-885c-f7a0-e01f-a824e0656e15@gmail.com> In-Reply-To: <7ac3ea6a-885c-f7a0-e01f-a824e0656e15@gmail.com> From: Neil Shepperd Date: Tue, 25 Apr 2017 21:16:41 +0000 Message-ID: To: Andy Furniss , Jonathan Morton , Dendari Marini Cc: cake@lists.bufferbloat.net Content-Type: multipart/alternative; boundary=001a114b85ce1bf8c0054e043ed8 Subject: Re: [Cake] Getting Cake to work better with Steam and similar applications X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Apr 2017 21:16:53 -0000 --001a114b85ce1bf8c0054e043ed8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Um, I wasn't sure if I should mention it, because it doesn't seem like it should be able to cause these kinds of issues. But, if you're using steam *= on linux*, there's a known bug where it makes hundreds (thousands?) of DNS queries per second, during downloads, which can cause issues if the DNS server on your router starts throttling. I don't know how or if that should affect the apparent performance of cake in different tests. But the workaround is to have a local DNS cache like dnsmasq on your host (and of course it's not an issue on Windows machines). On Tue, 25 Apr 2017 at 17:06 Andy Furniss wrote: > Jonathan Morton wrote: > > > >> On 25 Apr, 2017, at 21:22, Dendari Marini > >> wrote: > >> > >> The good news is that using switch0 as inbound and pppoe0 as > >> outbound works, and I was able to set up Steam as bulk using the > >> interface on the ER-X (used DSCP 8 and used a custom DPI category). > >> I confirmed this was working by looking at the bulk traffic > >> increasing (using the "tc -s qdisc" command) and by starting > >> another download (Steam gets pretty much nothing in this case). > >> > >> The bad news is this isn't enough to fix my gaming issue (still > >> having ping spikes, latency variation and packet loss), and even > >> using it with Steam configured to use just one connection didn't > >> change much from my previous testing. > >> > >> So I'm really confused :\ What could cause ping spikes in this > >> case (assuming the multiple connections aren't the issue)? > > > > As noted, it=E2=80=99s far more difficult to control latency from downs= tream > > of a bottleneck link. If a bulk sender decides to send burstily, > > those bursts will always collect in the dumb queue at the far end and > > delay other traffic. The only true solution is to install a smart > > queue at the upstream end - but that=E2=80=99s not under your control. > > > > You may see some improvement from wholesale reducing the inbound > > bandwidth, to say 10Mbit. This is especially true given the high > > asymmetry of your connection, which might require dropped acks > > upstream to keep filled downstream - and dropped acks will tend to > > increase burstiness of sending on unpaced senders. > > > > You should also try to ensure ECN is fully enabled on your LAN hosts, > > especially the ones running Steam. This will help to reduce > > retransmissions and loss-recovery cycles. > > Yea, good idea - hopefully steam servers will honor that. > > Plus remember what I said about raw - you have an upstream fail case > with sacks - also see below. > > I just looked at a tcpdump I made last night doing a steam d/l, mainly > to see if I got servers in the list from the steam link posted, I did. > > The other stand out thing is there are far more sacks than I would > normally expect to see. > > tcpdump -nnr steam2.pcap src host 192.168.0.224 | grep "sack" | wc -l > reading from file steam2.pcap, link-type EN10MB (Ethernet) > 28298 > > tcpdump -nnr steam2.pcap src host 192.168.0.224 | wc -l > reading from file steam2.pcap, link-type EN10MB (Ethernet) > 44094 > > incoming count - > > tcpdump -nnr steam2.pcap dst host 192.168.0.224 | wc -l > reading from file steam2.pcap, link-type EN10MB (Ethernet) > 59319 > > That's a crazy amount especially as being in recovery means ack per packe= t. > > My connection is quite different from the OPs of course, ptm 66 meg sync > with cake at 60mbit for this test and the pcap was only over 12 seconds. > The servers are only 8ms away from me, 5 connections. > > I wonder if they are using bbr or something. > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake > --001a114b85ce1bf8c0054e043ed8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Um, I wasn't sure if I should mention it, because it d= oesn't seem like it should be able to cause these kinds of issues. But,= if you're using steam on linux, there's a known bug where i= t makes hundreds (thousands?) of DNS queries per second, during downloads, = which can cause issues if the DNS server on your router starts throttling. = I don't know how or if that should affect the apparent performance of c= ake in different tests. But the workaround is to have a local DNS cache lik= e dnsmasq on your host (and of course it's not an issue on Windows mach= ines).

On Tue, 25 Apr = 2017 at 17:06 Andy Furniss <adf.l= ists@gmail.com> wrote:
Jonat= han Morton wrote:
>
>> On 25 Apr, 2017, at 21:22, Dendari Marini <dendari92@gmail.com>
>> wrote:
>>
>> The good news is that using switch0 as inbound and pppoe0 as
>> outbound works, and I was able to set up Steam as bulk using the >> interface on the ER-X (used DSCP 8 and used a custom DPI category)= .
>> I confirmed this was working by looking at the bulk traffic
>> increasing (using the "tc -s qdisc" command) and by star= ting
>> another download (Steam gets pretty much nothing in this case). >>
>> The bad news is this isn't enough to fix my gaming issue (stil= l
>> having ping spikes, latency variation and packet loss), and even >> using it with Steam configured to use just one connection didn'= ;t
>> change much from my previous testing.
>>
>> So I'm=C2=A0 really confused :\ What could cause ping spikes i= n this
>> case (assuming the multiple connections aren't the issue)?
>
> As noted, it=E2=80=99s far more difficult to control latency from down= stream
> of a bottleneck link.=C2=A0 If a bulk sender decides to send burstily,=
> those bursts will always collect in the dumb queue at the far end and<= br> > delay other traffic.=C2=A0 The only true solution is to install a smar= t
> queue at the upstream end - but that=E2=80=99s not under your control.=
>
> You may see some improvement from wholesale reducing the inbound
> bandwidth, to say 10Mbit.=C2=A0 This is especially true given the high=
> asymmetry of your connection, which might require dropped acks
> upstream to keep filled downstream - and dropped acks will tend to
> increase burstiness of sending on unpaced senders.
>
> You should also try to ensure ECN is fully enabled on your LAN hosts,<= br> > especially the ones running Steam.=C2=A0 This will help to reduce
> retransmissions and loss-recovery cycles.

Yea, good idea - hopefully steam servers will honor that.

Plus remember what I said about raw - you have an upstream fail case
with sacks - also see below.

I just looked at a tcpdump I made last night doing a steam d/l, mainly
to see if I got servers in the list from the steam link posted, I did.

The other stand out thing is there are far more sacks than I would
normally expect to see.

tcpdump -nnr steam2.pcap src host 192.168.0.224 | grep "sack" | w= c -l
reading from file steam2.pcap, link-type EN10MB (Ethernet)
28298

tcpdump -nnr steam2.pcap src host 192.168.0.224 | wc -l
reading from file steam2.pcap, link-type EN10MB (Ethernet)
44094

incoming count -

tcpdump -nnr steam2.pcap dst host 192.168.0.224 | wc -l
reading from file steam2.pcap, link-type EN10MB (Ethernet)
59319

That's a crazy amount especially as being in recovery means ack per pac= ket.

My connection is quite different from the OPs of course, ptm 66 meg sync with cake at 60mbit for this test and the pcap was only over 12 seconds. The servers are only 8ms away from me, 5 connections.

I wonder if they are using bbr or something.
_______________________________________________
Cake mailing list
Cake@lists.= bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake
--001a114b85ce1bf8c0054e043ed8--