From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0B2343CB38 for ; Tue, 18 Aug 2020 02:41:09 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1597732868; bh=1+DSJhAUySE7veo04YNf8BZVo4h+PgPYxAU000I7If0=; h=X-UI-Sender-Class:Date:In-Reply-To:References:Subject:To:From; b=bkK0JoFld1C1ePIkEIwfEPgHkoYHfvt9D1P5RzXvz0AALzLVYXyOeCAsu4qPVLVOh CEvnGff6QUQOY8DcXFnvwi2Hlcq53+AcA7ZcqE5qpopOMZHtIplq3DKsKVmfdzVLrU SXRlRkgmmA8Eg+Z628b+5lhASiPTVixOZmJ3BMCI= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.42.159] ([77.3.153.8]) by mail.gmx.com (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MdefJ-1kgz8n18Ub-00ZfnG; Tue, 18 Aug 2020 08:41:08 +0200 Date: Tue, 18 Aug 2020 08:41:06 +0200 User-Agent: K-9 Mail for Android In-Reply-To: <530c76a4-d51f-29d0-c9de-2a7ba70de664@gmail.com> References: <530c76a4-d51f-29d0-c9de-2a7ba70de664@gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----F4F9A1WJSCAUNPFSXEOF6DW6QT52L1" Content-Transfer-Encoding: 7bit To: bloat@lists.bufferbloat.net,Jonathan Morton From: Sebastian Moeller Message-ID: X-Provags-ID: V03:K1:Hccdtzgd96JShBt33xeU01lM4Jh2z3AatFPezWeM+12r5PmXQYd 9CPRJH1+fCYi+g4XxbtjAiPI51cb09+AYYu7lmvbiP3aS419sYpcX3bnNAbWeDa8nDnRixq 1ldTcJDHNRMe2rFFmtMpoBMMG7WUPqmHkz0Fp/DolGXHvcnuKu3eR8OCIXgSStlQhwKeKbU /kFs1aIR6wINyDbAZ1cdQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:6cNsUYaGfm8=:EklHrzautU2StoGA0U33Ki 6ZYtDpUfUEzJbf9Oer8/f7MsVhwXaMPIE91m5wsSyXGsOoeNBX5ThYhpHA9ngKY6W+/M3sqIV R2+oFQOxPgV+mjogWh4hemne9wPb6bI91dLMn2xoJ/c0SAUXc2IBhvTZg8R5r8AFsuRwP8HAC 4kQvyMgcJMhlNSiwl2pmXvYE+UhYLubPLixpF5w9X47Bwjn33m/+LzyJVweBK5PCVzEiRpTmg 9cBD2cIBDFly027vFhNLLzSS3Nt4ZeWnPPj5SwwyuEeqoSrcAvRg/sJoEqx4AQZGbmdW06TTe Mw5ZQxWJBvyqd97jpmsHB8hS1xDtwDLylLT15MS+4b64J/MaZyUAc4O0DaoB6rFls7pReZxir 4tbetuyr7qKNXLJFY7uHXgKhkax9FHd020HiIKiCi4iqu1tsx72cgDrj7Yd0rSgfZwN2JWcEH W4V+k4UCrDmGlupCQKUw3aA6APWKM4YEJhgxpi0zQhwA0M8MgCBMRCnKb8mr0Yn9ECXBiOTwH Ij0yFnTA9BBcdQxcJbOueJYPAuSkfjZCmy7UNiPSo6l55E0kW3IOCcj0CemgtRIMoP7FYMKha sqK2rRMzXOP2n4iIB81B0ItUB3VsY/hqom6gpUlPMqL2lGPiNKc+c3F2TzTnPxzzoYmnU2EFC 1wSmrYnl5dKsw4glxKgoZDpXXqxO/1yNlZXtyqCFqxpllia4DHwtJYUTdaD9yEDDPs3Y8aLqk gzJ+SUM3av3R/n2L5dpF7EOsnQyqp7sSZiC+F5x4qf+acjL2behpkpkSjOOISRfrBmhobtpS5 83gJk0z2oBm2mMwTgqoA6XdUJe5JGUfv0LYvNJF0fW2nk6uGQNTphLM92RUc9LvQX6USCI/gl kHJNpDl9iK0YT93PLHWwUkM1UTGy1UzZJHfmEIygaA10xihYnbEBw8c1v8LgA+upKpvV3daau EeKwTw5UFYQENp0KAa9IsZbT/TrQaFbW+246ZaAOJn2IoG8jZ9uEiFWpjpUY5+PPifysMl4cV S0ZODhIf9I93qtCgs6EC+mFc2aasiBSLfcsxkE+kf1L5TY9DusP7lmZnQeeKs0+VkSvOSjMdG hxJtzBq2WGShu+tdFrpc7/pd2DJifJ0Y7M7185j3HfKv+irgLSY6VvdufdklZdq0Kon60mAWO SijUiyKzRTxPQIRRv5EBpznsnDSwfj7zWgrF9YWDlhnI7Xi+ILgaUx9onabPF9zrfI3ALoQ3c UUL0qWAJdw0fKz0is Subject: Re: [Bloat] cake + ipv6 X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 06:41:10 -0000 ------F4F9A1WJSCAUNPFSXEOF6DW6QT52L1 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi there, A few related thoughts=2E Some people use sldnsmasq to populate ipsets bas= ed on dnsnames to figure out which IP addresses are used by, say YouTube, a= nd then dscp mark them to their own desire=2E Others use iptables rules to = detect bulk flows by their accumulated transfervolume and then dscp mark t= hem=2E The common thread here is that these techniques fo not work with the= IFB way of ingress shaping, since the IFB runs before iptables/dnsmadq/ips= et actually has seen the packets and hence they carry their original useles= s DSCP marks=2E If your router does not use wifi, you can instantiate the i= ngress Shaper on the egress side of the interface from the CPU to the lan s= ide, or you can use a veth pair to route ingress traffic into the lan bridg= e and then instantiate the ingress Shaper on the egress side of that veth= =2E=2E=2E Best Regards Sebastian P=2ES=2E: As far as I can tell IPv6 end hosts will cycle through new addre= sses, but typically should only use a few concurrently=2E It would be quite= interesting to figure out whether the Xbox updates truly use multiple IP a= ddresses in the first place or whether this is a thundering herd problem ca= used by synchronized updates by multiple hosts, no? On 18 August 2020 06:15:55 CEST, Jonathan Morton = wrote: >On 18/08/2020 06:44, Daniel Sterling wrote: >> =2E=2E=2Eis it possible to identify (and thus classify) >> plain old bulk downloads, as separate from video streams? They're >both >> going to use http / https (or possibly QUIC) -- and they're both >> likely to come from CDN networks=2E=2E=2E I can't think of a simple way= to >> tell them apart=2E > >If there was an easy way to do it, I would already have done so=2E We >are=20 >unfortunately hamstrung by some bad design and deployment around=20 >Diffserv, which might otherwise provide a useful end-to-end visible=20 >signal here=2E > >> Is this enough of a problem that people would try to make a list of >> netblocks / prefixes that belong to video vs other CDN content? > >It's possible that someone is doing this, but I don't specifically know > >of such a source of information=2E It would of course be better to find >a=20 >solution that didn't rely on white/black lists, which have a >distressing=20 >habit of going stale=2E > >But one of the more reliable ways might be to use Autonomous System >(AS)=20 >information=2E ASes are an organisational unit used for assigning IP=20 >address ranges and for routing, and usually correspond to a >more-or-less=20 >significant Internet organisation=2E It should be feasible to map an=20 >observed IP address to an AS, then look up the address blocks assigned=20 >to that AS, thereby capturing a whole range of related IP addresses=2E > >> I do notice video streams are much more bursty than plain downloads >> for me, but that may not hold for all users=2E >>=20 >> That is, for me at least, a video stream may average 5mbps over, say, >> 1 minute, but it will sit at 0mbps for a while and then burst at >> 20mbps for a bit=2E > >Correct, YouTube at least likes to fetch a big block of data from disk=20 >and send it all at once, then rely on the client buffer to tide it over > >while the disk services other requests=2E It makes some sense when you= =20 >consider how slow disk seeks are relative to the number of clients they > >need to support, each of which will generally be watching a different=20 >video (or at least a different part of the same one)=2E > >However, this burstiness disappears on the wire just when you would >like=20 >to use it to identify traffic, ie=2E when the video traffic saturates the > >bandwidth available to it=2E If there's only just enough bandwidth, or= =20 >even *less* than what is required, then YouTube sends data continuously > >into the client buffer, trying to keep it as full as possible=2E > >There are no easy answers here=2E But I've suggested some things to look > >for and try out=2E > > - Jonathan Morton >_______________________________________________ >Bloat mailing list >Bloat@lists=2Ebufferbloat=2Enet >https://lists=2Ebufferbloat=2Enet/listinfo/bloat --=20 Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E ------F4F9A1WJSCAUNPFSXEOF6DW6QT52L1 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi there,

A few related thoughts=2E Some pe= ople use sldnsmasq to populate ipsets based on dnsnames to figure out which= IP addresses are used by, say YouTube, and then dscp mark them to their ow= n desire=2E Others use iptables rules to detect bulk flows by their accumul= ated transfervolume and then dscp mark them=2E The common thread here is t= hat these techniques fo not work with the IFB way of ingress shaping, since= the IFB runs before iptables/dnsmadq/ipset actually has seen the packets a= nd hence they carry their original useless DSCP marks=2E If your router doe= s not use wifi, you can instantiate the ingress Shaper on the egress side o= f the interface from the CPU to the lan side, or you can use a veth pair to= route ingress traffic into the lan bridge and then instantiate the ingress= Shaper on the egress side of that veth=2E=2E=2E

Best Regards
= Sebastian

P=2ES=2E: As far as I can tell IPv6 end hosts will cy= cle through new addresses, but typically should only use a few concurrently= =2E It would be quite interesting to figure out whether the Xbox updates tr= uly use multiple IP addresses in the first place or whether this is a thund= ering herd problem caused by synchronized updates by multiple hosts, no?
On 18 August 2020 06:15:55 CEST, Jonathan M= orton <chromatix99@gmail=2Ecom> wrote:
On 18/08/2020 06:44, Daniel Sterling wrote:
=2E=2E=2Eis it possible to iden=
tify (and thus classify)
plain old bulk downloads, as separate from vide=
o streams? They're both
going to use http / https (or possibly QUIC) -- =
and they're both
likely to come from CDN networks=2E=2E=2E I can't think=
 of a simple way to
tell them apart=2E

If there was = an easy way to do it, I would already have done so=2E We are
unfortuna= tely hamstrung by some bad design and deployment around
Diffserv, which= might otherwise provide a useful end-to-end visible
signal here=2E
=
Is this enough of a pr= oblem that people would try to make a list of
netblocks / prefixes that = belong to video vs other CDN content?

It's possible tha= t someone is doing this, but I don't specifically know
of such a source= of information=2E It would of course be better to find a
solution tha= t didn't rely on white/black lists, which have a distressing
habit of g= oing stale=2E

But one of the more reliable ways might be to use Auto= nomous System (AS)
information=2E ASes are an organisational unit used= for assigning IP
address ranges and for routing, and usually correspon= d to a more-or-less
significant Internet organisation=2E It should be = feasible to map an
observed IP address to an AS, then look up the addre= ss blocks assigned
to that AS, thereby capturing a whole range of relat= ed IP addresses=2E

= I do notice video streams are much more bursty than plain downloads
for = me, but that may not hold for all users=2E

That is, for me at least,= a video stream may average 5mbps over, say,
1 minute, but it will sit a= t 0mbps for a while and then burst at
20mbps for a bit=2E

Correct, YouTube at least likes to fetch a big block of data from dis= k
and send it all at once, then rely on the client buffer to tide it ov= er
while the disk services other requests=2E It makes some sense when = you
consider how slow disk seeks are relative to the number of clients = they
need to support, each of which will generally be watching a differ= ent
video (or at least a different part of the same one)=2E

Howe= ver, this burstiness disappears on the wire just when you would like
to= use it to identify traffic, ie=2E when the video traffic saturates the bandwidth available to it=2E If there's only just enough bandwidth, or even *less* than what is required, then YouTube sends data continuously <= br>into the client buffer, trying to keep it as full as possible=2E

= There are no easy answers here=2E But I've suggested some things to look <= br>for and try out=2E

- Jonathan Morton
Bloat mailing list
B= loat@lists=2Ebufferbloat=2Enet
https://lists=2Ebufferbloat=2Enet/listinfo/bloat

--
Sent from my Android device with K-9 M= ail=2E Please excuse my brevity=2E ------F4F9A1WJSCAUNPFSXEOF6DW6QT52L1--