From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3AAC13B29D for ; Fri, 3 Apr 2020 16:44:49 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1585946687; bh=CXE12kqLpef3btK78R34Glo3T/CU7da4JwUaAD9inOY=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Kj6Epxb3iALQLGT5eD4CNxEQ2ER+Ixi8wLsb91lpg+QF27qq+MkCQ6V1nFzgCG0qQ 9K0U8i9lpEz41USVzaNeAEzyutK/YhM+RsGd+DRhmp3I/M+BEPgvPDnZtlwMXpKuf8 qy0srPDocjY/3zZiGqR1kS6jKXaNMDzLFdlyHLYw= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from hms-beagle2.lan ([95.112.6.42]) by mail.gmx.com (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1Mq2nA-1ixljZ0UTq-00n927; Fri, 03 Apr 2020 22:44:47 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.14\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 3 Apr 2020 22:44:45 +0200 Cc: "Alexander E. Patrakov" , Cake List Content-Transfer-Encoding: quoted-printable Message-Id: References: To: =?utf-8?Q?Dave_T=C3=A4ht?= X-Mailer: Apple Mail (2.3445.104.14) X-Provags-ID: V03:K1:3P6IgRyYE/y6Yv28aZwe/ZGziHFTwJHArGL0cKRJjOxEx27UNaV /NCBJOBi6z5yX+ig4v5LSVYS7zAOGOCkVSvYg9wITKCklWuytRhZXErfrvnyvThcN4x+9+U 3YEQeA9TTyiD1V+PZpiM7TngWC+VRZ5QMAj8HsXiC75DY7a6USk2P9X2RHiN82uLEcn3ZtN PdA2oPlfgds+NJjqpchSQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:C69kDcPQ1PE=:e+AyeuAtEi/JzTBCdTpcwK /hKHwU6VmTb7HCn4A72kPMLX0Dhd0rdsCCrFHiDpDah/BqQ/wrV83vVx7CgGCBAvO9DxZe5Z9 aakO92toPABvXL6FN0P3p0fz0NOHLvqqMZEFNiA1hLpHfSCIahsvSONz2xSZ+Z76KCzYUw1Xx w3EH2drMb2XyjYXfR190fBmXgDj95/w6q8gmRQCO6Osot0rTKKUjOvrYbZ1fc297KSFeevbg/ Yf3beJtwbUAacP+eNFhzT3Bs4XFrP0gdZCV60Jvm9K++7GTw9YiWbfkuYI6q5TmZJrk0IFtqX SRjr9owD82IbdkZjMEpFLqq/282mVNCbppwvCUjuacRXFsnat5p6Z3N4O14g81HlvBrykhJNE m9rgt16FsSV92IOXFQ6iIlRSIk5YQ991wnf5GH3yVkzgqDdE/sWx2jMVnedCGMaASR+NJO7nx 0WnB2Q9DdowNoBIim6x9Y/b0NgVkqT8ScHIh5KkKqa+aSEP4EdoNP1LBA1uEGtTlw88kz54o6 wdCWZ3VvbUDdioTcvnL7VqrpRZPiZK95bIraL/k7ISuJ++qfPY3DB6mkPgLUswgUnbLfOsAa5 TsQo97fptJ455aLkZEeCNMmZJkjiOWyHiJus2WokMnGlKbkVU/xE18yz1cWRFidauZrqvgWxC qLlkkTGeSialSMBTTXFP+UWr/gNVt8RzEMI0BIJHKeqC3eL+dNht8putJxWazppYShSZMfgTN eTJ8Bhqh7f3L6g3eZeTMr4EDf0IUUT8Yh8cjslquv8GPf5WWoxtxSyM2Hn6VUSmHYxYEq06YB yPSXXLmRSkC2blpdaWH5aQmJogLamZflcIUciOHDBKaQ0BkPqNdztHbA44R0e1rekPjodYI+r 8KLLZtatf1YhE9Kz4GAZtXSNaYV+eNQTD7jSv2UtYtG25Pm43Sw5v+Bl1luNYs02Ad8hUUXzT pYFEPabKsYH4ltSS8/JngORnid23gZB9gOdbpgVqqPIPRuLtnWyfR/eqrwffjrUEusDAhDZpm Ek8HkfFLvC0iMtCdgug7pfTe8KLiomrjtBPn3kDtkytQXuztqtBk7raiymZi4krJjjBDvtsyp bU7QddkBVA9VgnzBxlKyhwxBerDTK4zMkPJn2Z/bFtgrQvLtqaGMoTVAmVvco7RRgUtnoKZ7Q xV+d8jfG1wxSC4C1YSAD2NCKNYqRHaxo0dtbAipUEdVHvS1JM10Kkfhs+AholRApkgpyrVrH1 NWAZXf7LjJZncXDsV Subject: Re: [Cake] tc-cake(8) needs to explain a common mistake X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Apr 2020 20:44:49 -0000 > On Apr 3, 2020, at 20:49, Dave Taht wrote: >=20 > so nice to know cake has made it to russia!!! >=20 > On Fri, Apr 3, 2020 at 11:46 AM Alexander E. Patrakov > wrote: >>=20 >> Hello, >>=20 >> there is a recurring cargo cult pattern in many forums (e.g. = OpenWRT): >> people keep suggesting various overhead compensation parameters to >> tc-cake without checking what's the bottleneck. They just assume that >> it is always related to the link-layer technology of the connection. >>=20 >> This assumption is mostly incorrect, and this needs to be explained = in >> the manual page to stop the cargo cult. E.g., here in Russia, in the >> past year, I had a 1Gbit/s link (1000BASE-X) but they shaped my >> connection down to 500 Mbit/s because that's the bandwidth that I = paid >> for. I.e. the link from my router to the ISP equipment was not the >> bottleneck, it was the ISP's shaper. >>=20 >> How about the following addition to the tc-cake(8) manual page, just >> before "Manual Overhead Specification"? Feel free to edit. >>=20 >> General considerations >> ------------------------------- >>=20 >> Do not blindly set the overhead compensation parameters to match the >> internet connection link type and protocols running on it. Doing so >> makes sense only if that link (and not something further in the path, >> like the ISP's shaper) is indeed the bottleneck. Well, In general yes, but in reality a competent ISP will = configure its shapers to account for the link properties, so assuming = the access link's overhead/encapsulation gives a reasonable first guess = at what values might be optimal. >>=20 >> Example 1: the ADSL modem connects at 18 Mbit/s, but the ISP further >> throttles the speed to 15 Mbit/s because that's what the user pays >> for, and does so with a shaper that has bufferbloat. Then, the "adsl" >> keyword is likely not appropriate, because the ISP's shaper operates >> on the IP level. The bandwidth needs to be set slightly below 15 >> Mbit/s. Let's run the number shall we? I simply make a few assumptions = here to get things started, but the exact numbers really do not matter = too much. With that said, let's assume TCP/IPv4 and ATM/AAL5, PPPoE, = LLC/SNAP, RFC-2684; Overhead (bytes): PPP (2), PPPoE (6), Ethernet Header (14), Ethernet PAD = [8] (0), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR (8) : = Total 40 Let's see what the link will be able to deliver for "full" MTU 1500 = packets (quotes as the MTU1500 will only carry to the PPPoE endpoint, = internet MTU is going to be 1492) gross-rate * ((payload size) / (on the wire size)) =3D net speedtest = result (let's use this as proxy as this is what people can easily = verify/check) MTU1500: 18.000 * ((1500-20-20-8) / (ceil((1500-8+40)/48)*53)) =3D = 15.410 MTU150: 18.000 * ((150-20-20-8) / (ceil((150-8+40)/48)*53)) =3D = 8.66037735849 MTU75: 18.000 * ((75-20-20-8) / (ceil((75-8+40)/48)*53)) =3D = 3.05660377358 Now the IP-level shaper at ~80% of the link-speed, if it does not = account for the ATM/AAL5 "celling" even if it gets the overhead = correctly will give the following: MTU1500: 15.000 * ((1500-20-20-8) / (ceil((1500-8+40)/1)*1)) =3D = 14.2167101828 MTU150: 15.000 * ((150-20-20-8) / (ceil((150-8+40)/1)*1)) =3D = 8.40659340659 MTU75: 15.000 * ((75-20-20-8) / (ceil((75-8+40)/1)*1)) =3D 3.78504672897 So for large enough packets static accounting for ATM/AAL5 works = reasonably well, but for small packets it fails. That is why most ISP-grade equipment allows not only to configure the = per-packet-overhead for end-user links but also can deal with ATM/AAL5. = And as far as I understand most competent ISPs actually configure their = traffic-shapers for ADSL links to do this, because DSLAMs are really = more like L2-switches with fancy media-converters attached and deal not = terribly well with overload and queueing into the switch fabric. That in turn leads to the following situation: MTU1500: 15.000 * ((1500-20-20-8) / (ceil((1500-8+40)/48)*53)) =3D = 12.842 MTU150: 15.000 * ((150-20-20-8) / (ceil((150-8+40)/48)*53)) =3D 7.217 MTU75: 15.000 * ((75-20-20-8) / (ceil((75-8+40)/48)*53)) =3D 2.547 which will obviously not cause packet buffering in the DSLAM for any = packet size mix the link might encounter. AND that in turn means that = the actual bottleneck link (the ISP's traffic shaper) still behaves like = it would employ ATM/AAL5 encapsulation, and hence the end-user's SQM = instance should do as well. >>=20 >> Example 2: the ADSL modem connects at 18 Mbit/s, and the user pays = for >> "as fast as the modem can get" connection. Then, the "adsl" keyword = is >> relevant, and the bandwidth needs to be set to 18 Mbit/s. IMHO that is simply a less opaque version of example 1, but I = agree here ATM/AAL5 accounting needs to be done... >>=20 >> Example 3: the user has a 100BASE-TX Ethernet connection, and pays = for >> the full 100 Mbit/s bandwidth (i.e. there is no shaper further up). >> Then, the "ethernet" keyword is relevant, and the bandwidth needs to >> be set to 100 Mbit/s. Yes, if the true bottleneck is an ethernet link, then ethernet = overhead needs to be accounted for, so overhead 38 bytes (or 42 if VLANs = are in use) would be correct.=20 I fully concur that properly accounting for the properties of the true = bottleneck is the challenge here. And with few exceptions this is hard, = as we have two dependent variables to set gross shaper rate and = per-packet-overhead. Looking at the numbers, I have reluctantly become = convinced, that getting the per-packet-overhead correctly is not that = important as long as one does not under-estimate it, so if in doubt, I = rather recommend to err on the side of too much (but I value low latency = above maximum throughput, like most people starting to look at sqm). Best Regards Sebastian P.S.: I am of the opinion, that = https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details = had very sane and un-cargo-culty advice about the overhead topic: "Getting [overhead and link layer accounting] exactly right is less = important than getting it close, and over-estimating by a few bytes is = generally better at keeping bufferbloat down than underestimating. With = this in mind, to get started, set the Link Layer Adaptation options = based on your connection to the Internet. " I am less sure about the paragraph you added recently, as it does not = seem to consider all the applicable subtleties. >>=20 >> -- >> Alexander E. Patrakov >> CV: http://pc.cd/PLz7 >=20 >=20 >=20 > --=20 > Make Music, Not War >=20 > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-435-0729 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake