From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E7E5E3B2A0 for ; Fri, 10 Jun 2016 10:26:19 -0400 (EDT) Received: from [10.34.152.96] ([80.187.99.142]) by mail.gmx.com (mrgmx102) with ESMTPSA (Nemesis) id 0MC7em-1bKD4Y4AL2-008rDM; Fri, 10 Jun 2016 16:26:16 +0200 User-Agent: K-9 Mail for Android In-Reply-To: References: <7409a52d-8c81-25f0-e070-c7638fdf9d83@gmail.com> <0B4652E3-F061-479D-8BB3-EA0193333A51@gmx.de> <3d4cdbf4-42dd-a371-2d44-f1bad6505684@gmail.com> <735b7505-584f-6c94-2047-110254e73ee8@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable From: Sebastian Moeller Date: Fri, 10 Jun 2016 16:26:01 +0200 To: Dennis Fedtke CC: cake@lists.bufferbloat.net Message-ID: <9D507B5F-65EA-4877-9046-343BAC5E79AA@gmx.de> X-Provags-ID: V03:K0:Zgh2hAyn837Yj/ZtkL7xZXl6l85JmswF4qAA45He1H9VOZstN0U Z9bdjfKZcbDatz7qfUZtR1HIGriuaeYoB/Xhu/1+0m1u8X13l/pf5+iDb8kqp5Qg95sXUka 4IG+NYRpVgyt/nIW8k9dct7TEfT96lZ4wzFrES2kS+60TfGWI033+k4fDhfL9AvAZYQJA+d Ubl9LUSCCmd9KJ1mrOrog== X-UI-Out-Filterresults: notjunk:1;V01:K0:TRmKkgv5eqg=:CmSvcpT3zuklbH8dk+iAIZ 6vh6G/CErYeFEZ/effUmGAjPfb+ymPFWi+LtDl5ZVY3c/+mqZCPVRwyToL2pExR0sbnvhzHbn 0jqDSVj7D3iY+9cs7hUSFzBRlQJL7ExigeAeD0itJDz67qhoK8dm8VzJg/KbnZx5FKyH2432r fT8mNHZ79HB/KyOHdwDq/S8JEeGp5qhkoEiiq24gaDwMjipWl62gJnbgtc8P/8ShP/s+WDDLV xPxun0AyMlkLB6HNbLzj7ika+aWypiymW/D1IJsPZuQ9awmDNLW0v1noN8ejxngs7DO8nTV86 2K0WOHIj3FPPkm2aAUiyfGi5hZhqy/EwnMfHAYnYWOgoB8r0LFnwffOdO6gGme3dKqduxZQPB NOa+hHssZ/tCSNMHZxsYkLpE7ReV1yGdXeEXNw5TEgIlo9oyK6hcfjTfx8o8jM8SLQVyv+QTb w7T0m7wvZKmXKrpPmuqsbP2DqqRy9nMS0eRNWE58hgcI3oZ57ey9nKNNR4O8FxhthJ+CeMY8t 4oHLQIe/JNpYoMwvurNM0nW4m1VTDc7/J4jUSPLjY7a8E6HBg+vGnP4BXSWhohQazTm2mji8r 89bBIOlZzoRQ0oJ4yFpMN56LV5hne6DkEL0MDB9DdDbJTttyE4dY7WbDnYLxCMaKt2XnTwdnF /ETTszut9wkJtF8m+CyilE2AOKvuvVRFYPTohxHcb8TquKGIjnuzQvF1Y2GPvQJ7IeH5/klZ8 vSmwZ2flJq0/8AK2VFFVrm6XCp8yaek3CzocfF4zil+AYn1Lj5rf7DVM29PzM4LtbzRd3hqbF u7UJBXUr+zB2VI27cLdWsK5yZeDL5ysUP+NgdIbBfsGZ/8wrVY0FA0V7Rmt4P0OviJ4FP6srt m+U1ntnGP0+f78ieGqQlcnr/JqrBiMqUDZEENkDOJWv/8heuQ6Ey4uC7hU7MxCgfGE1PTgB1I +lHq2BgNR3XfubdPBJ88M6ufTTuz/BnaA54rvcEEX739JEM2/26IT Subject: Re: [Cake] New to cake. Some questions X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jun 2016 14:26:20 -0000 Hi Dennis,? On June 10, 2016 4:05:11 PM GMT+02:00, Dennis Fedtke wrote: >Hi Sebastian, > >yes this is wired connection=2E As i stated my ping times always vary=20 >independently of target=2E >My ISP is overloaded in certain regions=2E So i assume they do some=20 >shaping/limiting on certain protocols (icmp for example) >Connection speed is 200/20 Mbit=2E Okay that is a Docsis cable Link, so no atm encapsulation at all=2E There = a several lines of reasoning to one to this conclusion, but mainly ATM link= s top out at 22Mbps, and in Germany typically at 16-17Mbps, and your ISP is= a pure cable=2E Company=2E So you can stop running the overhead detector a= s that only works on ATM links, sorry=2E I have not yet found a way to meas= ure the overhead without ATM cells=2E >ISP is unitymedia which doesn't allow you to use your own hardware=2E The law changed an soon, August I believe they will have to give you the a= ccess information, but you will still need a cable modem or Docsis router= =2E=2E=2E >So actually i have to run my router behind theirs with exposed host=20 >enabled :< > >Ping response: > >ping -s 1400 -c 1 109=2E90=2Ex=2Ex >PING 109=2E90=2E28=2E1 (109=2E90=2E28=2E1) 1400(1428) bytes of data=2E >1408 bytes from 109=2E90=2E28=2E1: icmp_seq=3D1 ttl=3D253 time=3D11=2E6 m= s > >--- 109=2E90=2E28=2E1 ping statistics --- >1 packets transmitted, 1 received, 0% packet loss, time 0ms >rtt min/avg/max/mdev =3D 11=2E677/11=2E677/11=2E677/0=2E000 ms > >This looks good or? Yes the ping is fine, I assume that your node is quite overbooked an= d you the 4ms variance from the Docsis grant request system or so=2E But I = am no Docsis expert, so that could be wrong=2E=2E=2E > >Yes i am from germany=2E So you are from germany too? Yes=2E > >Thanks for your time and help :) Happy to be able to help=2E=2E=2E=2E > > >Best regards >Dennis Freundlichem Gruessen Sebastian > >Am 10=2E06=2E2016 um 15:02 schrieb moeller0: >> Hi Dennis, >> >>> On Jun 10, 2016, at 14:43 , Dennis Fedtke >wrote: >>> >>> Hi Sebastian, >>> >>> i used the default setting of 1000=2E >> Okay, that should work i assume unless you have a very fast link=E2=80= =A6 >What link at what ISP do you actually have? >> >>> But it seems that my isp is dropping icmp packets if there are >exceeding some sending threshold=2E >> I would be amazed if they did, a sympotom of that would be rsate >reduction to all ICMP probe flows independent of target host=2E If >however you only see this with specific hosts it is very likely that >that host rate limits its ICMP responses=2E In either case try another >host further upstream=2E II think I has reasonable decent results with >targeting 8=2E8=2E8=2E8, googles dns servers=2E >> >>> So there is a lot of none usable ping data=2E >> Again, try another host=E2=80=A6 >> >>> I increased the send delay to 50ms=2E 25 ms already shows dropped >requests=2E >> That might also help, as long as you stay below their throttling >rate the chosen host might still work okay=2E >> >>> This is the third run now=2E Waiting for completion=2E >> Well, sorry that the method is not as slick and streamlined, but >there are no guarded good ICMP reflectors available on the net=2E >> >>> The ping target is my first hop=2E >> Try the next hop then ;) >> >>> Actually my ping always varies around +-5ms even at idle and >independently of ping target=2E >> This is via wifi/wlan? If so try from a wired connection instead=2E >> >>> When i look through the ping file the increase in ping times are >actually appear to be random to me=2E >> Well, we expect variability of the individual =E2=80=9Ctrials=E2=80=9D= to exist, >that is why we collect so many and try to select the best measure in >the matlab code to remove the unwanted variance=2E Could you post a link >to both of the generated plots please, the first one showing te >different aggregation measures might be helpful in diagnosing the >issues deeper=2E >> >>> So how to test if my isp responses with fixed icmp packet size? >> You could try manually=2E In the folloewing example I pinged >gstatic=2Ecom (which belongs to googles CDN as far as I know): >> >> bash-3=2E2$ ping -s 1 -c 1 gstatic=2Ecom >> PING gstatic=2Ecom (216=2E58=2E213=2E195): 1 data bytes >> 9 bytes from 216=2E58=2E213=2E195: icmp_seq=3D0 ttl=3D55 >> >> --- gstatic=2Ecom ping statistics --- >> 1 packets transmitted, 1 packets received, 0=2E0% packet loss >> >> >> bash-3=2E2$ ping -s 64 -c 1 gstatic=2Ecom >> PING gstatic=2Ecom (216=2E58=2E213=2E195): 64 data bytes >> 72 bytes from 216=2E58=2E213=2E195: icmp_seq=3D0 ttl=3D55 time=3D19=2E4= 46 ms >> >> --- gstatic=2Ecom ping statistics --- >> 1 packets transmitted, 1 packets received, 0=2E0% packet loss >> round-trip min/avg/max/stddev =3D 19=2E446/19=2E446/19=2E446/0=2E000 ms >> >> >> bash-3=2E2$ ping -s 65 -c 1 gstatic=2Ecom >> PING gstatic=2Ecom (216=2E58=2E213=2E195): 65 data bytes >> 72 bytes from 216=2E58=2E213=2E195: icmp_seq=3D0 ttl=3D55 time=3D21=2E1= 38 ms >> wrong total length 92 instead of 93 >> >> --- gstatic=2Ecom ping statistics --- >> 1 packets transmitted, 1 packets received, 0=2E0% packet loss >> round-trip min/avg/max/stddev =3D 21=2E138/21=2E138/21=2E138/0=2E000 ms >> bash-3=2E2$ >> >> >> bash-3=2E2$ ping -s 1400 -c 1 gstatic=2Ecom >> PING gstatic=2Ecom (216=2E58=2E213=2E195): 1400 data bytes >> 72 bytes from 216=2E58=2E213=2E195: icmp_seq=3D0 ttl=3D55 time=3D6=2E87= 8 ms >> wrong total length 92 instead of 1428 >> >> --- gstatic=2Ecom ping statistics --- >> 1 packets transmitted, 1 packets received, 0=2E0% packet loss >> round-trip min/avg/max/stddev =3D 6=2E878/6=2E878/6=2E878/0=2E000 ms >> >> Once I try to send 65 Bytes of ICMP payload the response is cut short >to 92 bytes, the same might happen with your isp=2E But also if all your >ISP does is rate limiting the ICMP packests that still can lead to to >much variance in the RTTs=E2=80=A6 >> >> >>> Im in central europe too :D >> Ah, then you just have a different work/sleep cycle than I do ;)=2E >Where in central Europe, if I might as Ii am, as you might have guessed >based in Germany=E2=80=A6 >> >> Best Regards >> Sebastian >> >>> Thanks :) >>> >>> >>> Am 10=2E06=2E2016 um 07:20 schrieb moeller0: >>>> Hi Dennis, >>>> >>>> >>>>> On Jun 10, 2016, at 02:49 , Dennis Fedtke >wrote: >>>>> >>>>> Hi Sebastian, >>>>> >>>>> Sorry this is positive or? >>>> I would say that is unclear=E2=80=A6 >>>> >>>>> But i need more samples ? >>>> I would try with more samples, after checking that the ping times >in the recorded data file actually are larger for larger probes than >for smaller, some hosts will reply with a fixed maximum ICMP packet >instead of returning the received packet, thereby reducing the signal >range (as only the upload leg of the link is meaning fully contributing >useful differential signal=2E >>>> BTW I am in central europe so at times of the day my responses can >be very sporadic, as I either am at work or sleeping ;) >>>> >>>> Best Regards >>>> Sebastian >>>> >>>>> Thanks :) >>>>> >>>>> >>>>> Am 10=2E06=2E2016 um 01:11 schrieb moeller0: >>>>>> Hi Dennis, >>>>>> >>>>>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke > wrote: >>>>>>> >>>>>>> Hi Sebastian, >>>>>>> >>>>>>> thank you for your answers :) >>>>>>> >>>>>>> The ATM overhead detector script is currently running=2E >>>>>>> I read the wiki about it but im not quite sure how to interpret >the plot=2E >>>>>>> I mean what info should i read from it? maximum packet size? >>>>>> The relevant number is reported as =E2=80=9CEstimated overhead pre= ceding >the IP header=E2=80=9D in the top part of the second figure created by th= e >script=2E But that is only relevant=2Euseful if you see a nice step like >plot in figure 2 as well ( the second figure in >https://github=2Ecom/moeller0/ATM_overhead_detector/wiki as positive and >the fourth figure as negative example=2E >>>>>> >>>>>>> If yes do i set the overhead in cake? Or do i set iptables to >clamp to new mtu/mss? >>>>>> If you use plain cake and you know the numerical overhead (NN) >the easiest is to add the following to your cake invocation: =E2=80=9Catm >overhead NN=E2=80=9D >>>>>> >>>>>> Please note that if you use cake on an ethernet interface the >kernel will already account for 14 byte of ethernet overhead, so if the >script told you 44 as actual overhead, you use =E2=80=9Doverhead 30=E2=80= =9D to address >that=2E If you use a pppoe interface the kernel will most likely not add >the 14 bytes for you, so then you would use =E2=80=9Coverhead 44=E2=80=9D= (I excluded >the atm option in the last examples for clarity=E2=80=A6) >>>>>> >>>>>>> Regarding UDP paket dropping problem: >>>>>>> I just read some forums and users stated that under heavy load >cake starts to drop udp packets which causes lag ingame=2E >>>>>>> My idea was to set ingress/egress to diffserv4 and apply the EF >dscp mark on those packets=2E >>>>>> Ell, not a bad idea, but often the problem are in the incoming >traffic, and unfortunately with the ifb we use we can not use iptables, >but only tc, and remarking with tc is unpleasant=2E >>>>>> >>>>>>> Will this even work? if yes how to do this? iptables? >>>>>> No, you wuld need tp use tc=2E >>>>>> >>>>>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports >5000:5500 -j DSCP --set-dscp-class EF >>>>>>> >>>>>>> Like thia? Is prerouting correct here? (Taken from layer cake >script) >>>>>> This will affect outgoing packets and might be a good idea in >your specific case=2E >>>>>> >>>>>> BUT why don=E2=80=99t you try the default behaviour with specific r= ules >and tricks and report success or failure back to us, after all the >fastest/easiest classification is one one does not need to perform at >all=2E >>>>>> >>>>>>> For the squash and wash feature=2E >>>>>>> Im asking because if i choose to squash in the advanced options >of sqm scripts=2E >>>>>>> The dscp fields/marks will be overwritten by iptables to 0 >(besteffort)=2E (layer cake script) >>>>>>> So then it makes no sense to manually set dscp fields/marks or? >(Or even setting diffserv) >>>>>> No unfortunately on ingress cake sees the packets before >iptables, so the effective behavioral emulation of wash/squash by cake >is to set ingress cake to besteffort (basically cake ignores the dscp >field which functionally is identical to all packets having the same >value)=2E The squashing by iptables just clears the dscp marls so that >internal networking elements like potentially wifi liknks are not >confuzed by the dscp information=2E >>>>>> >>>>>>> Did i understand this correctly=2E Per rfc isps should not provide >dscp fields/marks? >>>>>> Not exactly, per RFC DSCPs are only ever valid/defined inside a >DSCP domain and your ISPs domain ends before it reaches your CPE=2E Since >you have no control over your ISPs markings, they can be very much not >like you want them to be (Dave That reported that his ISP re-mapped >almost 1/3 or so of packets into the CS1 background class)=2E So it is >recomended that each DSCP domain re-mapps the code points at its entry >point, which in your case is your router=E2=80=A6 >>>>>> >>>>>> Best Regards >>>>>> Sebastian >>>>>> >>>>>>> Thank you=2E >>>>>>> >>>>>>> >>>>>>> >>>>>>> Am 09=2E06=2E2016 um 23:30 schrieb moeller0: >>>>>>>> Hi Dennis, >>>>>>>> >>>>>>>> let me start with a disclaimer, I am not the best information >source for cake on this mailing list, but I assume the others will >chime in if I say something questionable=E2=80=A6 >>>>>>>> >>>>>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke > wrote: >>>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> Currently im running lede + cake + sqm_scripts and i have some >questions: >>>>>>>>> >>>>>>>>> 1=2E What is considered the =E2=80=9Coptimal" setup atm for cake= ? >>>>>>>> The same as without cake; really, proper per-packet-overhead >accounting is important for bandwidth shaping, especially for ATM >-based links=2E I would recommend to follow the method on >https://github=2Ecom/moeller0/ATM_overhead_detector to m\empirically >measure whether your link uses ATM encapsulation and what exact >overhead is in use=2E >>>>>>>> >>>>>>>>> e=2Eg=2E which cake script should i use piece or layer cake? >>>>>>>> piece_of_cake has only one tier of priority, while layer_cake >currently offers 4=2E Packets are put into the different priority bands >based on the content of their TOS/DSCP filed in the IP header; if this >is greek to you, I guess piece_of_cake most likely is what you are >looking for=2E=2E >>>>>>>> >>>>>>>>> 2=2E Recently squash and wash was removed=2E >>>>>>>>> But the sqm scripts were not updated=2E In the advanced options >should i set that the dcsp marks are kept? >>>>>>>> This really is an implementation detail that has no immediate >effect if you choose piece_of_cake as typically only the bottleneck is >sensitive to DSCP based priority banding=2E (Typically in that if you are >unlucky your WLAN will use the DSCP marks to move packets into 4 >different priority classes, which is fine if you want that, but bad for >not sanity checked packets coming in from the wider internet (one is >not supposed to assume incoming packets have sensible dscp markings as >per RFC) that is why the wash/squash option is missed by some of us, >independent of the fact that it was a layering violation)=2E >>>>>>>> >>>>>>>>> 3=2E Should i use advanced options in sqm scripts and set >triple-isolate + diffserv8 ? >>>>>>>> If you understand what these options do and believe that this >is the best for your network go ahead, otherwise=E2=80=A6 The triple-isol= ate >option will try to be fair to host_IP addresses first and then for each >hostIP fair to each flow, but for that to do something you will most >likely want this requires that cake sees internal IP addresses of your >end-hosts=2E In the typical configuration with SQM on the WAN interface >of a NAT router all internal addresses are replaced with the external >IP address of the router it self and triple-isolates per host fairness >will pretty much be equal to per flow fairness (not exactly, but in >essence)=2E So if you want to try tiple-isolate or its better defined >brothers dual-srchost and dual-dsthost you would need to instantiate >SQM on an internal interface like LAN=2E But then the direction of >ingress and egress from the routers perspective changes with regards to >the internet download and upload direction and you will need to put the >internet upload bandwidth into the download field of the sqm GUI and >vice versa=2E Also SQM on an internal interface will also shape internal >traffic over the same interface, and that often affects traffic to and >from the wifi/wlan radios to the lan switch=E2=80=A6 (I guess you would h= ave >preferred a shorter less vague response, but such are the constraints=E2= =80=A6) >>>>>>>> >>>>>>>>> 4=2E Is it recommend to enable diffserv on ingress? >>>>>>>> If you trust/konw/have confirmed that your upstream (ISP?) >sends you sensible and reasonable DSCP markings by all means enable >diffserv on ingress=2E But the default assumption should be that your >upstream used a dscp mapping that only makes sense for them and not for >you=2E >>>>>>>> >>>>>>>>> 5=2E Is there still the udp packet dropping problem? e=2Eg=2E ga= mes >that are using udp=2E >>>>>>>>> If yes does it make sense to apply diffserv classes manually? >How to do this? >>>>>>>> I am not sure what you mean, but if you test this and have >some findings please report here=E2=80=A6 >>>>>>>> >>>>>>>>> 6=2E is the autorate_ingress still under development? >>>>>>>>> This very interesting feature=2E especially for docsis networks= =2E >Will it be possible to set target ping time? >>>>>>>> The last tests did indicate that this feature is not ready for >primetime at least not on typically fixed bandwidth links and I assume >docsis links are fixed enough=2E >>>>>>>> >>>>>>>>> 6=2E What difference does it make to set a different rtt? >>>>>>>>> Setting lower rtt will reduce download speed i guess but will >it allow better ping times (because of lower downloadrate uh)? >>>>>>>>> What happens if rtt is set way higher? >>>>>>>> With the RTT parameter you in essence specify how much time >you give the endpoints of a flow to respond to a congestion signal (ECN >marking or packet drop) if you select this way to small you will >sacrifice bandwidth, if you set this too high you will accumulate more >latency under load=2E The good thing seems to be that this does not need >to be terribly precise, order of magnitude correctness seems to be >sufficient (at least in base2) >>>>>>>> >>>>>>>> >>>>>>>>> Thank you! >>>>>>>> I am sure the real experts will also chime in=E2=80=A6 >>>>>>>> >>>>>>>> Best Regards >>>>>>>> Sebastian >>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Cake mailing list >>>>>>>>> Cake@lists=2Ebufferbloat=2Enet >>>>>>>>> https://lists=2Ebufferbloat=2Enet/listinfo/cake >>>>>>> _______________________________________________ >>>>>>> Cake mailing list >>>>>>> Cake@lists=2Ebufferbloat=2Enet >>>>>>> https://lists=2Ebufferbloat=2Enet/listinfo/cake >>> --=20 Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E