From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yh0-x22b.google.com (mail-yh0-x22b.google.com [IPv6:2607:f8b0:4002:c01::22b]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id CF87921F18C for ; Wed, 22 Apr 2015 03:17:46 -0700 (PDT) Received: by yhrr66 with SMTP id r66so12591708yhr.3 for ; Wed, 22 Apr 2015 03:17:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=4bzysw0MQOrFpnVNIGbimuI1f4vAkGBF+ZsVmQl3UN8=; b=Of8p5wlRdTIEsBU5N6Qkor10YYjZFiq/zF9oj9XE6r7ppOuxaKHalG9jE4g8A0JIuw yYV0eKzm64mvWdYN45bLEmmuQpfQvXxrwrFh9zBro3SVPNvscIktkPcTV88ZL2s/N15V Ydvv29U/dE3/Y6cRL1OG2LWpjrAI4jtNas57rvfi04KT0g8JQW8Y+2kmjDOfD0NKGnbG NESKw3FwD1rKs9Ox/iOHXCYq9xLrrMeyzvZH/Wj4TfKJFsB7mSGCQ2ohq23oAdvJqshO Z27VXIxNEPJR76j+JEcVpgV3ZiJQjPpwvk6y5v1JYBoBNToYVcehElVR/GsGkMT8WdtU qCyQ== X-Received: by 10.170.207.8 with SMTP id y8mr23638469yke.64.1429697865243; Wed, 22 Apr 2015 03:17:45 -0700 (PDT) MIME-Version: 1.0 Sender: white.phoenix@gmail.com Received: by 10.13.211.68 with HTTP; Wed, 22 Apr 2015 03:17:15 -0700 (PDT) In-Reply-To: <5E91FFA0-3DB1-4725-A6F3-AE6215269B93@gmx.de> References: <558D3A0C-75A0-4707-95DF-790F29F825AE@gmx.de> <5E91FFA0-3DB1-4725-A6F3-AE6215269B93@gmx.de> From: leetminiwheat Date: Wed, 22 Apr 2015 06:17:15 -0400 X-Google-Sender-Auth: m_WLDFCCvdqGi69dCTpQyTBPgSc Message-ID: To: Sebastian Moeller Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: cerowrt-devel Subject: Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Apr 2015 10:18:15 -0000 On Wed, Apr 22, 2015 at 5:19 AM, Sebastian Moeller wrote: > > On Apr 22, 2015, at 02:28 , leetminiwheat wrote= : > >> Correcton on P.S. section: 3 and 5t not 4 and 5t. > > This is confusing me ;) I just meant the switch vlan config, configuring port 4 to anything seems to break it. default is 0 1 2 3 5t my idea was to segregate a vlan on 3. I'll have to give it some more though= t. > Please find attached the most recent sqm-scripts and the most recent luci= -sqm (the relevant script) please copy sqm-cbi.lua to: > /usr/lib/lua/luci/model/cbi/sqm.lua and you should be as up to date as yo= u can be. Make sure to also move all the files from the attached sqm-script= s folder to the matching folders on your cerowrt router; that should hopefu= lly fix the leftover IFB issue to some degree (we currently do not clean up= when an interface goes away, only when the interface gets upped again) Thanks! much appreciated and easier than cherry picking from git, especially since I don't have a buildroot set up at the moment On Wed, Apr 22, 2015 at 5:03 AM, Sebastian Moeller wrote: > > On Apr 22, 2015, at 02:19 , leetminiwheat wrote= : > >> Sorry this is getting a bit off-topic here. >> >>> On Wed, Apr 15, 2015 at 5:05 AM, Sebastian Moeller wr= ote: >>> >>>> On Apr 15, 2015, at 03:35 , leetminiwheat wr= ote: >>> >>>> I assume tweaking ring parameters from default RX:128 and TX:32 >>>> doesn't matter anymore thenr? >>> >>> As far as I know we leave that alone, see: http://www.bufferbloa= t.net/projects/bloat/wiki/Linux_Tips: >>> =E2=80=9CSet the size of the ring buffer for the network interface >>> >>> NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3= .3, which has Byte Queue Limits instead, which does a far better job." >>> >> I noticed Dave mentioned on a mailing list that changing tx ring still >> does have some benefit, and his notes in debloat script suggest BQL >> doesn't always work as implied. > > Could you cite that note please? I can not find it, @Dave maybe y= ou could comment on the notes applicability? /usr/sbin/debloat { * Byte Queue Limits is supposed to have a rate limiter that works. It is not very effective at less than 100Mbit. I get ~32k peak there and with GSO on, at 100Mbit, I have seen latency spikes of up to 70ms. (Not recently tested, however) A per queue limit of 2-3 large packets appears to be the best compromise at 100Mbit and below. So typically I hammer down BQL to 4.5k or 3k at < 100Mbit, and turn GSO/TSO off, and as a result see ping against load latencies in the 1 to 2ms range, which is about what you would expect. I have tried 1500 bytes, which limited the top end performance to about 84Mbit. } Can't seem to find the mailing list archive where Dave mentioned tx ring still having some further benefit in addition to BQL. >> Also, I tested some custom HFSC+fq_codel qos scripts here: >> https://github.com/zcecc22/qos-nxt > > Inteeresting, does his never sqm-qos work better for you? > >> He defaults to 90% (meaning you have to adjust your b/w limits), and >> the 2-bin codel doesn't seem to work very well. Seemed like an >> interesting compromise between simple and simplest, but the results >> were pretty terrible. > > If you have a RRUL plot to share that would be helpful. Actually, I linked the wrong QoS scripts, those were his old ones which I haven't tested. These are the newer simplified ones: https://github.com/zcecc22/sqm-qos nxt.qos plot: http://screencloud.net/v/jHza nxt_v2.qos plot: http://screencloud.net/v/oe8x Note: adjusted QoS caps to my full provisioned rate since these scripts limit to 90% anyways (and I use 90% in simple.qos/simplest.qos) as you can see, lots of latency spikes. I'm not sure what these scripts are intended to accomplish, perhaps they're more optimized for lower speed connections. I haven't tested his older scripts but they looked more advanced and even changed the tx rings and such, they're located here: > >> I'd like to test CAKE more, but it seems >> 3.10.50-1 doesn't have the required kernel support. >> >> Recent changes in barrier breaker to txring seem pretty dumb, they >> default to 256 txring now I believe, ticket here was closed with >> "worksforme" https://dev.openwrt.org/ticket/13072 so I'm reluctant to >> upgrade, plus I don't fully understand the extent of which Dave's >> kernel hacks impact things. Closer inspection/comparison/diffs are >> needed if I'm to upgrade and integrate Cero's tweaks. > > I am probably off here, but I assume that with a properly sizes B= QL the actual tx ring does not matter for latency anymore and can be select= ed to keep the hardware happy ;) > >> >> Oddly enough, simplest.qos on WAN gives me better throughput/latency >> at times (likely due to less overhead), but other times simple.qos is >> doing what it should and giving more throughput and lower latency to >> higher priority traffic. I seem to get better RRUL tests with LIMIT=3D >> blank, and ILIMIT/ELIMIT set to auto which results in this: > > Hmm, LIMIT defaults to 1001 basically so we can see that it was n= ot set, but left at its default. I believe that at one time limit defaulted= to 10240 packets which was to large for a wndr3700, that=E2=80=99s why I c= hanged that to 1001, but I have a hard time believing that the difference b= etween 1001 and 1024 packets should affect RRUL noticeably=E2=80=A6 but as = always if the data supports that notion I am willing to adapt... Yeah I'm not entirely sure... I ran about 20 tests last night with different settings but didn't extensively test auto vs 1001, i do see this in the output though but likely the wireless: SQM: get_limit: CURLIMIT: SQM: cur_target: auto cur_bandwidth: 1536 SQM: get_limit: CURLIMIT: SQM: cur_target: auto cur_bandwidth: 1536 SQM: get_limit: CURLIMIT: SQM: cur_target: auto cur_bandwidth: 1536 SQM: egress shaping activated SQM: Perform DSCP based filtering on ingress. (3-tier classification) SQM: get_limit: CURLIMIT: SQM: cur_target: auto cur_bandwidth: 1192 SQM: get_limit: CURLIMIT: SQM: cur_target: auto cur_bandwidth: 1192 SQM: get_limit: CURLIMIT: SQM: cur_target: auto cur_bandwidth: 1192 >> image of RRUL 45s graph here with simple.qos, no tx changes, auto >> LIMIT on FiOS 32/25 (30Mb/22.5Mb QoS): https://screencloud.net/v/tVV0 >> - looks pretty good to me, > > So what is a bit weird is that you see the priority banding in th= e download direction (upper plot) but not in the egress direction, for IPv4= this typically looks different, were you by any chance using IPv6 for this= test? This was purely ipv4, verizon still has not rolled out ipv6 and I try to remove as much of it from Cero as possible. I did notice after the fact that the upload was a little inconsistent but not entirely sure what that can be attributed to. I speedtest at 25mbps upload, and rate limit to 22.5mbps, but verizon could be screwing with it. I did test 20mbps as well but didn't see any noticeable differences. I was mostly looking at the latency and separation between priority queues. I may run some more at lower speed again, my upload can be inconsistent depending on destination (verizon often routes me through really cheap backbone routes, as a result I need to ssh tunnel my gaming traffic through a friend's server at the datacenter he works at) >> I never top out my wireless >> since it's used only for mobile phones anyways and I run HT20 which >> seems to be more reliable/less latency. > > As compared to ht40? On 5GHz or on 2.4GHz? probably applies to both, but HT40 has more overhead. I heard some talk about cleaning up WiFi recently though, including fast-tx paths but at the moment it only works in client/ap mode with single client. recent kernel changes regarding WiFi seem based on some really bad assumptions though. >> however my guest wifi I do >> need to limit and segregate via firewall so I have it enabled. > > Silly question, why do you need to limit your guest wifi? At most= I would disallow the guest the use of the highest priority queue (sort of = keeping it as a poor man=E2=80=99s control plane), but otherwise let them h= ave their fair share of the full bandwidth after all they are my guests ;) = (or alternatively put that traffic into the BK queue so they will never rea= lly delay your traffic) prioritization doesn't always work, as we can see with bittorrent traffic. and it's not so much for "guests" but more of a hotspot since I'm in a densely populated suburban area. Also I don't want to encourage them to leach off me if they have their own faster WiFi, or can just get their own home internet. I know some may disagree with me but it IS my portion of the internet that *I* pay for. I pay probably way more than I should for my fiber service and it seems to have latency issues upstream with >200 simultaneous connections sending or receiving data, something that might not ever be avoidable using QoS on my end. I know I'm on GPON probably on a fiber switch/hub thingy (forget the name) with 32-64 other customers, about a mile from the nearest main office. >> port4 is unused and likely internally >> reserved for unknown purposes. I'm still trying to figure out how it >> maps an interface to an actual port, since I'd like to assign a single >> switch switch port to it's own subnet for my FiOS router instead of >> having to use a secondary router to clone the ge00 interface on the >> backend router to forward FiOS ports to the verizon/FiOS MOCA bridge >> router in order for alerts to display on set-top boxes such as caller >> ID. There has to be a way of doing this without needing 3 routers... >> My current thoughts are to remove the port (port3 in this case) from >> the switch and make a new switch config with just 4 and 5t and somehow >> make a new interface on that for the FiOS router, but assigning the >> same ip address as the default gateway/route from ge00 on that port >> might confuse routing. More information on their rather complicated >> and seemingly unnecessary config with a backend router is located >> here: http://www.dslreports.com/faq/verizonfios/3.0_Networking#16710 > > Again, this is far away from my area of expertise, but I would si= mple use a switch as poor man=E2=80=99s aggregation network between the ONT= =E2=80=99s ethernet-port and the main-router; and then I would connrect the= actiontec=E2=80=99s wan port to the same switch. With a bit of vlan? confi= guration it should be possible to have the actiontec only see the main-rout= er (to not confuse the ONT, but maybe that is not necessary), and use a fix= ed route from the main-router to the actiontec=E2=80=99s network with the d= evices you want to access. I guess at that level of abstraction I avoid all= the pesky issues cropping up when trying to implement that ;) Yeah that was another option I saw, of using a switch with dedicated VLANs but I don't really have the hardware to do that. putting both on a simple dumb switch sounds bad if they both request an IP upstream. I will give it some more thought to somehow create a separate virtual network inside the WNDR3800 just for the ISP router. somehow just port forwarding alone doesn't do it.