[Bloat] Multiple WAN ports & SQM?

[Bloat] Multiple WAN ports & SQM?

[Rich Brown]

Given the crummy internet service in my area (DSL, max of 15mbps/1mbps), I wonder if we could improve things by getting a second connection from our ISP and "bonding" the two links together in my OpenWrt router.

I see both Multiwan (which is self-described as old) and mwan3. 

But neither would seem to offer the kinds of latency control (SQM/fq_codel/cake) that the cool kids in networking have come to expect. 

Any recommendations from this group for such an effort? Thanks.

Rich

Re: [Bloat] Multiple WAN ports & SQM?

[Daniel Sterling]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

When I had both DSL and cable modem, I compiled Linux with this patch set to make multi gateway NAT work and it worked great

http://ja.ssi.bg/#routes

Should be able to use that plus ifb+cake on each NIC to do the right thing, aye?

As an aside, I'm kind of furious that NAT fix never got merged upstream :( it's so useful for multiple uplinks

-- Dan

> On May 3, 2020, at 10:23 AM, Rich Brown <richb.hanover@gmail.com> wrote:
> 
> Given the crummy internet service in my area (DSL, max of 15mbps/1mbps), I wonder if we could improve things by getting a second connection from our ISP and "bonding" the two links together in my OpenWrt router.
> 
> I see both Multiwan (which is self-described as old) and mwan3. 
> 
> But neither would seem to offer the kinds of latency control (SQM/fq_codel/cake) that the cool kids in networking have come to expect. 
> 
> Any recommendations from this group for such an effort? Thanks.
> 
> Rich
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

[-- Attachment #2: Type: text/html, Size: 1748 bytes --]

Re: [Bloat] Multiple WAN ports & SQM?

[Dave Taht]

not huge on bonding, simpler to just get the two uplinks and split
flows across them with an sqm instance for each and a tc hash
directing flows at one or another.

On Sun, May 3, 2020 at 7:30 AM Daniel Sterling
<sterling.daniel@gmail.com> wrote:
>
> When I had both DSL and cable modem, I compiled Linux with this patch set to make multi gateway NAT work and it worked great
>
> http://ja.ssi.bg/#routes
>
> Should be able to use that plus ifb+cake on each NIC to do the right thing, aye?
>
> As an aside, I'm kind of furious that NAT fix never got merged upstream :( it's so useful for multiple uplinks
>
> -- Dan
>
> On May 3, 2020, at 10:23 AM, Rich Brown <richb.hanover@gmail.com> wrote:
>
> Given the crummy internet service in my area (DSL, max of 15mbps/1mbps), I wonder if we could improve things by getting a second connection from our ISP and "bonding" the two links together in my OpenWrt router.
>
> I see both Multiwan (which is self-described as old) and mwan3.
>
> But neither would seem to offer the kinds of latency control (SQM/fq_codel/cake) that the cool kids in networking have come to expect.
>
> Any recommendations from this group for such an effort? Thanks.
>
> Rich
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Bloat] Multiple WAN ports & SQM?

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 2343 bytes --]

I guess the question is, what Rich needs more urgently, more aggregate rate or more single-flow performance?
Then for bonding one needs a dedicated head-end device on the internet side of things, while mwan3 on the router alone should work with any independent links for failover and load sharing, IIRC.

Best Regards
        Sebastian

On 3 May 2020 16:33:56 CEST, Dave Taht <dave.taht@gmail.com> wrote:
>not huge on bonding, simpler to just get the two uplinks and split
>flows across them with an sqm instance for each and a tc hash
>directing flows at one or another.
>
>On Sun, May 3, 2020 at 7:30 AM Daniel Sterling
><sterling.daniel@gmail.com> wrote:
>>
>> When I had both DSL and cable modem, I compiled Linux with this patch
>set to make multi gateway NAT work and it worked great
>>
>> http://ja.ssi.bg/#routes
>>
>> Should be able to use that plus ifb+cake on each NIC to do the right
>thing, aye?
>>
>> As an aside, I'm kind of furious that NAT fix never got merged
>upstream :( it's so useful for multiple uplinks
>>
>> -- Dan
>>
>> On May 3, 2020, at 10:23 AM, Rich Brown <richb.hanover@gmail.com>
>wrote:
>>
>> Given the crummy internet service in my area (DSL, max of
>15mbps/1mbps), I wonder if we could improve things by getting a second
>connection from our ISP and "bonding" the two links together in my
>OpenWrt router.
>>
>> I see both Multiwan (which is self-described as old) and mwan3.
>>
>> But neither would seem to offer the kinds of latency control
>(SQM/fq_codel/cake) that the cool kids in networking have come to
>expect.
>>
>> Any recommendations from this group for such an effort? Thanks.
>>
>> Rich
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>
>
>
>-- 
>Make Music, Not War
>
>Dave Täht
>CTO, TekLibre, LLC
>http://www.teklibre.com
>Tel: 1-831-435-0729
>_______________________________________________
>Bloat mailing list
>Bloat@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/bloat

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2525 bytes --]

Re: [Bloat] Multiple WAN ports & SQM?

[David Lang]

[-- Attachment #1: Type: text/plain, Size: 2327 bytes --]

I've been wanting to set this sort of thing up (the best DSL I can get is 8/1, 
and that's with a bondd DSL setup) but have not been able to find a good 
tutorial in setting things up.

anyone have any pointers?

David Lang

  On Sun, 3 May 2020, Dave Taht wrote:

> Date: Sun, 3 May 2020 07:33:56 -0700
> From: Dave Taht <dave.taht@gmail.com>
> To: Daniel Sterling <sterling.daniel@gmail.com>
> Cc: Rich Brown <richb.hanover@gmail.com>, bloat <bloat@lists.bufferbloat.net>
> Subject: Re: [Bloat] Multiple WAN ports & SQM?
> 
> not huge on bonding, simpler to just get the two uplinks and split
> flows across them with an sqm instance for each and a tc hash
> directing flows at one or another.
>
> On Sun, May 3, 2020 at 7:30 AM Daniel Sterling
> <sterling.daniel@gmail.com> wrote:
>>
>> When I had both DSL and cable modem, I compiled Linux with this patch set to make multi gateway NAT work and it worked great
>>
>> http://ja.ssi.bg/#routes
>>
>> Should be able to use that plus ifb+cake on each NIC to do the right thing, aye?
>>
>> As an aside, I'm kind of furious that NAT fix never got merged upstream :( it's so useful for multiple uplinks
>>
>> -- Dan
>>
>> On May 3, 2020, at 10:23 AM, Rich Brown <richb.hanover@gmail.com> wrote:
>>
>> Given the crummy internet service in my area (DSL, max of 15mbps/1mbps), I wonder if we could improve things by getting a second connection from our ISP and "bonding" the two links together in my OpenWrt router.
>>
>> I see both Multiwan (which is self-described as old) and mwan3.
>>
>> But neither would seem to offer the kinds of latency control (SQM/fq_codel/cake) that the cool kids in networking have come to expect.
>>
>> Any recommendations from this group for such an effort? Thanks.
>>
>> Rich
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>
>
>
> -- 
> Make Music, Not War
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Multiple WAN ports & SQM?

[Daniel Sterling]

[-- Attachment #1: Type: text/plain, Size: 3262 bytes --]

This is what came up with when I was running both DSL and cable modems.  I
don’t use it any more but it should still work

https://gist.githubusercontent.com/eqhmcow/9967292/raw/d091ed7430d3c161971cbf0d20c3267b77f0c5ce/multi-routes.sh

This script lets you use multiple internet connections, both
simultaneously and transparently, both from the router itself and via
boxes behind the router through NAT.


For this to actually work with NAT, you most definitely need to
recompile your linux kernel with the "Routing and
# Netfilter" patches from http://www.ssi.bg/~ja/#routes . (Using the
latest full patchset is fine.)



On Sun, May 3, 2020 at 8:26 PM David Lang <david@lang.hm> wrote:

> I've been wanting to set this sort of thing up (the best DSL I can get is
> 8/1,
> and that's with a bondd DSL setup) but have not been able to find a good
> tutorial in setting things up.
>
> anyone have any pointers?
>
> David Lang
>
>   On Sun, 3 May 2020, Dave Taht wrote:
>
> > Date: Sun, 3 May 2020 07:33:56 -0700
> > From: Dave Taht <dave.taht@gmail.com>
> > To: Daniel Sterling <sterling.daniel@gmail.com>
> > Cc: Rich Brown <richb.hanover@gmail.com>, bloat <
> bloat@lists.bufferbloat.net>
> > Subject: Re: [Bloat] Multiple WAN ports & SQM?
> >
> > not huge on bonding, simpler to just get the two uplinks and split
> > flows across them with an sqm instance for each and a tc hash
> > directing flows at one or another.
> >
> > On Sun, May 3, 2020 at 7:30 AM Daniel Sterling
> > <sterling.daniel@gmail.com> wrote:
> >>
> >> When I had both DSL and cable modem, I compiled Linux with this patch
> set to make multi gateway NAT work and it worked great
> >>
> >> http://ja.ssi.bg/#routes
> >>
> >> Should be able to use that plus ifb+cake on each NIC to do the right
> thing, aye?
> >>
> >> As an aside, I'm kind of furious that NAT fix never got merged upstream
> :( it's so useful for multiple uplinks
> >>
> >> -- Dan
> >>
> >> On May 3, 2020, at 10:23 AM, Rich Brown <richb.hanover@gmail.com>
> wrote:
> >>
> >> Given the crummy internet service in my area (DSL, max of
> 15mbps/1mbps), I wonder if we could improve things by getting a second
> connection from our ISP and "bonding" the two links together in my OpenWrt
> router.
> >>
> >> I see both Multiwan (which is self-described as old) and mwan3.
> >>
> >> But neither would seem to offer the kinds of latency control
> (SQM/fq_codel/cake) that the cool kids in networking have come to expect.
> >>
> >> Any recommendations from this group for such an effort? Thanks.
> >>
> >> Rich
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >>
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >
> >
> >
> > --
> > Make Music, Not War
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-435-0729
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat

[-- Attachment #2: Type: text/html, Size: 5538 bytes --]

Re: [Bloat] Multiple WAN ports & SQM?

[Daniel Sterling]

[-- Attachment #1: Type: text/plain, Size: 3669 bytes --]

I assume mwan3 doesn’t need this patch set, so you can also look at the
openwrt code to see how it configures mwan3

Once you have the dual routing configured you can just apply cake on each
WAN NIC directly

On Mon, May 4, 2020 at 7:28 AM Daniel Sterling <sterling.daniel@gmail.com>
wrote:

> This is what came up with when I was running both DSL and cable modems.  I
> don’t use it any more but it should still work
>
>
> https://gist.githubusercontent.com/eqhmcow/9967292/raw/d091ed7430d3c161971cbf0d20c3267b77f0c5ce/multi-routes.sh
>
> This script lets you use multiple internet connections, both simultaneously and transparently, both from the router itself and via boxes behind the router through NAT.
>
>
> For this to actually work with NAT, you most definitely need to recompile your linux kernel with the "Routing and
> # Netfilter" patches from http://www.ssi.bg/~ja/#routes . (Using the latest full patchset is fine.)
>
>
>
> On Sun, May 3, 2020 at 8:26 PM David Lang <david@lang.hm> wrote:
>
>> I've been wanting to set this sort of thing up (the best DSL I can get is
>> 8/1,
>> and that's with a bondd DSL setup) but have not been able to find a good
>> tutorial in setting things up.
>>
>> anyone have any pointers?
>>
>> David Lang
>>
>>   On Sun, 3 May 2020, Dave Taht wrote:
>>
>> > Date: Sun, 3 May 2020 07:33:56 -0700
>> > From: Dave Taht <dave.taht@gmail.com>
>> > To: Daniel Sterling <sterling.daniel@gmail.com>
>> > Cc: Rich Brown <richb.hanover@gmail.com>, bloat <
>> bloat@lists.bufferbloat.net>
>> > Subject: Re: [Bloat] Multiple WAN ports & SQM?
>> >
>> > not huge on bonding, simpler to just get the two uplinks and split
>> > flows across them with an sqm instance for each and a tc hash
>> > directing flows at one or another.
>> >
>> > On Sun, May 3, 2020 at 7:30 AM Daniel Sterling
>> > <sterling.daniel@gmail.com> wrote:
>> >>
>> >> When I had both DSL and cable modem, I compiled Linux with this patch
>> set to make multi gateway NAT work and it worked great
>> >>
>> >> http://ja.ssi.bg/#routes
>> >>
>> >> Should be able to use that plus ifb+cake on each NIC to do the right
>> thing, aye?
>> >>
>> >> As an aside, I'm kind of furious that NAT fix never got merged
>> upstream :( it's so useful for multiple uplinks
>> >>
>> >> -- Dan
>> >>
>> >> On May 3, 2020, at 10:23 AM, Rich Brown <richb.hanover@gmail.com>
>> wrote:
>> >>
>> >> Given the crummy internet service in my area (DSL, max of
>> 15mbps/1mbps), I wonder if we could improve things by getting a second
>> connection from our ISP and "bonding" the two links together in my OpenWrt
>> router.
>> >>
>> >> I see both Multiwan (which is self-described as old) and mwan3.
>> >>
>> >> But neither would seem to offer the kinds of latency control
>> (SQM/fq_codel/cake) that the cool kids in networking have come to expect.
>> >>
>> >> Any recommendations from this group for such an effort? Thanks.
>> >>
>> >> Rich
>> >> _______________________________________________
>> >> Bloat mailing list
>> >> Bloat@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/bloat
>> >>
>> >> _______________________________________________
>> >> Bloat mailing list
>> >> Bloat@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/bloat
>> >
>> >
>> >
>> > --
>> > Make Music, Not War
>> >
>> > Dave Täht
>> > CTO, TekLibre, LLC
>> > http://www.teklibre.com
>> > Tel: 1-831-435-0729
>> > _______________________________________________
>> > Bloat mailing list
>> > Bloat@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/bloat
>
>

[-- Attachment #2: Type: text/html, Size: 6257 bytes --]

Re: [Bloat] Multiple WAN ports & SQM?

[Arie]

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

Already mentioned in other replies, but you can just run SQM on the
separate links and load balance those using mwan3. These are my mwan3 rules
to balance a BVVDSL and DOCSIS connection:  https://i.imgur.com/eAd4Bl5.png

Unsticky for ports that can be safely balanced without stickiness (e.g.
steam downloads on port 80)
CB is the cable modem
sticky_even for consistent HTTPS connections based on even LAN IP
sticky_odd for consistent HTTPS connections based on odd LAN IP
default_rule balances everything else across the links with a 10m sticky
timeout



On Sun, 3 May 2020 at 16:23, Rich Brown <richb.hanover@gmail.com> wrote:

> Given the crummy internet service in my area (DSL, max of 15mbps/1mbps), I
> wonder if we could improve things by getting a second connection from our
> ISP and "bonding" the two links together in my OpenWrt router.
>
> I see both Multiwan (which is self-described as old) and mwan3.
>
> But neither would seem to offer the kinds of latency control
> (SQM/fq_codel/cake) that the cool kids in networking have come to expect.
>
> Any recommendations from this group for such an effort? Thanks.
>
> Rich
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 1919 bytes --]