From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qv1-xf33.google.com (mail-qv1-xf33.google.com [IPv6:2607:f8b0:4864:20::f33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id F377C3B2A4 for ; Thu, 2 Feb 2023 10:15:29 -0500 (EST) Received: by mail-qv1-xf33.google.com with SMTP id d13so1151334qvj.8 for ; Thu, 02 Feb 2023 07:15:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=CGHyb8gRgblRjptkTCG7LzELg/pJSKbTPuBJ2V+smB0=; b=ZHxDdth4UD3ivEsu/aBWFmSi+N/WvY+bvHizNy5F5xAhf/C+q5nPU5NjB7vnSRbAV3 7/jfQbLsHniNK5ysXuMIYTGJqFTmvsD7U3mEkHKtNw2sUL+Os1aGCIc8P5ro+6s7PMz+ c9l+jiDcTMCB/54dcRzphQamXSF6sZH3yFe35g+vMLuDjQkAGe/fZeum2MVbePdeQkv+ qcbeQZSc6mDSYC9Gf/Smv47kQ6t1+qV30conbR12YWEiw8D9sRRnWtM/uucow0uJULgO c8S/3JjFkJlAadmg6D6HMzg5DVmYFD8acqGfZxNtERynA2jzQC9B/aoVi+BJ2B044I1/ jPrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=CGHyb8gRgblRjptkTCG7LzELg/pJSKbTPuBJ2V+smB0=; b=djSaqoxPp17sP1accex3Y+9BJptBQXaMiTKjPeYTKic4NZxOnVZmMPr4DSlgEWSdQm QAAIbaWtAjM9qx7qq1dC1cL9zry8PjJczLl93rVVoyxcpfAGUssm4of5l6FwnZSyANkq +SFi0EV32ket/M9+nEczn1yDvSCuQjpuQCHW7tpffcuobWYm0jA2kj4Z7/zS6oUD4Vts LsHhcLZY+DWuwAEmFlZiuMe9YSQC5Z0Orap0UY/6iH4wwXjlcKdFYjuoImgzbplBUskM f1kOZY2zy26iwvIfGzTI2wjEhjsnA1EAv72tTxiAhvkS1zsf8JknNa+oYafA0E/Lw9Rd 0ozw== X-Gm-Message-State: AO0yUKXzCjfYSp4uxTF0t8stQ/Dn5R3Eyd+b7TpyHpsRpI6Px6FHuN9B 3g0kcSbpfZ+T4IkDVstTJGUkMcJ2Kds= X-Google-Smtp-Source: AK7set+dW1vhiPutZgbFUH6UM+p3keIgzVzlns8Kxh9Rl5D/q4jtc7kJYqkv3853ZV4QAlhd5XlJ6g== X-Received: by 2002:ad4:40d2:0:b0:53a:a0b4:99e5 with SMTP id x18-20020ad440d2000000b0053aa0b499e5mr11471957qvp.5.1675350929235; Thu, 02 Feb 2023 07:15:29 -0800 (PST) Received: from nysos (pool-72-83-152-181.washdc.fios.verizon.net. [72.83.152.181]) by smtp.gmail.com with ESMTPSA id g15-20020ae9e10f000000b00728bbe45888sm3964521qkm.10.2023.02.02.07.15.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Feb 2023 07:15:28 -0800 (PST) Date: Thu, 2 Feb 2023 10:15:23 -0500 From: Paul Tagliamonte To: Kenneth Porter Cc: Bufferbloat Mailing List Message-ID: References: <2DD7056F81C7C7313FBA93EA@[10.96.7.39]> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2DD7056F81C7C7313FBA93EA@[10.96.7.39]> X-Hack-The-Planet: Yes Subject: Re: [Bloat] speedtest-cli on multihomed gateway X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2023 15:15:30 -0000 I wasn't going to reply, since I figured others would get here first with more constructive notes; but since I don't see any, here's some pointers, but alas, not anything concrete; a lot is still left as an exercise to the reader. Sorry about that. Sorry this is a bit long, i'm going to try to make this as helpful as I can. On Wed, Feb 01, 2023 at 12:20:56PM -0800, Kenneth Porter via Bloat wrote: > # ip netns add comcast-1 > # ip link set eno4 netns comcast-1 > # ip netns exec comcast-1 speedtest-cli Network Namespaces work like the other Linux namespaces (ok fine, not *all* of them, but most of them) -- when you create a new one, you're in an entirely different universe that is not connected to your existing world. This world doesn't talk to other namespaces, unless you use something like a `ip link add link-name0 type veth peer link-name1`, and move one end of the veth "wormhole" into the network namespace, leave the other out of it, and use it to bridge. This is fundementally how things like Docker work. All this to say, by moving eno4 to netns comcast-1, the host won't be able to meaningfully use it anymore. This is likely not what you want (an outage during the speedtest), so my guess is you'd want a network namespace, veth pair with one end on the host, one end in the network namespace, and a bridge to join the veth0 to comcast-1. Let's create two network veth interfaces (it's like a pipe if you've never used one directly before), on-host0 which will live on my host's network namespace, and `in-ns0` which will be moved into the network namespace once we set it up. These are bad names I'm picking to make this super explicit. prompts like `host$` are done via bash on my host, perhaps with sudo. Prompts like `ns$` are done via bash inside a network namespace. Also, perhaps with sudo. ``` host$ ip link add on-host0 type veth peer in-ns0 ``` Note `ip link`. It'll show `on-host0` and `in-ns0` in the host namespace, and also down. Let's add a network namespace. ``` host$ sudo ip netns add bloat host$ sudo ip netns exec bloat $(which bash) ns$ ip link 1: lo: mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 ns$ ``` ``` host$ ip link set in-ns0 netns bloat host$ ``` ``` ns$ ip link 1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 17: in-ns0@if18: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether fe:a8:45:16:c5:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 ``` You'll note that I currently have no interfaces, *none* of the host interfaces show -- and even `lo` is down. Let's fix that. ``` ns$ ip link set lo up ns$ ip link set in-ns0 up ns$ ip addr 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 17: in-ns0@if18: mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000 link/ether fe:a8:45:16:c5:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 ``` You'll note `in-ns0` is still LAYERDOWN, this is because the host veth end is still down. The last big thing is because this is its own network namespace, you'll need to add an IP address to the veth device depending on configuration (e.g., you could NAT from the veth on-host0 -> comcast-1, bridge it, or whatever your setup and upstream will allow), and then you can set up your routes as required. ``` ns$ ip route ns$ ``` >From here on out, I suspect you've got it, given you're juggling 4 WAN ports, I guess you can fill in the rest of the blanks here, it's just a few routes/interface configurations on the host and inside your netns. FWIW, at some point this becomes almost exactly like a Docker container (or podman or what have you) without the nicities -- Docker and other container daemons/launchers are usually capable of automating this bridge+veth+netns dance for you. If that's not an option due to the platform, doing it by hand is doable, but requires a bit of work to debug. > At this point I'm not sure what I need to do to make the network > namespace usable. Best of luck, paultag -- :wq