From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D125D3CBDB for ; Thu, 17 Aug 2017 13:38:28 -0400 (EDT) Received: by mail-oi0-x22a.google.com with SMTP id g131so73503416oic.3 for ; Thu, 17 Aug 2017 10:38:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=BTF7ZOcyhvA1vV9OjribLxpIDDOHAlCBpd3WygNIpJs=; b=SQeojE2gkETpx5JIQnmc0UpBLCF0FNayPUhCkWVdW2GVsi+p1KpT9UQrK5dxG2RjzL 4eVakf+aeBcbIaYHgBpr9R5k5TVQADLGScUIqjFOdy9aG5OM8sot+uMdpFL09kO7gNZo INbSQI0yNH3uRLkanGTuzv5bmrMUzBkUXz3vx+X60tp2eumX9DgrmwBCxZ/Xef2nUjiG mRumRxcgbR8nloyswdNwjhCAbg/z9GB5c5svBKvHx5ASSk31QNV8XwPlviur6Fzw8YSd x+gl3RcjUznXh9leifcGNS9h0tUB3X0Mfqb3pYqM/VaI2u9dODQO+XLQ+6ZGXx16SNfK e01A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=BTF7ZOcyhvA1vV9OjribLxpIDDOHAlCBpd3WygNIpJs=; b=LQyGH7HZ8cp8lG6wR31evh2Xmo+k5dESgHvleczAsOdE7TRTLRl1hNK1W4Zak2HVc4 7dz1AacJIngOW+fbkMEdyTmPOeP0mRIFxfB1m87T1/SkqZ2yTNlo4nTV6MHQtJCqIXlc QOdp6NKVVa69oEkIXp79o6nepVOivjgIFCw5L4qTGY8J576zLACCbKjK+sdfGTO5LihJ f0284AIrPQoxQXEWHhu85e+LZDiH9utWwB2bxYPzjn6u9KJjfj/wDFkAtCrH/yh8NiT2 QShr3E+TpddJRsKs89jV4VpdOld2vb1mSdcwx+BQayFRBi0ocT+G7uvgR0P7MSj8tgHj 4Ktg== X-Gm-Message-State: AHYfb5jgC+faGF4+M5WZ7I6FG/4lWVRFCVF7AwdLJ3X39LmnHiW60/8B GS4R6zO9u5GtcQ== X-Received: by 10.202.78.150 with SMTP id c144mr7140610oib.12.1502991508118; Thu, 17 Aug 2017 10:38:28 -0700 (PDT) Received: from [10.142.0.20] ([185.15.109.151]) by smtp.gmail.com with ESMTPSA id k128sm3792853oih.50.2017.08.17.10.38.26 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 17 Aug 2017 10:38:27 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Pete Heist In-Reply-To: Date: Thu, 17 Aug 2017 19:38:20 +0200 Cc: cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <7CEC639E-A3CF-492A-BE4D-CD05558FFC93@gmail.com> References: To: davidxu06@gmail.com X-Mailer: Apple Mail (2.3124) Subject: Re: [Cake] flow isolation with ipip X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Aug 2017 17:38:29 -0000 I don=E2=80=99t know if this helps, but I think this should work. :) I = used IPIP tunnels (with and without FOU encapsulation) over a = point-to-point WiFi bridge as a way of testing Cake over WiFi without = traffic being prioritized by the Linux WiFi stack or WMM, for example. = The WiFi stack =E2=80=9Csees" the outer IPIP packet, and treats it with = whatever diffserv marking is on the outer packet, rather than what=E2=80=99= s on the inner packet that=E2=80=99s encapsulated. I applied Cake to the = tunnel device, which seemed to see the packets before encapsulation, and = it worked well. I think it should also work for flow isolation. I can go through my setup scripts and get more specific if need be, to = make sure I=E2=80=99m not leading anyone astray. I think the important = part is that Cake be applied to the tunnel device and not just a regular = device that=E2=80=99s carrying IPIP traffic... > Message: 1 > Date: Thu, 17 Aug 2017 02:55:17 +0300 > From: Jonathan Morton > To: Cong Xu > Cc: Cake List > Subject: Re: [Cake] flow isolation with ipip > Message-ID: > = > Content-Type: text/plain; charset=3D"utf-8" >=20 > Cake makes use of Linux' "packet dissecting" infrastructure. If the = latter > knows about the tunnelling protocol, Cake should naturally see the IP = and > port numbers of the inner payload rather than the outer tunnel. >=20 > I don't know, however, precisely what tunnels are supported. At = minimum, > don't ever expect encrypted tunnels to behave this way! >=20 > - Jonathan Morton >=20 > On 18 Jun 2017 21:13, "Cong Xu" wrote: >=20 >> Hi, >>=20 >> I wonder if cake's flow isolation works with the ipip tunnel? I hope = to >> guarantee the networking fair-share among containers/VMs in the same = host. >> Thus, I used sfq/fq to associate with each tc class created in = advance to >> provide both shaping and scheduling. The scripts roughly look like = this >> (Assume 2 containers hosting iperf client run in the same host. One >> container sends 100 parallel streams via -P 100 to iperf server = running in >> another host, the other one send 10 parallel streams with -P 10.): >>=20 >> tc qdisc add dev $NIC root handle 1: htb default 2 >> tc class add dev $NIC parent 1: classid 1:1 htb rate ${NIC_RATE}mbit >> burst 1m cburst 1m >> tc class add dev $NIC parent 1:1 classid 1:2 htb rate ${RATE1}mbit = ceil >> ${NIC_RATE}mbit burst 1m cburst 1m >> tc class add dev $NIC parent 1:1 classid 1:3 htb rate ${RATE2}mbit = ceil >> ${NIC_RATE}mbit burst1m cburst 1m >> tc qdisc add dev $NIC parent 1:2 handle 2 sfq perturb 10 >> tc qdisc add dev $NIC parent 1:3 handle 3 sfq perturb 10 >> tc filter ad ... >>=20 >> It works well, each container running iperf gets the almost same = bandwidth >> regardless of the flows number. (Without the sfq, the container = sending 100 >> streams acchieves much higher bandwidth than the 10 streams guy.) >>=20 >> -------------- simultaneous 2 unlimited (100 conns vs 10 conns) >> ------------- >> job "big-unlimited-client" created >> job "small-unlimited-client" created >> -------------- unlimited server <-- unlimited client (100 conns) >> ------------- >> [SUM] 0.00-50.01 sec 24.9 GBytes 4.22 Gbits/sec 16874 >> sender >> [SUM] 0.00-50.01 sec 24.8 GBytes 4.21 Gbits/sec >> receiver >>=20 >> -------------- unlimited server <-- unlimited client (10 conns) >> ------------- >> [SUM] 0.00-50.00 sec 24.4 GBytes 4.19 Gbits/sec 13802 >> sender >> [SUM] 0.00-50.00 sec 24.4 GBytes 4.19 Gbits/sec >> receiver >>=20 >> However, if the ipip is enabled, sfq dose not work anymore. >>=20 >> -------------- simultaneous 2 unlimited (100 conns vs 10 conns) >> ------------- >> job "big-unlimited-client" created >> job "small-unlimited-client" created >> -------------- unlimited server <-- unlimited client (100 conns) >> ------------- >> [SUM] 0.00-50.00 sec 27.2 GBytes 4.67 Gbits/sec 391278 >> sender >> [SUM] 0.00-50.00 sec 27.1 GBytes 4.65 Gbits/sec >> receiver >>=20 >> -------------- unlimited server <-- unlimited client (10 conns) >> ------------- >> [SUM] 0.00-50.00 sec 6.85 GBytes 1.18 Gbits/sec 64153 >> sender >> [SUM] 0.00-50.00 sec 6.82 GBytes 1.17 Gbits/sec >> receiver >>=20 >> The reason behind is that the src/dst ip addresses using ipip tunnel = are >> same for all flows which are the src/dst ip of the host NICs instead = of >> veth ip of each container/VM, and there is no ports number for the = outside >> header of ipip packet. I verified this by capturing the traffic on = NIC and >> analyzing it with wireshark. The real src/dst ip of container/VM is = visible >> on the tunnel device (e.g. tunl0). Theoretically, this issue can be = solved >> if I set up tc class and sfq on tunl0 instead of host NIC. I tried = it, >> unfortunately, it did not work either. fq does not work for the same >> reason, because both sfq and fq use the same flow classifier (src/dst = ips >> and ports). So, I just wonder if cake works with ipip tunnel or not. >>=20 >> I appreciate if you can provide any help based on your expertise. = Thanks. >>=20 >> Regards, >> Cong >>=20 >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake