From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 4CE7E3B2A4 for ; Wed, 6 Dec 2017 14:32:58 -0500 (EST) Received: by mail-wm0-x22a.google.com with SMTP id y82so24344287wmg.1 for ; Wed, 06 Dec 2017 11:32:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:cc:date:message-id:in-reply-to:references:reply-to :user-agent:mime-version:content-transfer-encoding; bh=l7Dzz3TKerNUdk9KDPMnnR8JCZB2v8WTH6Jg6JTVXn8=; b=eS952ykDbE+FaHKm1V1tJvB7PLDkAUGylR6+uVb9nakeobDH2am9Clf9EWT+7TlAQh vTRLmNaJpt5E9e2Cy4LZdXGuA4I/ydEd9y4LzlaOkgal7HyV30bjCu9VtWQ0k/3X+Ohp xuCgX3xADEVcuUOVWRreR2GJs6a+pvzCMy38AkmVIBeIGzviUQI9BpiZhanRk0fF3mQY OX+GUs15kOz04JquReRvciWk1fu8Fiil4qczMaso4kRpu5cNGPMcyXmz3zBfK86EAYi2 Dd2H2n/zD7FvpKo8Ta7aPRidhOT0EzC1zDADzjy3ZsIdG+n1zzTKSRch4BAMNS7tHgJt x0oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:cc:date:message-id:in-reply-to :references:reply-to:user-agent:mime-version :content-transfer-encoding; bh=l7Dzz3TKerNUdk9KDPMnnR8JCZB2v8WTH6Jg6JTVXn8=; b=SfRofUK0hTkQpB3zdNtSrw/jtnQjevqVvd9KxVhfHjmxVwSmoNz3SeXMbHUpTbbljW sI4F3PBh4ThFpMLqOwAlQNzQWcVNLTyE1q1wA08jFVsXMgbmrUgrloZ8jW7JqJExDyef bpLeCcw+vcQzheaQVIwjmsKCKIl2ATxGsh8XNIiDLjX2dqQIr5KjCWXOAzWZWq+cdD94 KmyoF8fuxZUlFFikCXMYqXnyb+QkO5PjjeYPsd9QkTdqomtT5w88mGQJXqoyE/CO1BYm wRJGgExPtU3w13zDCPqCt+1zn/sTvOpD4fhLaoZCNvAAmjqlp3rElxcC8R4eG6+Tx/b8 8PLg== X-Gm-Message-State: AJaThX6aeOH5VsJHOqa4ZbogygQENyAMMvgLmRsYmWth00h9Mr9lnP9Q iIkPeRMgdyM4QawfxX/gJUzsuZPx X-Google-Smtp-Source: AGs4zMZGL8Q8dNrNMseYqJY+rcD6Fhs8kSxEwEytMCc43agB/D0fsyplES9h9pcpUhwQenVY13+JyQ== X-Received: by 10.28.74.152 with SMTP id n24mr12518007wmi.7.1512588777331; Wed, 06 Dec 2017 11:32:57 -0800 (PST) Received: from [192.168.1.116] (cm149-53.liwest.at. [81.10.149.53]) by smtp.gmail.com with ESMTPSA id q15sm3653554wra.91.2017.12.06.11.32.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Dec 2017 11:32:56 -0800 (PST) From: xnor To: "Jonathan Morton" Cc: "Cake List" Date: Wed, 06 Dec 2017 19:32:58 +0000 Message-Id: In-Reply-To: References: <1512426648.21759.19.camel@gmail.com> <1512436395.8927.10.camel@gmail.com> <1512445837.3088.2.camel@gmail.com> <87vahkdgg0.fsf@nemesis.taht.net> Reply-To: xnor User-Agent: eM_Client/7.1.30794.0 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake vs fqcodel with 1 client, 4 servers X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Dec 2017 19:32:58 -0000 >At those speeds, you probably still have a large enough BDP per flow=20 >that TCP can respond appropriately. I'm testing at 512Kbps, where it's=20 >easy to get into single-packet BDPs. I've tried again with 96 connections to the same server, so 196 Kbps=20 each, and that reduces the gross download rate from over 17 to below 10=20 Mpbs. The inbound interface still receives 18.7 Mbps and crude latency=20 measurements with a ping running at the same time show only a slight=20 increase on average. The only problem I have with this (besides the amount of perfectly good=20 received data that needs to be discarded without ECN to keep TCP in=20 check... that's terrible protocol design) is that I do get 3x to 4x=20 spikes over the steady state latency when the download begins. That's=20 with "just" 16 additional connections starting. >A first-pass dynamic target adjustment is now pushed, and running here.=20 > It doesn't solve the problem completely, probably because it doesn't=20 >guarantee 4xMTU, only 1xMTU, but it looks like it might be beginning to=20 >help and I'm pretty sure it's the correct approach. Tuning it from=20 >here should be easy. > I don't know exactly what you're adjusting and what you're adjusting to,=20 but I see two things that would help: 1) Reduce the amount of data that needs to be dropped. Maybe the "pattern" of how packets are dropped can be improved such that=20 on *average* the sender sends at the same rate but overall less data=20 needs to be dropped at the receiver. For example, instead of dropping the just received packet because it=20 exceeded some threshold and doing this for each packet it could make=20 more sense to drop multiple packets at once, making the sender to slow=20 down significantly. The problem with this of course would be a more pronounced zig-zag=20 pattern in the bandwidth (with the implementation either limiting on the=20 peaks or average rate). (I'm not sure if this makes sense given common TCP implementations.) 2) Predict the slope of the bandwidth and slow down the sender=20 proactively, especially during slow-start. If the connection is in an exponential speed up phase then we have to=20 drop packets *before* we notice that the current (or even worse:=20 average) rate is above the configured rate. The situation gets worse if=20 multiple connections are in that phase at the same time, and ideally=20 that would have to be accounted for as well. Those are just some suggestions. I haven't studied cake's implementation=20 so maybe it already does something like that.