From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp107.iad3a.emailsrvr.com (smtp107.iad3a.emailsrvr.com [173.203.187.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id BD17B3CB35 for ; Sat, 18 May 2019 22:06:04 -0400 (EDT) Received: from smtp6.relay.iad3a.emailsrvr.com (localhost [127.0.0.1]) by smtp6.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 82DBA3AFF; Sat, 18 May 2019 22:06:04 -0400 (EDT) X-SMTPDoctor-Processed: csmtpprox beta Received: from smtp6.relay.iad3a.emailsrvr.com (localhost [127.0.0.1]) by smtp6.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 7C26C3B17; Sat, 18 May 2019 22:06:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=g001.emailsrvr.com; s=20190322-9u7zjiwi; t=1558231564; bh=KhxfxfH61+IUwNG/oqoev2iY5a/fhyuYFMBfeSQqo7I=; h=Date:Subject:From:To:From; b=HRL95tDOBtvGSxOlwX3Ea1qEc7vECcPhEC56sQBHOHv9ypA8pnEt6zHIG3faoSvp2 1KRBbZjLNn54zf+j0YYmCNgKfOI5YER29XcEGI4gDqJxq2EhmFt2ADvErTLaFNNVYU Btq4S5UvxNPi9AZEk3zZaTR2eYotY2LV2qXa910w= Received: from app62.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp6.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 503213AFF; Sat, 18 May 2019 22:06:04 -0400 (EDT) X-Sender-Id: dpreed@deepplum.com Received: from app62.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.7.12); Sat, 18 May 2019 22:06:04 -0400 Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app62.wa-webapps.iad3a (Postfix) with ESMTP id 3DAC160046; Sat, 18 May 2019 22:06:04 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Sat, 18 May 2019 22:06:04 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Sat, 18 May 2019 22:06:04 -0400 (EDT) From: "David P. Reed" To: "Jonathan Morton" Cc: "Jonathan Foulkes" , "Rich Brown" , "cerowrt-devel" , "bloat" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20190518220604000000_41658" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: <2936.1557856670@turing-police> <1557859131.759530583@apps.rackspace.com> <1557871532.754117608@apps.rackspace.com> <87lfz81x7b.fsf@toke.dk> <1557876841.69888745@apps.rackspace.com> <25460D05-4F53-4317-9722-2878B160BD7B@gmx.de> <2D23494F-7B4B-4426-AC57-8CFEA993D60B@jonathanfoulkes.com> <1558219006.29034759@apps.rackspace.com> Message-ID: <1558231564.2493747@apps.rackspace.com> X-Mailer: webmail/16.4.4-RC Subject: Re: [Cerowrt-devel] =?utf-8?q?=5BBloat=5D_=28no_subject=29?= X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2019 02:06:04 -0000 ------=_20190518220604000000_41658 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0ACorrection accepted. Between the US east and west coasts, the time of fl= ight of packets on fiber or cable is about 23 msec. (Boston-LA, driving rou= te, over fiber, at 207 Mm/sec).=0A =0ASo, if all intermediate links are equ= al in rate, at say, 10 Gb/sec, that means that there should be no more than= 10,000,000,000 * 0.023 bits actually in transit on the actual fiber, plus = a packet in each router between's outbound queue. Let's say a packet is aro= und 1500 bytes, or 12,000 bits, since that is the MTU we stupidly enforce e= ven today, and there are 10 hops (typical today between Boston and LA.)=0A = =0ASo we would expect the optimal window size sum, for all flows on any hop= of that path to be 10 * 12000 bits + 230,000,000 bits:=0A =0A230,120,000 b= its in flight between BOS and LAX. Divide that by 12,000 bits/packet, and y= ou get about 19,177 packets along that path. At most points along the path,= you would expect about 10 different flows or more to be in flight, so ther= e would be, optimally, about 1,918 1500 byte packets. Each flow would get 1= Gb/sec as its share.=0A =0AIf the connection is limited to < 1 Gb/sec at e= ither endpoint, then there's no reason for any intermediate node to buffer = that much of the flow.=0A =0AThis gives a reasonable understanding of where= AQM should be ending up in terms of the cwnd needed to sustain max through= put and optimum end-to-end latency (which would be about 23 msec + 10 hops = * 12000 / 1 Gb/sec. =3D 23.012 msec. for a packet to get from one end to t= he other).=0A =0A =0A =0AOn Saturday, May 18, 2019 6:57pm, "Jonathan Morton= " said:=0A=0A=0A=0A> > On 19 May, 2019, at 1:36 am,= David P. Reed =0A> wrote:=0A> >=0A> > Pardon, but cwn= d should NEVER be larger than the number of forwarding hops=0A> between sou= rce and destination.=0A> > Kleinrock and students recently proved that the = optimum cwnd for both=0A> throughput and minimized latency is achieved when= there is one packet or less in=0A> each outbound queue from source to dest= ination (including cross traffic - meaning=0A> other flows sharing the same= outbound queue.=0A> =0A> This argument holds only if time-of-flight *betwe= en* nodes is negligible. =0A> Trivially, a geosynchronous satellite hop add= s only two nodes but approximately=0A> half a second to the one-way path de= lay, with potentially thousands of packets=0A> existing only as radio waves= in the distance between, not in a queue.=0A> =0A> - Jonathan Morton=0A> = =0A> ------=_20190518220604000000_41658 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Correction accepted. B= etween the US east and west coasts, the time of flight of packets on fiber = or cable is about 23 msec. (Boston-LA, driving route, over fiber, at 207 Mm= /sec).

=0A

 

=0A

So, if = all intermediate links are equal in rate, at say, 10 Gb/sec, that means tha= t there should be no more than 10,000,000,000 * 0.023 bits actually in tran= sit on the actual fiber, plus a packet in each router between's outbound qu= eue. Let's say a packet is around 1500 bytes, or 12,000 bits, since that is= the MTU we stupidly enforce even today, and there are 10 hops (typical tod= ay between Boston and LA.)

=0A

 

=0A

So we would expect the optimal window size sum, for all flows = on any hop of that path to be 10 * 12000 bits + 230,000,000 bits:

=0A

 

=0A

230,120,000 bits in fli= ght between BOS and LAX. Divide that by 12,000 bits/packet, and you get abo= ut 19,177 packets along that path. At most points along the path, you would= expect about 10 different flows or more to be in flight, so there would be= , optimally, about 1,918 1500 byte packets. Each flow would get 1 Gb/sec as= its share.

=0A

 

=0A

If= the connection is limited to < 1 Gb/sec at either endpoint, then there'= s no reason for any intermediate node to buffer that much of the flow.

= =0A

 

=0A

This gives a reas= onable understanding of where AQM should be ending up in terms of the cwnd = needed to sustain max throughput and optimum end-to-end latency (which woul= d be about 23 msec + 10 hops * 12000 / 1 Gb/sec.  =3D 23.012 msec. for= a packet to get from one end to the other).

=0A

&nb= sp;

=0A

 

=0A

 

= =0A

On Saturday, May 18, 2019 6:57pm, "Jonathan Morton"= <chromatix99@gmail.com> said:

=0A
=0A

> > On 19 May, 2019, at 1:36 am,= David P. Reed <dpreed@deepplum.com>
> wrote:
> ><= br />> > Pardon, but cwnd should NEVER be larger than the number of f= orwarding hops
> between source and destination.
> > Kle= inrock and students recently proved that the optimum cwnd for both
>= ; throughput and minimized latency is achieved when there is one packet or = less in
> each outbound queue from source to destination (including= cross traffic - meaning
> other flows sharing the same outbound qu= eue.
>
> This argument holds only if time-of-flight *betwe= en* nodes is negligible.
> Trivially, a geosynchronous satellite h= op adds only two nodes but approximately
> half a second to the one= -way path delay, with potentially thousands of packets
> existing o= nly as radio waves in the distance between, not in a queue.
>
> - Jonathan Morton
>
>

=0A
 ------=_20190518220604000000_41658--