From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from uplift.swm.pp.se (ipv6.swm.pp.se [IPv6:2a00:801::f]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1C7CF3CB42 for ; Sun, 25 Nov 2018 01:44:36 -0500 (EST) Received: by uplift.swm.pp.se (Postfix, from userid 501) id D3877B8; Sun, 25 Nov 2018 07:44:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1543128273; bh=lP+7D4zvJiNrGe9YXBqBzD4gYGJ1BNN97tgOh37e6pI=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=h/8V02v2qP2QUvkQ3VpQ20MxUOYJJEflxkRCjoRS5JbJ1fizFy09IVK8u9ATRDTBj UCbQaYSo4eX/IMu/ZFVVBbSwPLG2BQTSdxhUoZyLFhwJmPX8rg+lUvLzgmE1Ol3VzY H1RvT0cWCcefSSzJ75/IrWa42ewFFoNL3792e+pM= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id CEB00B6; Sun, 25 Nov 2018 07:44:33 +0100 (CET) Date: Sun, 25 Nov 2018 07:44:33 +0100 (CET) From: Mikael Abrahamsson To: Dave Taht cc: bloat In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (DEB 67 2015-01-07) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Subject: Re: [Bloat] known buffer sizes on switches X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Nov 2018 06:44:36 -0000 On Sat, 24 Nov 2018, Dave Taht wrote: > https://people.ucsc.edu/~warner/buffer.html Nice resource, thanks. If someone wonders why things look the way they do, so it's all about on-die and off-die memory. Either you use off-die or on-die memory, often SRAM which requires 6 gates per bit. So spending half a billion gates gives you ~10MB buffer on-die. If you're doing off-die memory (DRAM or similar) then you'll get the gigabytes of memory seen in some equipment. There basically is nothing in between. As soon as you go off-die you might as well put at least 2-6 GB in there. Also, off-die memory takes IO capacity. A forwarding chip might have 4 "sides" with I/O lanes sets. If you put it in a 1RU device with no buffer, you can connect ports to all of the lanes. This gives you a very high port density low buffer size device and a very good price point. Now, if you want more buffer and more route memory (taking one "side" each) plus connecting it to a backplane (another side), you now only have a single "side" left for ports. This is why high route-count, high buffer, modular switches are so much more expensive compared low-route, low-buffer, fixed configuration ones. Above is principle, there are of course combinations and optimizations to be made so not all devices adhere exactly to the above. -- Mikael Abrahamsson email: swmike@swm.pp.se