From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 000C13B260 for ; Mon, 6 Jun 2016 17:16:11 -0400 (EDT) Received: by mail-wm0-x22f.google.com with SMTP id n184so110248446wmn.1 for ; Mon, 06 Jun 2016 14:16:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=TKbSrP+xN79Se9s9LenGui5+6ssyRL+UYzktxyB3pRc=; b=jZV7px4aEzzfwBV30GnFKB6jB7Pj1Sm/D3fYafc55EHh4dtp2Sg7uYzZ+P8InmFvsE qQTir8m6H+brOI+bBbJqXwzQSHZQDIZLTbhqqAV495PdeDVLxrfjyG04AuqaPQbNb5Xf qFxYA2VqX2+uNSbyulKXpesUhgRfyUxQrPe+wWO0+LPBBr+vTq1Z0f98GNtLa0z64XgQ sQ0z38eNwFKFZJGRTSOlA4FKsvLEgddQLIv0XhVyohBXi4KUWQK89CV9TX8utIgMyAgO 35ZyMRe6NxQA2r4wZsSCtEEFsmpeczH6jbdEOjaLMKUQxX4ed+QJWUKBDVZ4+vAv/0ft f+iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=TKbSrP+xN79Se9s9LenGui5+6ssyRL+UYzktxyB3pRc=; b=lfDJ8EQacLlR3nXQCQlNy2Gi0ZAHvawrPdKHUVBH/n1RYUpLIsYzTuGywBlc8d2B90 fvnHr4GC2/pMJ/OYb0alm/QUPFAqMVqGVlDP3w8ByT7EKhurE19Y8dmv5OqabZxdPjhH HsqHAc5lyZOYv4A28B3Pg42EWtPAT6DOx7rFNoSs4ZHj3/yTQSP7GNzHlWcQtuKJx1Ck r4HfZiP+WAgQ6pcNgHpoU26qzV5V8T9LvbudepmCkZK2coo4/k3eJmXFbpUaDCddaxgw tt+/SNdBa64Yc0qQH0vBSHcQqrSdVJR93v3xhFnbFQ2NDM1uWW7Pmv9O3WUcO8r3q4Z+ epEA== X-Gm-Message-State: ALyK8tL0d3rvFf1/3LW7H9mZauhLDLybIHeIQsTvt6c+5zbBPi7HoPPk3s5K8TaIe8/6Lbk7pFs/m/iagwrXXw== X-Received: by 10.28.20.139 with SMTP id 133mr625503wmu.19.1465247770805; Mon, 06 Jun 2016 14:16:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.37.195 with HTTP; Mon, 6 Jun 2016 14:16:10 -0700 (PDT) In-Reply-To: References: <55fdf513-9c54-bea9-1f53-fe2c5229d7ba@eggo.org> <871t4as1h9.fsf@toke.dk> <3D32F19B-5DEA-48AD-97E7-D043C4EAEC51@gmail.com> From: Ketan Kulkarni Date: Mon, 6 Jun 2016 14:16:10 -0700 Message-ID: To: Mikael Abrahamsson Cc: Jonathan Morton , "cerowrt-devel@lists.bufferbloat.net" Content-Type: multipart/alternative; boundary=001a1145b52ce30b990534a29473 Subject: Re: [Cerowrt-devel] trying to make sense of what switch vendors say wrt buffer bloat X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jun 2016 21:16:12 -0000 --001a1145b52ce30b990534a29473 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable some time back they had this whitepaper - "Why Big Data Needs Big Buffer Switches" http://www.arista.com/assets/data/pdf/Whitepapers/BigDataBigBuffers-WP.pdf the type of apps they talk about is big data, hadoop etc On Mon, Jun 6, 2016 at 11:37 AM, Mikael Abrahamsson wrote: > On Mon, 6 Jun 2016, Jonathan Morton wrote: > > At 100ms buffering, their 10Gbps switch is effectively turning any DC it= =E2=80=99s >> installed in into a transcontinental Internet path, as far as peak laten= cy >> is concerned. Just because RAM is cheap these days=E2=80=A6 >> > > Nono, nononononono. I can tell you they're spending serious money on > inserting this kind of buffering memory into these kinds of devices. Buyi= ng > these devices without deep buffers is a lot lower cost. > > These types of switch chips either have on-die memory (usually 16MB or > less), or they have very expensive (a direct cost of lowered port density= ) > off-chip buffering memory. > > Typically you do this: > > ports ---|------- > ports ---| | > ports ---| chip | > ports ---|------- > > Or you do this > > ports ---|------|---buffer > ports ---| chip |---TCAM > -------- > > or if you do a multi-linecard-device > > ports ---|------|---buffer > | chip |---TCAM > -------- > | > switch fabric > > (or any variant of them) > > So basically if you want to buffer and if you want large L2-L4 lookup > tables, you have to sacrifice ports. Sacrifice lots of ports. > > So never say these kinds of devices add buffering because RAM is cheap. > This is most definitely not why they're doing it. Buffer memory for them = is > EXTREMELY EXPENSIVE. > > -- > Mikael Abrahamsson email: swmike@swm.pp.se > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel > > --001a1145b52ce30b990534a29473 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
some time back they had this whitepaper -
"Why Bi= g Data Needs Big Buffer Switches"

the type of apps they talk about is big data, hadoop= etc


On Mon, Jun 6, 2016 at 11:37 AM, Mikael Abrahamsson <swmike@s= wm.pp.se> wrote:
On Mon, 6 Jun 2016, Jonathan Morton wrote:

At 100ms buffering, their 10Gbps switch is effectively turning any DC it=E2= =80=99s installed in into a transcontinental Internet path, as far as peak = latency is concerned.=C2=A0 Just because RAM is cheap these days=E2=80=A6

Nono, nononononono. I can tell you they're spending serious money on in= serting this kind of buffering memory into these kinds of devices. Buying t= hese devices without deep buffers is a lot lower cost.

These types of switch chips either have on-die memory (usually 16MB or less= ), or they have very expensive (a direct cost of lowered port density) off-= chip buffering memory.

Typically you do this:

ports ---|-------
ports ---|=C2=A0 =C2=A0 =C2=A0 |
ports ---| chip |
ports ---|-------

Or you do this

ports ---|------|---buffer
ports ---| chip |---TCAM
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0--------

or if you do a multi-linecard-device

ports ---|------|---buffer
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| chip |---TCAM
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0--------
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 switch fabric

(or any variant of them)

So basically if you want to buffer and if you want large L2-L4 lookup table= s, you have to sacrifice ports. Sacrifice lots of ports.

So never say these kinds of devices add buffering because RAM is cheap. Thi= s is most definitely not why they're doing it. Buffer memory for them i= s EXTREMELY EXPENSIVE.

--
Mikael Abrahamsson=C2=A0 =C2=A0 email: swmike@swm.pp.se
___________________= ____________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.= bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-d= evel


--001a1145b52ce30b990534a29473--