From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0C56B3B2A4; Sat, 17 Jul 2021 19:29:58 -0400 (EDT) Received: by mail-ed1-x536.google.com with SMTP id h8so17985384eds.4; Sat, 17 Jul 2021 16:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uxx51oj8IA51TDnm/tuLu8qZq/qMVxcPTK+n52ouYl8=; b=dgcTmLdkVv0eefJ/1axo35U/opLQKeMlKME2RS0i1GwJ8TV8q3Fyc2sblPAB3z9xhl XTFY3MX9bt59OsfcuIAgLdmZkhS5ELm6grrJZIEV4Q4yfQD4Q777nwRtySOk+fyZrj+k m/s92kgjDiildfx96itl5DFUSje9/C5Q1pP/VS0S+TeM/b2mvZ+X2wgWvtNFKPOGsnMO bTOmLUNU04BotebTJdRENn/GLL7JU6m+WWWndNlE3BpJoZyGhmwnrexjjoQjr1KVMMre li6G67C2IpC76dtXU+n/DvDRX2XsQNwvDaeDX99KEtJ7QaaR+h/jU1bDd8oPybCc7HcY /zsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uxx51oj8IA51TDnm/tuLu8qZq/qMVxcPTK+n52ouYl8=; b=kn7Uh3LiUtPk9iDPRX5ZGzdlmC7soLkidmelwXMNQEjaUUo1EnPIs6AWQldTgrWurK ah7rDOMgxFd2E+XLZrmDqDt2QDTTXPV3YjjbC1YYFU1uUhIKx3akrLMZ4+rCF/T91XOJ t9wmFiIkn5sc3xOvRTflfTnd8tKyGCYtDoEF1WZOo1hQ4/b3/9i6tb5pKfljk4dqOftG Um6s6fDwatOZJXOjUYOMKbPoyMbn4AwvzcWdGMk60KZb4BfIqgIwz/Pnn+oXL6kfjFz4 cEL2xQIO62p4a+4uuR0Fp3M3wwHa6ge+qjPju4AheVq5iVq3CXtxiyJ2zUGbeZeOnCGb iaOw== X-Gm-Message-State: AOAM5330wzWRhVppqlPpLP4VtGDQ6nTB4Wj1JLcRk8+rfvguIXijutR7 UMgYTFUORFYhJSR9zO3DTG4QtG2WqoiWnnqlRKY= X-Google-Smtp-Source: ABdhPJz4Nvs8ewtqTy1XkFsUyUKf06kE1fJyuJiT83+J6p2HxAAYspIAwZuKKenVQ4HuczZk24PHi2Yqj7szoYdStGo= X-Received: by 2002:a50:9f8e:: with SMTP id c14mr24829627edf.283.1626564597843; Sat, 17 Jul 2021 16:29:57 -0700 (PDT) MIME-Version: 1.0 References: <1625188609.32718319@apps.rackspace.com> <989de0c1-e06c-cda9-ebe6-1f33df8a4c24@candelatech.com> <1625773080.94974089@apps.rackspace.com> <1625859083.09751240@apps.rackspace.com> <1626111630.69692379@apps.rackspace.com> <9c3d61c1-7013-414e-964d-9e83f596e69d@candelatech.com> <1e8bdf58-2a21-f543-a248-be58bcbddbcf@candelatech.com> In-Reply-To: <1e8bdf58-2a21-f543-a248-be58bcbddbcf@candelatech.com> From: Aaron Wood Date: Sat, 17 Jul 2021 16:29:46 -0700 Message-ID: To: Ben Greear Cc: Bob McMahon , "starlink@lists.bufferbloat.net" , Make-Wifi-fast , Leonard Kleinrock , "David P. Reed" , Cake List , "codel@lists.bufferbloat.net" , cerowrt-devel , bloat Content-Type: multipart/alternative; boundary="0000000000000f0aa105c75a1540" Subject: Re: [Codel] [Bloat] Little's Law mea culpa, but not invalidating my main point X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jul 2021 23:29:59 -0000 --0000000000000f0aa105c75a1540 Content-Type: text/plain; charset="UTF-8" On Mon, Jul 12, 2021 at 1:32 PM Ben Greear wrote: > UDP is better for getting actual packet latency, for sure. TCP is > typical-user-experience-latency though, > so it is also useful. > > I'm interested in the test and visualization side of this. If there were > a way to give engineers > a good real-time look at a complex real-world network, then they have > something to go on while trying > to tune various knobs in their network to improve it. > I've always liked the smoke-ping visualization, although a single graph is only really useful for a single pair of endpoints (or a single segment, maybe). But I can see using a repeated set of graphs (Tufte has some examples), that can represent an overview of pairwise collections of latency+loss: https://www.edwardtufte.com/bboard/images/0003Cs-8047.GIF https://www.edwardtufte.com/tufte/psysvcs_p2 These work for understanding because the tiled graphs are all identically constructed, and the reader first learns how to read a single tile, and then learns the pattern of which tiles represent which measurements. Further, they are opinionated. In the second link above, the y axis is not based on the measured data, but standardized expected values, which (I think) is key to quick readability. You never need to read the axes. Much like setting up gauges such that "nominal" is always at the same indicator position for all graphs (e.g. straight up). At a glance, you can see if things are "correct" or not. That tiling arrangement wouldn't be great for showing interrelationships (although it may give you a good historical view of correlated behavior). One thought is to overlay a network graph diagram (graph of all network links) with small "sparkline" type graphs. For a more physical-based network graph, I could see visualizing the queue depth for each egress port (max value over a time of X, or percentage of time at max depth). Taken together, the timewise correlation could be useful (which peers are having problems communicating, and which ports between them are impacted?). I think getting good data about queue depth may be the hard part, especially catching transients and the duty cycle / pulse-width of the load (and then converting that to a number). Back when I uncovered the iperf application-level pacing granularity was too high 5 years ago, I called it them "millibursts", and maybe dtaht pointed out that link utilization is always 0% or 100%, and it's just a matter of the PWM of the packet rate that makes it look like something in between. https://burntchrome.blogspot.com/2016/09/iperf3-and-microbursts.html I'll let others try to figure out how build and tune the knobs, but the > data acquisition and > visualization is something we might try to accomplish. I have a feeling > I'm not the > first person to think of this, however....probably someone already has > done such > a thing. > > Thanks, > Ben > > On 7/12/21 1:04 PM, Bob McMahon wrote: > > I believe end host's TCP stats are insufficient as seen per the "failed" > congested control mechanisms over the last decades. I think Jaffe pointed > this out in > > 1979 though he was using what's been deemed on this thread as "spherical > cow queueing theory." > > > > "Flow control in store-and-forward computer networks is appropriate for > decentralized execution. A formal description of a class of "decentralized > flow control > > algorithms" is given. The feasibility of maximizing power with such > algorithms is investigated. On the assumption that communication links > behave like M/M/1 > > servers it is shown that no "decentralized flow control algorithm" can > maximize network power. Power has been suggested in the literature as a > network > > performance objective. It is also shown that no objective based only on > the users' throughputs and average delay is decentralizable. Finally, a > restricted class > > of algorithms cannot even approximate power." > > > > https://ieeexplore.ieee.org/document/1095152 > > > > Did Jaffe make a mistake? > > > > Also, it's been observed that latency is non-parametric in it's > distributions and computing gaussians per the central limit theorem for OWD > feedback loops > > aren't effective. How does one design a control loop around things that > are non-parametric? It also begs the question, what are the feed forward > knobs that can > > actually help? > > > > Bob > > > > On Mon, Jul 12, 2021 at 12:07 PM Ben Greear > wrote: > > > > Measuring one or a few links provides a bit of data, but seems like > if someone is trying to understand > > a large and real network, then the OWD between point A and B needs > to just be input into something much > > more grand. Assuming real-time OWD data exists between 100 to 1000 > endpoint pairs, has anyone found a way > > to visualize this in a useful manner? > > > > Also, considering something better than ntp may not really scale to > 1000+ endpoints, maybe round-trip > > time is only viable way to get this type of data. In that case, > maybe clever logic could use things > > like trace-route to get some idea of how long it takes to get 'onto' > the internet proper, and so estimate > > the last-mile latency. My assumption is that the last-mile latency > is where most of the pervasive > > assymetric network latencies would exist (or just ping 8.8.8.8 which > is 20ms from everywhere due to > > $magic). > > > > Endpoints could also triangulate a bit if needed, using some anchor > points in the network > > under test. > > > > Thanks, > > Ben > > > > On 7/12/21 11:21 AM, Bob McMahon wrote: > > > iperf 2 supports OWD and gives full histograms for TCP write to > read, TCP connect times, latency of packets (with UDP), latency of "frames" > with > > > simulated video traffic (TCP and UDP), xfer times of bursts with > low duty cycle traffic, and TCP RTT (sampling based.) It also has support > for sampling (per > > > interval reports) down to 100 usecs if configured with > --enable-fastsampling, otherwise the fastest sampling is 5 ms. We've > released all this as open source. > > > > > > OWD only works if the end realtime clocks are synchronized using > a "machine level" protocol such as IEEE 1588 or PTP. Sadly, *most data > centers don't > > provide > > > sufficient level of clock accuracy and the GPS pulse per second * > to colo and vm customers. > > > > > > https://iperf2.sourceforge.io/iperf-manpage.html > > > > > > Bob > > > > > > On Mon, Jul 12, 2021 at 10:40 AM David P. Reed < > dpreed@deepplum.com dpreed@deepplum.com > > >> wrote: > > > > > > > > > On Monday, July 12, 2021 9:46am, "Livingood, Jason" < > Jason_Livingood@comcast.com > > Jason_Livingood@comcast.com>>> said: > > > > > > > I think latency/delay is becoming seen to be as important > certainly, if not a more direct proxy for end user QoE. This is all still > evolving and I > > have > > > to say is a super interesting & fun thing to work on. :-) > > > > > > If I could manage to sell one idea to the management > hierarchy of communications industry CEOs (operators, vendors, ...) it is > this one: > > > > > > "It's the end-to-end latency, stupid!" > > > > > > And I mean, by end-to-end, latency to complete a task at a > relevant layer of abstraction. > > > > > > At the link level, it's packet send to packet receive > completion. > > > > > > But at the transport level including retransmission buffers, > it's datagram (or message) origination until the acknowledgement arrives > for that > > message being > > > delivered after whatever number of retransmissions, freeing > the retransmission buffer. > > > > > > At the WWW level, it's mouse click to display update > corresponding to completion of the request. > > > > > > What should be noted is that lower level latencies don't > directly predict the magnitude of higher-level latencies. But longer lower > level latencies > > almost > > > always amplfify higher level latencies. Often non-linearly. > > > > > > Throughput is very, very weakly related to these latencies, > in contrast. > > > > > > The amplification process has to do with the presence of > queueing. Queueing is ALWAYS bad for latency, and throughput only helps if > it is in exactly the > > > right place (the so-called input queue of the bottleneck > process, which is often a link, but not always). > > > > > > Can we get that slogan into Harvard Business Review? Can we > get it taught in Managerial Accounting at HBS? (which does address > logistics/supply chain > > queueing). > > > > > > > > > > > > > > > > > > > > > > > > This electronic communication and the information and any files > transmitted with it, or attached to it, are confidential and are intended > solely for the > > use of > > > the individual or entity to whom it is addressed and may contain > information that is confidential, legally privileged, protected by privacy > laws, or > > otherwise > > > restricted from disclosure to anyone else. If you are not the > intended recipient or the person responsible for delivering the e-mail to > the intended > > recipient, > > > you are hereby notified that any use, copying, distributing, > dissemination, forwarding, printing, or copying of this e-mail is strictly > prohibited. If you > > > received this e-mail in error, please return the e-mail to the > sender, delete it from your computer, and destroy any printed copy of it. > > > > > > -- > > Ben Greear >> > > Candela Technologies Inc http://www.candelatech.com > > > > > > This electronic communication and the information and any files > transmitted with it, or attached to it, are confidential and are intended > solely for the use of > > the individual or entity to whom it is addressed and may contain > information that is confidential, legally privileged, protected by privacy > laws, or otherwise > > restricted from disclosure to anyone else. If you are not the intended > recipient or the person responsible for delivering the e-mail to the > intended recipient, > > you are hereby notified that any use, copying, distributing, > dissemination, forwarding, printing, or copying of this e-mail is strictly > prohibited. If you > > received this e-mail in error, please return the e-mail to the sender, > delete it from your computer, and destroy any printed copy of it. > > > -- > Ben Greear > Candela Technologies Inc http://www.candelatech.com > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --0000000000000f0aa105c75a1540 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Mon, Jul 12, 2021 at 1:32 PM Ben Greea= r <greearb@candelatech.com> wrote:
UDP is be= tter for getting actual packet latency, for sure.=C2=A0 TCP is typical-user= -experience-latency though,
so it is also useful.

I'm interested in the test and visualization side of this.=C2=A0 If the= re were a way to give engineers
a good real-time look at a complex real-world network, then they have somet= hing to go on while trying
to tune various knobs in their network to improve it.
=
These work for understanding because the tiled graphs are al= l identically constructed, and the reader first learns how to read a single= tile, and then learns the pattern of which tiles represent which measureme= nts.

Further, they are opinionated.=C2=A0 In the s= econd link above, the y axis is not based on the measured data, but standar= dized expected values, which (I think) is key to quick readability.=C2=A0 Y= ou never need to read the axes.=C2=A0 Much like setting up gauges such that= "nominal" is always at the same indicator position for all graph= s (e.g. straight up).=C2=A0 At a glance, you can see if things are "co= rrect" or not.

That tiling arrangement wouldn= 't be great for showing interrelationships (although it may give you a = good historical view of correlated behavior).=C2=A0 One thought is to overl= ay a network graph diagram (graph of all network links) with small "sp= arkline" type graphs.

For a more physical-bas= ed network graph, I could see visualizing the queue depth for each egress p= ort (max value over a time of X, or percentage of time at max depth).
=

Taken together, the timewise correlation could be usefu= l (which peers are having problems communicating, and which ports between t= hem are impacted?).

I think getting good data abou= t queue depth may be the hard part, especially catching transients and the = duty cycle / pulse-width of the load (and then converting that to a number)= .=C2=A0 Back when I uncovered the iperf application-level pacing granularit= y was too high 5 years ago, I called it them "millibursts", and m= aybe dtaht pointed out that link utilization is always 0% or 100%, and it&#= 39;s just a matter of the PWM of the packet rate that makes it look like so= mething in between.



I'll let others try to figure out how build and tune the knobs, but the= data acquisition and
visualization is something we might try to accomplish.=C2=A0 I have a feeli= ng I'm not the
first person to think of this, however....probably someone already has done= such
a thing.

Thanks,
Ben

On 7/12/21 1:04 PM, Bob McMahon wrote:
> I believe end host's TCP stats are insufficient as seen per the &q= uot;failed" congested control mechanisms over the last decades. I thin= k Jaffe pointed this out in
> 1979 though he was using what's been deemed on this thread as &quo= t;spherical cow queueing theory."
>
> "Flow control in store-and-forward computer networks is appropria= te for decentralized execution. A formal description of a class of "de= centralized flow control
> algorithms" is given. The feasibility of maximizing power with su= ch algorithms is investigated. On the assumption that communication links b= ehave like M/M/1
> servers it is shown that no "decentralized flow control algorithm= " can maximize network power. Power has been suggested in the literatu= re as a network
> performance objective. It is also shown that no objective based only o= n the users' throughputs and average delay is decentralizable. Finally,= a restricted class
> of algorithms cannot even approximate power."
>
> https://ieeexplore.ieee.org/document/1095152 >
> Did Jaffe make a mistake?
>
> Also, it's been observed that latency=C2=A0is non-parametric in it= 's distributions and computing gaussians=C2=A0per the central limit the= orem for OWD feedback loops
> aren't effective. How does one design a control loop around things= that are non-parametric? It also begs the question, what are the feed forw= ard knobs that can
> actually help?
>
> Bob
>
> On Mon, Jul 12, 2021 at 12:07 PM Ben Greear <greearb@candelatech.com <mail= to:greearb@can= delatech.com>> wrote:
>
>=C2=A0 =C2=A0 =C2=A0Measuring one or a few links provides a bit of data= , but seems like if someone is trying to understand
>=C2=A0 =C2=A0 =C2=A0a large and real network, then the OWD between poin= t A and B needs to just be input into something much
>=C2=A0 =C2=A0 =C2=A0more grand.=C2=A0 Assuming real-time OWD data exist= s between 100 to 1000 endpoint pairs, has anyone found a way
>=C2=A0 =C2=A0 =C2=A0to visualize this in a useful manner?
>
>=C2=A0 =C2=A0 =C2=A0Also, considering something better than ntp may not= really scale to 1000+ endpoints, maybe round-trip
>=C2=A0 =C2=A0 =C2=A0time is only viable way to get this type of data.= =C2=A0 In that case, maybe clever logic could use things
>=C2=A0 =C2=A0 =C2=A0like trace-route to get some idea of how long it ta= kes to get 'onto' the internet proper, and so estimate
>=C2=A0 =C2=A0 =C2=A0the last-mile latency.=C2=A0 My assumption is that = the last-mile latency is where most of the pervasive
>=C2=A0 =C2=A0 =C2=A0assymetric network latencies would exist (or just p= ing 8.8.8.8 which is 20ms from everywhere due to
>=C2=A0 =C2=A0 =C2=A0$magic).
>
>=C2=A0 =C2=A0 =C2=A0Endpoints could also triangulate a bit if needed, u= sing some anchor points in the network
>=C2=A0 =C2=A0 =C2=A0under test.
>
>=C2=A0 =C2=A0 =C2=A0Thanks,
>=C2=A0 =C2=A0 =C2=A0Ben
>
>=C2=A0 =C2=A0 =C2=A0On 7/12/21 11:21 AM, Bob McMahon wrote:
>=C2=A0 =C2=A0 =C2=A0 > iperf 2 supports OWD and gives full histogram= s for TCP write to read, TCP connect times, latency of packets (with UDP), = latency of "frames" with
>=C2=A0 =C2=A0 =C2=A0 > simulated=C2=A0video=C2=A0traffic (TCP and UD= P), xfer times of bursts with low duty cycle traffic, and TCP RTT (sampling= based.) It also has support for sampling (per
>=C2=A0 =C2=A0 =C2=A0 > interval reports) down to 100 usecs if config= ured with --enable-fastsampling, otherwise the fastest sampling is 5 ms. We= 've released all this as open source.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > OWD only works if the end realtime clocks are= synchronized using a "machine level" protocol such as IEEE 1588 = or PTP. Sadly, *most data centers don't
>=C2=A0 =C2=A0 =C2=A0provide
>=C2=A0 =C2=A0 =C2=A0 > sufficient level of clock accuracy and the GP= S pulse per second * to colo and vm customers.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > https://iperf2.sourcef= orge.io/iperf-manpage.html
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > Bob
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > On Mon, Jul 12, 2021 at 10:40 AM David P. Ree= d <dpreed@deepp= lum.com <mailto:dpreed@deepplum.com> <mailto:dpreed@deepplum.com
>=C2=A0 =C2=A0 =C2=A0<mailto:dpreed@deepplum.com>>> wrote:
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0On Monday, July 12, 2021 9= :46am, "Livingood, Jason" <Jason_Livingood@comcast.com <mailto:Jason_Living= ood@comcast.com>
>=C2=A0 =C2=A0 =C2=A0<mailto:Jason_Livingood@comcast.com <mailto:Jason_Livingood@c= omcast.com>>> said:
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0 > I think latency/dela= y is becoming seen to be as important certainly, if not a more direct proxy= for end user QoE. This is all still evolving and I
>=C2=A0 =C2=A0 =C2=A0have
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0to say is a super interest= ing & fun thing to work on. :-)
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0If I could manage to sell = one idea to the management hierarchy of communications industry CEOs (opera= tors, vendors, ...) it is this one:
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0"It's the end-to-= end latency, stupid!"
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0And I mean, by end-to-end,= latency to complete a task at a relevant layer of abstraction.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0At the link level, it'= s packet send to packet receive completion.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0But at the transport level= including retransmission buffers, it's datagram (or message) originati= on until the acknowledgement arrives for that
>=C2=A0 =C2=A0 =C2=A0message being
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0delivered after whatever n= umber of retransmissions, freeing the retransmission buffer.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0At the WWW level, it's= mouse click to display update corresponding to completion of the request.<= br> >=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0What should be noted is th= at lower level latencies don't directly predict the magnitude of higher= -level latencies. But longer lower level latencies
>=C2=A0 =C2=A0 =C2=A0almost
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0always amplfify higher lev= el latencies. Often non-linearly.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0Throughput is very, very w= eakly related to these latencies, in contrast.
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0The amplification process = has to do with the presence of queueing. Queueing is ALWAYS bad for latency= , and throughput only helps if it is in exactly the
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0right place (the so-called= input queue of the bottleneck process, which is often a link, but not alwa= ys).
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >=C2=A0 =C2=A0 =C2=A0Can we get that slogan int= o Harvard Business Review? Can we get it taught in Managerial Accounting at= HBS? (which does address logistics/supply chain
>=C2=A0 =C2=A0 =C2=A0queueing).
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 >
>=C2=A0 =C2=A0 =C2=A0 > This electronic communication and the informa= tion and any files transmitted with it, or attached to it, are confidential= and are intended solely for the
>=C2=A0 =C2=A0 =C2=A0use of
>=C2=A0 =C2=A0 =C2=A0 > the individual or entity to whom it is addres= sed and may contain information that is confidential, legally privileged, p= rotected by privacy laws, or
>=C2=A0 =C2=A0 =C2=A0otherwise
>=C2=A0 =C2=A0 =C2=A0 > restricted from disclosure to anyone else. If= you are not the intended recipient or the person responsible for deliverin= g the e-mail to the intended
>=C2=A0 =C2=A0 =C2=A0recipient,
>=C2=A0 =C2=A0 =C2=A0 > you are hereby notified that any use, copying= , distributing, dissemination, forwarding, printing, or copying of this e-m= ail is strictly prohibited. If you
>=C2=A0 =C2=A0 =C2=A0 > received this e-mail in error, please return = the e-mail to the sender, delete it from your computer, and destroy any pri= nted copy of it.
>
>
>=C2=A0 =C2=A0 =C2=A0--
>=C2=A0 =C2=A0 =C2=A0Ben Greear <greearb@candelatech.com <mailto:greearb@candelatech.com>>
>=C2=A0 =C2=A0 =C2=A0Candela Technologies Inc http://www.candelatech.co= m
>
>
> This electronic communication and the information and any files transm= itted with it, or attached to it, are confidential and are intended solely = for the use of
> the individual or entity to whom it is addressed and may contain infor= mation that is confidential, legally privileged, protected by privacy laws,= or otherwise
> restricted from disclosure to anyone else. If you are not the intended= recipient or the person responsible for delivering the e-mail to the inten= ded recipient,
> you are hereby notified that any use, copying, distributing, dissemina= tion, forwarding, printing, or copying of this e-mail is strictly prohibite= d. If you
> received this e-mail in error, please return the e-mail to the sender,= delete it from your computer, and destroy any printed copy of it.


--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc=C2=A0 http://www.candelatech.com

_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
--0000000000000f0aa105c75a1540--