From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-x235.google.com (mail-qa0-x235.google.com [IPv6:2607:f8b0:400d:c00::235]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id ECF4D21F1AA for ; Thu, 5 Dec 2013 12:21:33 -0800 (PST) Received: by mail-qa0-f53.google.com with SMTP id j5so166639qaq.12 for ; Thu, 05 Dec 2013 12:21:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RNZbpTgojClVDewIp+sl9m5eml9ATNwcLgLx7I4uZc4=; b=icn7rBIlSgfPs2OLaQgKgzhqsNN8EtpSkqhverZTfVOz7bt9i0hSDbQ1BMozvL9DdW c6C8dXynlkB2yZQdDCXiw/OtfcWXyn028JnrtcpGM6yX71jKKN6mjNUUzFYj9KU649kf MhgWO5h8jn2Ff8dTu9wCyUPcl1ukM7BkyfTkBrNm6WeXQe5w597xq2LfGNZkr52GMY/s +fIJWMBkjfOSopdx833PjkkPEaHYirw4E+rYyanFgcbsYYHYQII5O+dFailgn27f/2IX biZEw6BA7+JYJxIfOFzELmuHyn3y0EU5CnkuAWwVyaUKzLkZNS80vsVQga6sv16rg3H7 AYlg== MIME-Version: 1.0 X-Received: by 10.49.120.8 with SMTP id ky8mr84046275qeb.29.1386274892919; Thu, 05 Dec 2013 12:21:32 -0800 (PST) Received: by 10.96.62.66 with HTTP; Thu, 5 Dec 2013 12:21:32 -0800 (PST) In-Reply-To: <269B8D3A-09C7-411E-9482-27CAA898A29F@gmail.com> References: <269B8D3A-09C7-411E-9482-27CAA898A29F@gmail.com> Date: Thu, 5 Dec 2013 14:21:32 -0600 Message-ID: From: Dong Mo To: Jonathan Morton Content-Type: multipart/alternative; boundary=047d7bb04e948dbc4e04eccf4688 Cc: codel@lists.bufferbloat.net Subject: Re: [Codel] New to codel-- how to run codel on a linux box? X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Dec 2013 20:21:34 -0000 --047d7bb04e948dbc4e04eccf4688 Content-Type: text/plain; charset=ISO-8859-1 That makes a lot of sense. I missed the point the fq separate ICMP and tcp flows. My current setting is Ubuntu sender A with codel (htb and fq and fq_codel happens here) ---1G---freebsd ipfw machine inject link delay of 20ms------1G-----Ubuntu receiver B machine. What I am trying to do is measure the delay of packets of the same flow experienced in A under different queuing policies (sfq or fq_codel). Is there a simple way to do this? Thanks -Mo 2013/12/5 Jonathan Morton > > On 5 Dec, 2013, at 9:49 pm, Dong Mo wrote: > > > And sending packet from the sender, where qdisc is set, to the receiver, > the queuing delay is quite small and is approximately the same as when > fq_codel enabled. > > > > However, if I am not using sfq, and send only one flow over this link, > the delay will explode again. > > > > So is sfq here cut off the queue length somehow? How should I make a > bufferbloat fair queue on linux box using tc qdisc? > > How are you measuring the delay? Ping? > > This is relevant because ICMP (ping) occupies a different "flow" than the > TCP stream(s) you are loading the link with. SFQ and fq_codel both service > flows fairly, ensuring that each gets some traffic through continuously. > So your pings have low latency even if SFQ's queue is full. > > The difference is that fq_codel keeps the queue of each flow short, while > SFQ allows each queue to grow up to the limit. The latter behaviour is > adequate in some situations, but has some bad consequences such as slow > recovery from packet loss on each flow. And because SFQ can *only* keep > the queue within limits by dropping packets, there *will* be packet loss on > a regular basis unless the receive window of the TCP flow is exhausted > first. By contrast fq_codel is ECN-aware, so on ECN-enabled flows it can > avoid dropping packets altogether while still keeping latency low enough > for rapid recovery if packet loss occurs anyway. > > It also often happens that packets dropped due to a full queue happen to > be at the end of a TCP connection, for which detection and recovery of the > problem is much slower (and usually on very human-visible timescales) than > in the middle of a connection. In this case SFQ is no help, because it > also drops from teh tail of the queue, where the last packets of a > connection arrive. By contrast fq_codel drops from the head of the queue > when required (ie. when ECN is not available), so unless two connections > share a flow (rare but not impossible), the lost packet will always be from > the middle of a connection and will be retransmitted quickly. > > That is the sort of behaviour you should test for when comparing fq_codel > and SFQ. A simple ping test under load is satisfied by both qdiscs. > > - Jonathan Morton > > --047d7bb04e948dbc4e04eccf4688 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
That makes a lot of sense.

I missed the= point the fq separate ICMP and tcp flows.

My curr= ent setting is=A0

Ubuntu sender A with codel (htb = and fq and fq_codel happens here) ---1G---freebsd ipfw machine inject link = delay of 20ms------1G-----Ubuntu receiver B machine.

What I am trying to do is measure the delay of pa= ckets of the same flow experienced in A under different queuing policies (s= fq or fq_codel).=A0

Is there a simple way to do th= is?

Thanks
-Mo



2013/12/5 Jonathan Mo= rton <chromatix99@gmail.com>

On 5 Dec, 2013, at 9:49 pm, Dong Mo wrote:

> And sending packet from the sender, where qdisc is set, to the receive= r, the queuing delay is quite small and is approximately the same as when f= q_codel enabled.
>
> However, if I am not using sfq, and send only one flow over this link,= the delay will explode again.
>
> So is sfq here cut off the queue length somehow? How should I make a b= ufferbloat fair queue on linux box using tc qdisc?

How are you measuring the delay? =A0Ping?

This is relevant because ICMP (ping) occupies a different "flow" = than the TCP stream(s) you are loading the link with. =A0SFQ and fq_codel b= oth service flows fairly, ensuring that each gets some traffic through cont= inuously. =A0So your pings have low latency even if SFQ's queue is full= .

The difference is that fq_codel keeps the queue of each flow short, while S= FQ allows each queue to grow up to the limit. =A0The latter behaviour is ad= equate in some situations, but has some bad consequences such as slow recov= ery from packet loss on each flow. =A0And because SFQ can *only* keep the q= ueue within limits by dropping packets, there *will* be packet loss on a re= gular basis unless the receive window of the TCP flow is exhausted first. = =A0By contrast fq_codel is ECN-aware, so on ECN-enabled flows it can avoid = dropping packets altogether while still keeping latency low enough for rapi= d recovery if packet loss occurs anyway.

It also often happens that packets dropped due to a full queue happen to be= at the end of a TCP connection, for which detection and recovery of the pr= oblem is much slower (and usually on very human-visible timescales) than in= the middle of a connection. =A0In this case SFQ is no help, because it als= o drops from teh tail of the queue, where the last packets of a connection = arrive. =A0By contrast fq_codel drops from the head of the queue when requi= red (ie. when ECN is not available), so unless two connections share a flow= (rare but not impossible), the lost packet will always be from the middle = of a connection and will be retransmitted quickly.

That is the sort of behaviour you should test for when comparing fq_codel a= nd SFQ. =A0A simple ping test under load is satisfied by both qdiscs.

=A0- Jonathan Morton


--047d7bb04e948dbc4e04eccf4688--