From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0DB4C3B25D; Sun, 1 May 2016 15:55:56 -0400 (EDT) Received: by mail-qg0-x230.google.com with SMTP id f92so62113472qgf.0; Sun, 01 May 2016 12:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=bHRmLln1Y21fjDrhF++IFWY5AYAvAmvPXIynMWcyGNg=; b=GaUaaeltZT/Dsyqwb49WMqr10WXjf+yvW6eIzrUa8ynWp0FCVy27iM2SYYNAxLWL/m QqzEkiS5d4s25jJHTThBw1toaikIS/9LOQwGTXTLrPSTDiY2xXbQx6C8j9EfyzZip2X7 CxxeU+Fc6aiqSGPD4H5X8i7BxmkYtXYmOiUt6FD65HMvLgaQMkRRI6IcOfHiwYUPZMKJ xo7us1WWZ9GUvIGOh1SutznEHD5hx1BRjxGQ9CXdMNgXz4rmKOmIf3y+R+suU/QK89Ok BgaSn8yjYOdDzEgnEpk0MdLQxdqfLAvNcOF6kxOnz55PDq6ZEIAN6FmkmAuzvDJLfBhV 4+VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=bHRmLln1Y21fjDrhF++IFWY5AYAvAmvPXIynMWcyGNg=; b=hqF4sE88DEGSGztjdySDFBF8WxIpKlqUBxTUOCNrIZq9LoMDVLrNT2L1ma/8RxfmKc lEpSiiwEJu6eYfW3gKsudlzW8Ngekj5ViqLkZS2CJvhxYMzHJpDVYNkGz2ENboVVbblf bALSMj64Vsa1Kejd3Hn+TKtMigWGIvh2f0c2u4KriuBxx8sIRa7i8PJuos+s5g0DhX/2 SlfVDPY1FlkC8DDoe5cQkpOQwOiZQLA2WOyVDqRA53ZiNV4c2ZBKMqARg/5UTlpuVA5w jY3WDY52z/hZFis4alYlXcEsGvn0w+sIOzWhxS1xnQ+a0baudHSRLxTNBlrQymqN/5/m Ztfw== X-Gm-Message-State: AOPr4FU9Q9wn3OesG7oKwWNnINLJoC4tlYlJ78dHHG/zdXkdH9h1h/HgS+g9ziM1kXBNzg== X-Received: by 10.141.7.84 with SMTP id j81mr29278173qhd.29.1462132555651; Sun, 01 May 2016 12:55:55 -0700 (PDT) Received: from [100.96.106.16] ([100.96.106.16]) by smtp.googlemail.com with ESMTPSA id z64sm8126147qkb.29.2016.05.01.12.55.54 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 01 May 2016 12:55:54 -0700 (PDT) Message-ID: <1462132553.5535.207.camel@edumazet-glaptop3.roam.corp.google.com> From: Eric Dumazet To: Jonathan Morton Cc: Dave Taht , make-wifi-fast@lists.bufferbloat.net, "codel@lists.bufferbloat.net" , ath10k Date: Sun, 01 May 2016 12:55:53 -0700 In-Reply-To: <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> References: <1462125592.5535.194.camel@edumazet-glaptop3.roam.corp.google.com> <865DA393-262D-40B6-A9D3-1B978CD5F6C6@gmail.com> <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 02 May 2016 15:49:25 -0400 Subject: Re: [Make-wifi-fast] [Codel] fq_codel_drop vs a udp flood X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 May 2016 19:55:56 -0000 On Sun, 2016-05-01 at 11:46 -0700, Eric Dumazet wrote: > Just drop half backlog packets instead of 1, (maybe add a cap of 64 > packets to avoid too big burts of kfree_skb() which might add cpu > spikes) and problem is gone. > I used following patch and it indeed solved the issue in my tests. (Not the DDOS case, but when few fat flows are really bad citizens) diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c index a5e420b3d4ab..0cb8699624bc 100644 --- a/net/sched/sch_fq_codel.c +++ b/net/sched/sch_fq_codel.c @@ -135,11 +135,11 @@ static inline void flow_queue_add(struct fq_codel_flow *flow, skb->next = NULL; } -static unsigned int fq_codel_drop(struct Qdisc *sch) +static unsigned int fq_codel_drop(struct Qdisc *sch, unsigned int max) { struct fq_codel_sched_data *q = qdisc_priv(sch); struct sk_buff *skb; - unsigned int maxbacklog = 0, idx = 0, i, len; + unsigned int maxbacklog = 0, idx = 0, i, len = 0; struct fq_codel_flow *flow; /* Queue is full! Find the fat flow and drop packet from it. @@ -153,15 +153,26 @@ static unsigned int fq_codel_drop(struct Qdisc *sch) idx = i; } } + /* As the search was painful, drop half bytes of this fat flow. + * Limit to max packets to not inflict too big latencies, + * as kfree_skb() might be quite expensive. + */ + maxbacklog >>= 1; + flow = &q->flows[idx]; - skb = dequeue_head(flow); - len = qdisc_pkt_len(skb); + for (i = 0; i < max;) { + skb = dequeue_head(flow); + len += qdisc_pkt_len(skb); + kfree_skb(skb); + i++; + if (len >= maxbacklog) + break; + } + sch->qstats.drops += i; + sch->qstats.backlog -= len; q->backlogs[idx] -= len; - sch->q.qlen--; - qdisc_qstats_drop(sch); - qdisc_qstats_backlog_dec(sch, skb); - kfree_skb(skb); - flow->dropped++; + sch->q.qlen -= i; + flow->dropped += i; return idx; } @@ -170,14 +181,14 @@ static unsigned int fq_codel_qdisc_drop(struct Qdisc *sch) unsigned int prev_backlog; prev_backlog = sch->qstats.backlog; - fq_codel_drop(sch); + fq_codel_drop(sch, 1U); return prev_backlog - sch->qstats.backlog; } static int fq_codel_enqueue(struct sk_buff *skb, struct Qdisc *sch) { struct fq_codel_sched_data *q = qdisc_priv(sch); - unsigned int idx, prev_backlog; + unsigned int idx, prev_backlog, prev_qlen; struct fq_codel_flow *flow; int uninitialized_var(ret); @@ -206,16 +217,15 @@ static int fq_codel_enqueue(struct sk_buff *skb, struct Qdisc *sch) return NET_XMIT_SUCCESS; prev_backlog = sch->qstats.backlog; - q->drop_overlimit++; - /* Return Congestion Notification only if we dropped a packet - * from this flow. - */ - if (fq_codel_drop(sch) == idx) - return NET_XMIT_CN; + prev_qlen = sch->q.qlen; + ret = fq_codel_drop(sch, 64U); + q->drop_overlimit += prev_qlen - sch->q.qlen; + + /* As we dropped packet(s), better let upper stack know this */ + qdisc_tree_reduce_backlog(sch, prev_qlen - sch->q.qlen, + prev_backlog - sch->qstats.backlog); - /* As we dropped a packet, better let upper stack know this */ - qdisc_tree_reduce_backlog(sch, 1, prev_backlog - sch->qstats.backlog); - return NET_XMIT_SUCCESS; + return ret == idx ? NET_XMIT_CN : NET_XMIT_SUCCESS; } /* This is the specific function called from codel_dequeue()