From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x244.google.com (mail-lj1-x244.google.com [IPv6:2a00:1450:4864:20::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 143863BA8E for ; Thu, 5 Jul 2018 19:48:21 -0400 (EDT) Received: by mail-lj1-x244.google.com with SMTP id y17-v6so3060592ljy.8 for ; Thu, 05 Jul 2018 16:48:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=l2Q+uEymeDyQ/Hmc7e0P0uQN5LypVt5JDiZUhNkLZW4=; b=vQZ09y+G0UDsqptT1ffGjT2t6vDzepd2SKcPOPl4sfqgOS2bPPatB6E2zYhVtPCkrL 983ZWUnhHSO7Uv9VAcWlTFZA7vyLuvH1lM6S2fCATuZsFc3jKc1aM4ylnTbptYL0hmSt DQi4XO2bMLqvSAHiDT2/VSzsNZqc6dBlPVxUchMppJcNMndD/UKwNu18OrxUpJmHgY4p MPQFpuUQf/sxf/yjRKCxZikTxjCC7fPJ6zuW60EzxgwymHr6pdwpv2KIGXmH6o6XvUda ordTeGCPl/BWZGPQhKVbgaSZtC3+3ZcIujW4kOCnEXuF/g+/VzyowF+P/IO83gdszs1u DpxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=l2Q+uEymeDyQ/Hmc7e0P0uQN5LypVt5JDiZUhNkLZW4=; b=RdBinQwy7yF3K8O1ngcdOLk90j0eDkluKws61eiNzvZlpovQrRvwiUD+2wK/zxCg6S Qrv36FWVAOeMUH7B+QmRI0Xw6BZtnXbA7jPrQ0lUlT9/eAxZNSIuCru+Lrave/FoRkGH 1oElUST4v8x3qym3lGK1qAmtsJus4UQd+lRbPUQShceJq/gJFGOUik6I6sxuDclr9r21 vMyA+eZuoLXkYUzW3on0hRQYIpcXCBB8nU4s4oPXEhS8yT92woTKyB9WjRT5EkSIeG28 w8Y+ee7JZKsepyiVGOgKjNfGPXf/MxD0tPA8V2yrmMS5kkuMywTHx5enOhmCtdMXfX4h VAtg== X-Gm-Message-State: APt69E0L1HJGB6zOpDgLj6uQd/6IgWFF39kvzkFYjl1bY7CSPwBrzL+e wfaSBZrRF95WHCyQYaCUs4qLUj8aniqxKUO6R3Y= X-Google-Smtp-Source: AAOMgpdJgyw/KhTyPD+AVKi127M6ywJR+a8xtpPxaIv3JVIBl1zO5ioLiqva+JYOpbUchXpc9yPkhJyCAfW74CU64qI= X-Received: by 2002:a2e:590e:: with SMTP id n14-v6mr5044092ljb.128.1530834499648; Thu, 05 Jul 2018 16:48:19 -0700 (PDT) MIME-Version: 1.0 References: <17AF79A0-0213-44E3-95B9-62795A644A47@heistp.net> <87lgatj13k.fsf@toke.dk> <87fu11ipir.fsf@toke.dk> <871scligay.fsf@toke.dk> <2AE036E5-BD3D-4176-9476-9EC824EC1D18@darbyshire-bryant.me.uk> <87r2klh1fz.fsf@toke.dk> <87lgath01v.fsf@toke.dk> <52B2B44D-4382-404C-8F6D-03F12A72B11F@heistp.net> <31667353-48F2-4FAB-AC05-163680451719@toke.dk> <48ECB6C8-5D22-4785-A6CE-696D87EC5496@toke.dk> <73DD74AD-C2E7-4A12-AE49-C06D4486660E@gmail.com> <87fu10haw7.fsf@toke.dk> <8736wxco28.fsf@toke.dk> In-Reply-To: <8736wxco28.fsf@toke.dk> From: Georgios Amanakis Date: Thu, 5 Jul 2018 19:48:07 -0400 Message-ID: To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Cc: Jonathan Morton , Cake List Content-Type: multipart/alternative; boundary="0000000000008fc0700570492ea5" Subject: Re: [Cake] cake at 60gbit X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2018 23:48:21 -0000 --0000000000008fc0700570492ea5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I am going to give it a try, with your patch applied tonight and report. Thank you! George On Thu, Jul 5, 2018, 6:31 PM Toke H=C3=B8iland-J=C3=B8rgensen wrote: > Toke H=C3=B8iland-J=C3=B8rgensen writes: > > > Jonathan Morton writes: > > > >>> On 3 Jul, 2018, at 1:23 am, Toke H=C3=B8iland-J=C3=B8rgensen > wrote: > >>> > >>> My hunch is that this has something to do with the way mlx5 uses > >>> multiple receive queues (and thus multiple CPUs). Which is probably > >>> different from veth... > >> > >> At this stage I'm pretty confident it has nothing to do with Cake, and > >> everything to do with the Mellanox hardware and driver. It does strike > >> me that Linux' default handling of multiqueue hardware doesn't map > >> very well to the qdisc interface. > > > > Well, it doesn't happen with fq_codel, so even if it is a driver bug, i= t > > is being triggered by cake specifically... > > Right, so finally got some time to investigate this further. > > I suspected that cake_dequeue() was looping forever, so I added some > debug statements to investigate this; and turns out I was right. Using > the debug patch below, in unlimited mode I get loop aborts on loop 'i' > for unlimited mode and loop 'l' if I enable the shaper at 70 gbit. It > happens pretty reliably, but only when I load up the link sufficiently > (need 4-6 TCP flows which get ~50 Gbps of total throughput). > > The weird thing is that what appears to be happening, is that cake > somehow gets into a state where sch->q.qlen is >0 while all tin backlogs > are 0. I have no clue how this happens; as far as I can tell, all > changes to tin_backlog are paired with a change to q.qlen. The only > thing outside of cake itself that modifies q.qlen is peek(), which is > not being used here. > > I'm giving up for tonight; if anyone else has any ideas, I'm all ears. > > -Toke > > Sample debug output: > > [ 5456.068281] Loop counter i hit 100k; aborting! i 100001 j 0 k 180 l 3 = m > 0 qlen 2 qbkllog 33184 tin 2 deficit 172 tot backlog 0 > > With this debug patch: > > @@ -1892,6 +1892,20 @@ static struct sk_buff *cake_dequeue(struct Qdisc > *sch) > u64 delay; > u32 len; > > + int i=3D0,j=3D0,k=3D0,l=3D0,m=3D0; > + > +#define COUNT_LOOP(v) do { \ > + if (++v > 100000) { \ > + int tot_bkl =3D 0; = \ > + struct cake_tin_data *t; \ > + int n; \ > + for(n=3D0,t =3D q->tins; n < CAKE_MAX_TINS; n++,t= ++) > \ > + tot_bkl +=3D t->tin_backlog; = \ > + net_warn_ratelimited("Loop counter " #v " hit > 100k; aborting! i %d j %d k %d l %d m %d qlen %d qbkllog %d tin %d defici= t > %d tot backlog %d", i, j, k, l, m, sch->q.qlen, sch->qstats.backlog, > q->cur_tin, b->tin_deficit, tot_bkl); \ > + return NULL; \ > + } \ > + } while(0); > + > begin: > if (!sch->q.qlen) > return NULL; > @@ -1912,6 +1926,7 @@ begin: > /* In unlimited mode, can't rely on shaper timings, just > balance > * with DRR > */ > + i=3D0; > while (b->tin_deficit < 0 || > !(b->sparse_flow_count + b->bulk_flow_count)) { > if (b->tin_deficit <=3D 0) > @@ -1923,6 +1938,7 @@ begin: > q->cur_tin =3D 0; > b =3D q->tins; > } > + COUNT_LOOP(i); > } > } else { > /* In shaped mode, choose: > @@ -1960,8 +1976,10 @@ retry: > head =3D &b->old_flows; > if (unlikely(list_empty(head))) { > head =3D &b->decaying_flows; > - if (unlikely(list_empty(head))) > + if (unlikely(list_empty(head))) { > + COUNT_LOOP(j); > goto begin; > + } > } > } > } > @@ -2008,6 +2026,7 @@ retry: > flow->set =3D CAKE_SET_SPARSE_WAIT; > } > } > + COUNT_LOOP(k); > goto retry; > } > > @@ -2050,6 +2069,7 @@ retry: > srchost->srchost_refcnt--; > dsthost->dsthost_refcnt--; > } > + COUNT_LOOP(l); > goto begin; > } > > @@ -2075,6 +2095,8 @@ retry: > kfree_skb(skb); > if (q->rate_flags & CAKE_FLAG_INGRESS) > goto retry; > + > + COUNT_LOOP(m); > } > > b->tin_ecn_mark +=3D !!flow->cvars.ecn_marked; > > > > --0000000000008fc0700570492ea5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: base64 PGRpdiBkaXI9ImF1dG8iPkkgYW0gZ29pbmcgdG8gZ2l2ZSBpdCBhIHRyeSwgd2l0aCB5b3VyIHBh dGNoIGFwcGxpZWQgdG9uaWdodCBhbmQgcmVwb3J0LjxkaXYgZGlyPSJhdXRvIj5UaGFuayB5b3Uh PC9kaXY+PGRpdiBkaXI9ImF1dG8iPjxicj48L2Rpdj48ZGl2IGRpcj0iYXV0byI+R2VvcmdlwqA8 L2Rpdj48L2Rpdj48YnI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiPk9u IFRodSwgSnVsIDUsIDIwMTgsIDY6MzEgUE0gVG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2VuICZsdDs8 YSBocmVmPSJtYWlsdG86dG9rZUB0b2tlLmRrIj50b2tlQHRva2UuZGs8L2E+Jmd0OyB3cm90ZTo8 YnI+PC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAg MCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+VG9r ZSBIw7hpbGFuZC1Kw7hyZ2Vuc2VuICZsdDs8YSBocmVmPSJtYWlsdG86dG9rZUB0b2tlLmRrIiB0 YXJnZXQ9Il9ibGFuayIgcmVsPSJub3JlZmVycmVyIj50b2tlQHRva2UuZGs8L2E+Jmd0OyB3cml0 ZXM6PGJyPg0KPGJyPg0KJmd0OyBKb25hdGhhbiBNb3J0b24gJmx0OzxhIGhyZWY9Im1haWx0bzpj aHJvbWF0aXg5OUBnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIiByZWw9Im5vcmVmZXJyZXIiPmNo cm9tYXRpeDk5QGdtYWlsLmNvbTwvYT4mZ3Q7IHdyaXRlczo8YnI+DQomZ3Q7PGJyPg0KJmd0OyZn dDsmZ3Q7IE9uIDMgSnVsLCAyMDE4LCBhdCAxOjIzIGFtLCBUb2tlIEjDuGlsYW5kLUrDuHJnZW5z ZW4gJmx0OzxhIGhyZWY9Im1haWx0bzp0b2tlQHRva2UuZGsiIHRhcmdldD0iX2JsYW5rIiByZWw9 Im5vcmVmZXJyZXIiPnRva2VAdG9rZS5kazwvYT4mZ3Q7IHdyb3RlOjxicj4NCiZndDsmZ3Q7Jmd0 OyA8YnI+DQomZ3Q7Jmd0OyZndDsgTXkgaHVuY2ggaXMgdGhhdCB0aGlzIGhhcyBzb21ldGhpbmcg dG8gZG8gd2l0aCB0aGUgd2F5IG1seDUgdXNlczxicj4NCiZndDsmZ3Q7Jmd0OyBtdWx0aXBsZSBy ZWNlaXZlIHF1ZXVlcyAoYW5kIHRodXMgbXVsdGlwbGUgQ1BVcykuIFdoaWNoIGlzIHByb2JhYmx5 PGJyPg0KJmd0OyZndDsmZ3Q7IGRpZmZlcmVudCBmcm9tIHZldGguLi48YnI+DQomZ3Q7Jmd0Ozxi cj4NCiZndDsmZ3Q7IEF0IHRoaXMgc3RhZ2UgSSYjMzk7bSBwcmV0dHkgY29uZmlkZW50IGl0IGhh cyBub3RoaW5nIHRvIGRvIHdpdGggQ2FrZSwgYW5kPGJyPg0KJmd0OyZndDsgZXZlcnl0aGluZyB0 byBkbyB3aXRoIHRoZSBNZWxsYW5veCBoYXJkd2FyZSBhbmQgZHJpdmVyLiBJdCBkb2VzIHN0cmlr ZTxicj4NCiZndDsmZ3Q7IG1lIHRoYXQgTGludXgmIzM5OyBkZWZhdWx0IGhhbmRsaW5nIG9mIG11 bHRpcXVldWUgaGFyZHdhcmUgZG9lc24mIzM5O3QgbWFwPGJyPg0KJmd0OyZndDsgdmVyeSB3ZWxs IHRvIHRoZSBxZGlzYyBpbnRlcmZhY2UuPGJyPg0KJmd0Ozxicj4NCiZndDsgV2VsbCwgaXQgZG9l c24mIzM5O3QgaGFwcGVuIHdpdGggZnFfY29kZWwsIHNvIGV2ZW4gaWYgaXQgaXMgYSBkcml2ZXIg YnVnLCBpdDxicj4NCiZndDsgaXMgYmVpbmcgdHJpZ2dlcmVkIGJ5IGNha2Ugc3BlY2lmaWNhbGx5 Li4uPGJyPg0KPGJyPg0KUmlnaHQsIHNvIGZpbmFsbHkgZ290IHNvbWUgdGltZSB0byBpbnZlc3Rp Z2F0ZSB0aGlzIGZ1cnRoZXIuPGJyPg0KPGJyPg0KSSBzdXNwZWN0ZWQgdGhhdCBjYWtlX2RlcXVl dWUoKSB3YXMgbG9vcGluZyBmb3JldmVyLCBzbyBJIGFkZGVkIHNvbWU8YnI+DQpkZWJ1ZyBzdGF0 ZW1lbnRzIHRvIGludmVzdGlnYXRlIHRoaXM7IGFuZCB0dXJucyBvdXQgSSB3YXMgcmlnaHQuIFVz aW5nPGJyPg0KdGhlIGRlYnVnIHBhdGNoIGJlbG93LCBpbiB1bmxpbWl0ZWQgbW9kZSBJIGdldCBs b29wIGFib3J0cyBvbiBsb29wICYjMzk7aSYjMzk7PGJyPg0KZm9yIHVubGltaXRlZCBtb2RlIGFu ZCBsb29wICYjMzk7bCYjMzk7IGlmIEkgZW5hYmxlIHRoZSBzaGFwZXIgYXQgNzAgZ2JpdC4gSXQ8 YnI+DQpoYXBwZW5zIHByZXR0eSByZWxpYWJseSwgYnV0IG9ubHkgd2hlbiBJIGxvYWQgdXAgdGhl IGxpbmsgc3VmZmljaWVudGx5PGJyPg0KKG5lZWQgNC02IFRDUCBmbG93cyB3aGljaCBnZXQgfjUw IEdicHMgb2YgdG90YWwgdGhyb3VnaHB1dCkuPGJyPg0KPGJyPg0KVGhlIHdlaXJkIHRoaW5nIGlz IHRoYXQgd2hhdCBhcHBlYXJzIHRvIGJlIGhhcHBlbmluZywgaXMgdGhhdCBjYWtlPGJyPg0Kc29t ZWhvdyBnZXRzIGludG8gYSBzdGF0ZSB3aGVyZSBzY2gtJmd0O3EucWxlbiBpcyAmZ3Q7MCB3aGls ZSBhbGwgdGluIGJhY2tsb2dzPGJyPg0KYXJlIDAuIEkgaGF2ZSBubyBjbHVlIGhvdyB0aGlzIGhh cHBlbnM7IGFzIGZhciBhcyBJIGNhbiB0ZWxsLCBhbGw8YnI+DQpjaGFuZ2VzIHRvIHRpbl9iYWNr bG9nIGFyZSBwYWlyZWQgd2l0aCBhIGNoYW5nZSB0byBxLnFsZW4uIFRoZSBvbmx5PGJyPg0KdGhp bmcgb3V0c2lkZSBvZiBjYWtlIGl0c2VsZiB0aGF0IG1vZGlmaWVzIHEucWxlbiBpcyBwZWVrKCks IHdoaWNoIGlzPGJyPg0Kbm90IGJlaW5nIHVzZWQgaGVyZS48YnI+DQo8YnI+DQpJJiMzOTttIGdp dmluZyB1cCBmb3IgdG9uaWdodDsgaWYgYW55b25lIGVsc2UgaGFzIGFueSBpZGVhcywgSSYjMzk7 bSBhbGwgZWFycy48YnI+DQo8YnI+DQotVG9rZTxicj4NCjxicj4NClNhbXBsZSBkZWJ1ZyBvdXRw dXQ6PGJyPg0KPGJyPg0KWyA1NDU2LjA2ODI4MV0gTG9vcCBjb3VudGVyIGkgaGl0IDEwMGs7IGFi b3J0aW5nISBpIDEwMDAwMSBqIDAgayAxODAgbCAzIG0gMCBxbGVuIDIgcWJrbGxvZyAzMzE4NCB0 aW4gMiBkZWZpY2l0IDE3MiB0b3QgYmFja2xvZyAwPGJyPg0KPGJyPg0KV2l0aCB0aGlzIGRlYnVn IHBhdGNoOjxicj4NCjxicj4NCkBAIC0xODkyLDYgKzE4OTIsMjAgQEAgc3RhdGljIHN0cnVjdCBz a19idWZmICpjYWtlX2RlcXVldWUoc3RydWN0IFFkaXNjICpzY2gpPGJyPg0KwqAgwqAgwqAgwqAg dTY0IGRlbGF5Ozxicj4NCsKgIMKgIMKgIMKgIHUzMiBsZW47PGJyPg0KPGJyPg0KK8KgIMKgIMKg IMKgaW50IGk9MCxqPTAsaz0wLGw9MCxtPTA7PGJyPg0KKzxicj4NCisjZGVmaW5lIENPVU5UX0xP T1AodikgZG8ge8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgXDxicj4NCivCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoGlmICgrK3YgJmd0OyAxMDAwMDApIHvCoCDCoCDCoCDCoCDCoCDCoCDC oFw8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpbnQgdG90X2JrbCA9 IDA7wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgXDxicj4N CivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHN0cnVjdCBjYWtlX3Rpbl9kYXRh ICp0O8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIFw8YnI+DQorwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpbnQgbjvCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBcPGJyPg0KK8KgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgZm9yKG49MCx0ID0gcS0mZ3Q7dGluczsgbiAmbHQ7IENB S0VfTUFYX1RJTlM7IG4rKyx0KyspwqAgwqAgwqAgwqAgXDxicj4NCivCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHRvdF9ia2wgKz0gdC0mZ3Q7dGluX2JhY2ts b2c7wqAgwqAgwqAgwqAgwqAgwqAgwqAgXDxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoG5ldF93YXJuX3JhdGVsaW1pdGVkKCZxdW90O0xvb3AgY291bnRlciAmcXVvdDsg I3YgJnF1b3Q7IGhpdCAxMDBrOyBhYm9ydGluZyEgaSAlZCBqICVkIGsgJWQgbCAlZCBtICVkIHFs ZW4gJWQgcWJrbGxvZyAlZCB0aW4gJWQgZGVmaWNpdCAlZCB0b3QgYmFja2xvZyAlZCZxdW90Oywg aSwgaiwgaywgbCwgbSwgc2NoLSZndDtxLnFsZW4sIHNjaC0mZ3Q7cXN0YXRzLmJhY2tsb2csIHEt Jmd0O2N1cl90aW4sIGItJmd0O3Rpbl9kZWZpY2l0LCB0b3RfYmtsKTsgXDxicj4NCivCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHJldHVybiBOVUxMO8KgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIFw8YnI+DQorwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqB9wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBcPGJyPg0KK8KgIMKgIMKgIMKg fSB3aGlsZSgwKTs8YnI+DQorPGJyPg0KwqBiZWdpbjo8YnI+DQrCoCDCoCDCoCDCoCBpZiAoIXNj aC0mZ3Q7cS5xbGVuKTxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIHJldHVybiBOVUxMOzxi cj4NCkBAIC0xOTEyLDYgKzE5MjYsNyBAQCBiZWdpbjo8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCAvKiBJbiB1bmxpbWl0ZWQgbW9kZSwgY2FuJiMzOTt0IHJlbHkgb24gc2hhcGVyIHRpbWlu Z3MsIGp1c3QgYmFsYW5jZTxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgKiB3aXRoIERS Ujxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgKi88YnI+DQorwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqBpPTA7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgd2hpbGUgKGItJmd0O3Rp bl9kZWZpY2l0ICZsdDsgMCB8fDxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIShiLSZndDtzcGFyc2VfZmxvd19jb3VudCArIGItJmd0O2J1bGtfZmxvd19jb3VudCkpIHs8 YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpZiAoYi0mZ3Q7dGluX2Rl ZmljaXQgJmx0Oz0gMCk8YnI+DQpAQCAtMTkyMyw2ICsxOTM4LDcgQEAgYmVnaW46PGJyPg0KwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgcS0mZ3Q7Y3VyX3Rp biA9IDA7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgYiA9IHEtJmd0O3RpbnM7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgfTxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoENPVU5UX0xPT1Ao aSk7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgfTxicj4NCsKgIMKgIMKgIMKgIH0gZWxz ZSB7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgLyogSW4gc2hhcGVkIG1vZGUsIGNob29z ZTo8YnI+DQpAQCAtMTk2MCw4ICsxOTc2LDEwIEBAIHJldHJ5Ojxicj4NCsKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIGhlYWQgPSAmYW1wO2ItJmd0O29sZF9mbG93czs8YnI+DQrC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpZiAodW5saWtlbHkobGlzdF9lbXB0 eShoZWFkKSkpIHs8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCBoZWFkID0gJmFtcDtiLSZndDtkZWNheWluZ19mbG93czs8YnI+DQotwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpZiAodW5saWtlbHkobGlzdF9l bXB0eShoZWFkKSkpPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgaWYgKHVubGlrZWx5KGxpc3RfZW1wdHkoaGVhZCkpKSB7PGJyPg0KK8KgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgQ09VTlRf TE9PUChqKTs8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCBnb3RvIGJlZ2luOzxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoH08YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCB9PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgfTxicj4NCsKgIMKgIMKg IMKgIH08YnI+DQpAQCAtMjAwOCw2ICsyMDI2LDcgQEAgcmV0cnk6PGJyPg0KwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgZmxvdy0mZ3Q7c2V0ID0gQ0FLRV9T RVRfU1BBUlNFX1dBSVQ7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg fTxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIH08YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqBDT1VOVF9MT09QKGspOzxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGdvdG8gcmV0 cnk7PGJyPg0KwqAgwqAgwqAgwqAgfTxicj4NCjxicj4NCkBAIC0yMDUwLDYgKzIwNjksNyBAQCBy ZXRyeTo8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCBzcmNob3N0LSZndDtzcmNob3N0X3JlZmNudC0tOzxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGRzdGhvc3QtJmd0O2RzdGhvc3RfcmVmY250LS07 PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgfTxicj4NCivCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoENPVU5UX0xPT1AobCk7PGJyPg0KwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgZ290byBiZWdpbjs8YnI+DQrCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCB9PGJyPg0KPGJyPg0KQEAgLTIwNzUsNiArMjA5NSw4IEBAIHJldHJ5Ojxicj4N CsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGtmcmVlX3NrYihza2IpOzxicj4NCsKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIGlmIChxLSZndDtyYXRlX2ZsYWdzICZhbXA7IENBS0VfRkxBR19JTkdSRVNT KTxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGdvdG8gcmV0cnk7PGJy Pg0KKzxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoENPVU5UX0xPT1AobSk7PGJyPg0KwqAg wqAgwqAgwqAgfTxicj4NCjxicj4NCsKgIMKgIMKgIMKgIGItJmd0O3Rpbl9lY25fbWFyayArPSAh IWZsb3ctJmd0O2N2YXJzLmVjbl9tYXJrZWQ7PGJyPg0KPGJyPg0KPGJyPg0KPGJyPg0KPC9ibG9j a3F1b3RlPjwvZGl2Pg0K --0000000000008fc0700570492ea5--