From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot0-x242.google.com (mail-ot0-x242.google.com [IPv6:2607:f8b0:4003:c0f::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A28253B29E for ; Wed, 25 Jan 2017 17:03:17 -0500 (EST) Received: by mail-ot0-x242.google.com with SMTP id 65so25208274otq.2 for ; Wed, 25 Jan 2017 14:03:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=bV4cisCrY78TkpwCjOteJoV+jEjq9V5/acHBHpuMIUY=; b=UeSQt/iKAIPVmQ+bMy8Kk2wl94Uy7qTM6/KJ5VzQkhpcN36eufDuNRkhxPTl2VAsuC 2IZ1PFgC/B9JD9+74gDfKbGzfnrBdy+MXY68f9J/rNIgtdRZxcAfE7O62rhCGe1a5He1 Fh6uvnp5oxmNIfK94pxm4UwjL2byc1LNr5ynwyaDCBYlliK4m3UkSf6wMAsdGdUMhLBA w0bBTrQgPKf7CV5o1tGkxFcOa0KGSmejptuKJ9wrtQHvWQPWx7cPxBnJVbrgpGoNAOO3 3g1oYlcmZr3+rZB0iWFOdQjlvntDLevXeb9CUdf5uCXk27MRq9//uagY2r4iRcnnFru2 fkvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=bV4cisCrY78TkpwCjOteJoV+jEjq9V5/acHBHpuMIUY=; b=pGeSP1+q+KtYqhg+wmqEgQU7WG40aPNiuO0AwjVLG7fbQt0JINRVr8FOCN3g/maQES Drh2nSZmkmo4TDPp0r+uaiMarUBzkJnClmhjM2JNeb+JKRYeCdjcMK/s8JCI9aHG3S0T Y4CpPmb1JVODB73sVL+HccCNL3XlBDSsR24jLdP22C1ItfbvODGzsp49SrZ3HCHgbzfu 2FI3YUHLHLedhPnr1V3AzGdocTQdcLMYrL18LDT1LRn0jd4nEJRn2tx3h1EAUJLfkIbu GfSH3Xk2giaDsTmYB4i704EmHzCGNr0qFCcyevPT6pg45AdRuRqb7c+JgFT0PcsvAluS jP6w== X-Gm-Message-State: AIkVDXJGvK3wK360P5gxn183bNDRWIqLMm3jmXfsB0xvRjcudmoC0yRu2O3jELWyrRpm32IoWUd5Wr6hH74KAg== X-Received: by 10.157.18.246 with SMTP id g109mr22482458otg.10.1485381796863; Wed, 25 Jan 2017 14:03:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.1.21 with HTTP; Wed, 25 Jan 2017 14:03:16 -0800 (PST) In-Reply-To: <1485380927.5145.61.camel@edumazet-glaptop3.roam.corp.google.com> References: <58D7E8D3-5661-4126-80D9-2F2D4E478B10@gmail.com> <908CA0EF-3D84-4EB4-ABD8-3042668E842E@gmail.com> <8B89C257-99D4-4A9A-96ED-A103A2089A46@gmail.com> <1485380927.5145.61.camel@edumazet-glaptop3.roam.corp.google.com> From: Hans-Kristian Bakke Date: Wed, 25 Jan 2017 23:03:16 +0100 Message-ID: To: bloat@lists.bufferbloat.net Content-Type: multipart/alternative; boundary=94eb2c05539c5ba5560546f26697 Subject: Re: [Bloat] Initial tests with BBR in kernel 4.9 X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2017 22:03:17 -0000 --94eb2c05539c5ba5560546f26697 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =E2=80=8BPerhaps the mail didn't arrive properly, but the fq performance is= okay now. I don't know why it was completely out for a couple of tests. It was most likely my mistake or some very bad timing for testing. I see that on my physical hosts tsc is also the default with hpet in the list of available clocksources, just as in the VM (where kvm-clock is a paravirtualized version of the host tsc) so the same behaviour is probably to be expected in the VM as on the physical hosts. As far as I know I have not seen that it is a requirement to actually run: echo hpet > /sys/devices/system/clocksource/clocksource0/current_clocksourc= e ... before using fq as most (physical) linux hosts use tsc as the default clock source today unless the kernel detects unreliabilities. =E2=80=8B On 25 January 2017 at 22:48, Eric Dumazet wrote: > I do not know any particular issues with FQ in VM > > If you have a recent tc binary (iproute2 package) you can get some > infos, as mentioned in this commit changelog : > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/ > linux.git/commit/?id=3Dfefa569a9d4bc4b7758c0fddd75bb0382c95da77 > > > $ tc -s qd sh dev eth0 | grep latency > 0 gc, 0 highprio, 32490767 throttled, 2382 ns latency > > > On Wed, 2017-01-25 at 22:31 +0100, Hans-Kristian Bakke wrote: > > kvm-clock is a paravirtualized clock that seems to use the CPUs TSC > > capabilities if they exist. But it may not be perfect: > > > > > > https://access.redhat.com/documentation/en-US/Red_Hat_ > Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_ > Installation_Guide/chap-Virtualization_Host_Configuration_and_Guest_ > Installation_Guide-KVM_guest_timing_management.html > > > > > > On 25 January 2017 at 22:29, Hans-Kristian Bakke > > wrote: > > Actually I think that is because it may be using the newer > > TSC: > > dmesg | grep clocksource > > [ 0.000000] clocksource: kvm-clock: mask: > > 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: > > 881590591483 ns > > [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff > > max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns > > [ 0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: > > 0xffffffff, max_idle_ns: 19112604467 ns > > [ 0.092665] clocksource: jiffies: mask: 0xffffffff > > max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns > > [ 0.366429] clocksource: Switched to clocksource kvm-clock > > [ 0.378974] clocksource: acpi_pm: mask: 0xffffff > > max_cycles: 0xffffff, max_idle_ns: 2085701024 ns > > [ 1.666474] tsc: Refined TSC clocksource calibration: > > 3200.013 MHz > > [ 1.666479] clocksource: tsc: mask: 0xffffffffffffffff > > max_cycles: 0x2e20562a1bb, max_idle_ns: 440795285529 ns > > > > > > > > On 25 January 2017 at 22:26, Jonathan Morton > > wrote: > > > > > On 25 Jan, 2017, at 23:20, Hans-Kristian Bakke > > wrote: > > > > > > =E2=80=8B[ 0.000000] ACPI: HPET 0x00000000BFFE274F= 000038 > > (v01 BOCHS BXPCHPET 00000001 BXPC 00000001) > > > [ 0.000000] ACPI: HPET id: 0x8086a201 base: > > 0xfed00000 > > > [ 0.000000] clocksource: hpet: mask: 0xffffffff > > max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns > > > [ 0.000000] hpet clockevent registered > > > [ 0.362335] hpet0: at MMIO 0xfed00000, IRQs 2, 8, > > 0 > > > [ 0.362339] hpet0: 3 comparators, 64-bit > > 100.000000 MHz counter > > > [ 0.661731] rtc_cmos 00:00: alarms up to one day, > > y3k, 114 bytes nvram, hpet irqs > > > > Conspicuously absent here is a line saying > > =E2=80=9Cclocksource: Switched to clocksource hpet=E2= =80=9D. That may > > be worth examining in more detail. > > > > - Jonathan Morton > > > > > > > > > > > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > > --94eb2c05539c5ba5560546f26697 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
=E2=80=8BPerhaps the mail didn't arrive properly, but the f= q performance is okay now. I don't know why it was completely out for a= couple of tests. It was most likely my mistake or some very bad timing for= testing.

I see that on my physical hosts tsc is also the default with= hpet in the list of available clocksources, just as in the VM (where kvm-c= lock is a paravirtualized version of the host tsc) so the same behaviour is= probably to be expected in the VM as on the physical hosts. As far as I kn= ow I have not seen that it is a requirement to actually run:

echo hpet > = /sys/devices/system/clocksource/clocksource0/current_clocksource
=

<= /font>
... before using fq as most (physical) linux hosts use tsc as the default = clock source today unless the kernel detects unreliabilities.=C2=A0<= /div>
= =E2=80=8B

On 25 January 2017 at 22:48, Eric Dumazet <eric.dumazet@gmail.com= > wrote:
I do not know any = particular issues with FQ in VM

If you have a recent tc binary (iproute2 package) you can get some
infos, as mentioned in this commit changelog :

https://git.kernel.org/cgit/linux/kernel/git/torvalds= /linux.git/commit/?id=3Dfefa569a9d4bc4b7758c0fddd75bb0382c95= da77


$ tc -s qd sh dev eth0 | grep latency
0 gc, 0 highprio, 32490767 throttled, 2382 ns latency


On Wed, 2017-01-25 at 22:31 +0100, Hans-Kristian Bakke wrote:
> kvm-clock is a paravirtualized clock that seems to use the CPUs TSC > capabilities if they exist. But it may not be perfect:
>
>
> http= s://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linu= x/6/html/Virtualization_Host_Configuration_and_Guest_Install= ation_Guide/chap-Virtualization_Host_Configuration_and_Guest_Installation_Guide-KVM_guest_timing_management.html
>
>
> On 25 January 2017 at 22:29, Hans-Kristian Bakke <hkbakke@gmail.com>
> wrote:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Actually I think that is because it m= ay be using the newer
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0TSC:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dmesg | grep clocksource
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 0.000000] clocksource:= kvm-clock: mask:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffffffffffff max_cycles: 0x1cd4= 2e4dffb, max_idle_ns:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0881590591483 ns
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 0.000000] clocksource:= refined-jiffies: mask: 0xffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0max_cycles: 0xffffffff, max_idle_ns: = 7645519600211568 ns
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 0.000000] clocksource:= hpet: mask: 0xffffffff max_cycles:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff, max_idle_ns: 19112604467 = ns
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 0.092665] clocksource:= jiffies: mask: 0xffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0max_cycles: 0xffffffff, max_idle_ns: = 7645041785100000 ns
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 0.366429] clocksource:= Switched to clocksource kvm-clock
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 0.378974] clocksource:= acpi_pm: mask: 0xffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0max_cycles: 0xffffff, max_idle_ns: 20= 85701024 ns
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 1.666474] tsc: Refined= TSC clocksource calibration:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03200.013 MHz
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0[=C2=A0 =C2=A0 1.666479] clocksource:= tsc: mask: 0xffffffffffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0max_cycles: 0x2e20562a1bb, max_idle_n= s: 440795285529 ns
>
>
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On 25 January 2017 at 22:26, Jonathan= Morton
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<chromatix99@gmail.com> wrote:
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> On 2= 5 Jan, 2017, at 23:20, Hans-Kristian Bakke
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<hkbakke@gmail.com> wrote:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> =E2= =80=8B[=C2=A0 =C2=A0 0.000000] ACPI: HPET 0x00000000BFFE274F 000038
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(v01 BOCH= S=C2=A0 BXPCHPET 00000001 BXPC 00000001)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> [=C2= =A0 =C2=A0 0.000000] ACPI: HPET id: 0x8086a201 base:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfed0000= 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> [=C2= =A0 =C2=A0 0.000000] clocksource: hpet: mask: 0xffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0max_cycle= s: 0xffffffff, max_idle_ns: 19112604467 ns
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> [=C2= =A0 =C2=A0 0.000000] hpet clockevent registered
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> [=C2= =A0 =C2=A0 0.362335] hpet0: at MMIO 0xfed00000, IRQs 2, 8,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> [=C2= =A0 =C2=A0 0.362339] hpet0: 3 comparators, 64-bit
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0100.00000= 0 MHz counter
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> [=C2= =A0 =C2=A0 0.661731] rtc_cmos 00:00: alarms up to one day,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0y3k, 114 = bytes nvram, hpet irqs
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Conspicuo= usly absent here is a line saying
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=E2=80=9C= clocksource: Switched to clocksource hpet=E2=80=9D.=C2=A0 That may
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0be worth = examining in more detail.
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Jonath= an Morton
>
>
>
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat= .net
> https://lists.bufferbloat.net/listinfo/bloat



--94eb2c05539c5ba5560546f26697--