[Cake] new 12 core box

[Cake] new 12 core box

[Dave Taht]

Linux spaceheater 4.15.0-rc1-cake-5 #1 SMP PREEMPT Sat Dec 2 12:50:23
PST 2017 x86_64 x86_64 x86_64 GNU/Linux

I had all kinds of trouble trying to install anything later than
ubuntu 16.04.3 on it.

A kernel build still took an hour, and it turns out that time is
dominated by the single threaded debian packager, where less cores at
more clock would help. Still have time for naps, except that it's so
loud to require earplugs.

-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] new 12 core box

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]


> On Dec 2, 2017, at 11:10 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> Linux spaceheater 4.15.0-rc1-cake-5 #1 SMP PREEMPT Sat Dec 2 12:50:23
> PST 2017 x86_64 x86_64 x86_64 GNU/Linux
> 
> I had all kinds of trouble trying to install anything later than
> ubuntu 16.04.3 on it.
> 
> A kernel build still took an hour, and it turns out that time is
> dominated by the single threaded debian packager, where less cores at
> more clock would help. Still have time for naps, except that it's so
> loud to require earplugs.

I like the name. “dornierdox” doesn’t have the same smooth ring: https://en.wikipedia.org/wiki/Dornier_Do_X <https://en.wikipedia.org/wiki/Dornier_Do_X> (Powerplant: 12× Curtiss Conqueror water-cooled V12) Who decides to build that?

I still think a box like that will be useful for veth testing.

I’ve been really happy with the 2x APU2 you suggested, for their four cores and silence. I even think 2-3 more of them could be useful for different testing topologies as we discussed. Next time though, I won’t futz around when attaching the heat spreader, just fix the thermal adhesive, mount it to the board straight away and be careful not to twist anything. First attempt: apu2a. Got it right: apu2b. The second box is consistently 5C cooler, though both are well within spec, usually under 65C under full load.

sysadmin@apu2a:~$ sensors
k10temp-pci-00c3
temp1:        +52.4°C  (high = +70.0°C)
                       (crit = +105.0°C, hyst = +104.0°C)

sysadmin@apu2b:~$ sensors
k10temp-pci-00c3
temp1:        +46.6°C  (high = +70.0°C)


[-- Attachment #2: Type: text/html, Size: 2610 bytes --]

Re: [Cake] new 12 core box

[Dave Taht]

On Sun, Dec 3, 2017 at 3:37 AM, Pete Heist <peteheist@gmail.com> wrote:
>
> On Dec 2, 2017, at 11:10 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
> Linux spaceheater 4.15.0-rc1-cake-5 #1 SMP PREEMPT Sat Dec 2 12:50:23
> PST 2017 x86_64 x86_64 x86_64 GNU/Linux
>
> I had all kinds of trouble trying to install anything later than
> ubuntu 16.04.3 on it.
>
> A kernel build still took an hour, and it turns out that time is
> dominated by the single threaded debian packager, where less cores at
> more clock would help. Still have time for naps, except that it's so
> loud to require earplugs.
>
>
> I like the name. “dornierdox” doesn’t have the same smooth ring:
> https://en.wikipedia.org/wiki/Dornier_Do_X (Powerplant: 12× Curtiss
> Conqueror water-cooled V12)

Heh. I had not heard of that plane. Ceiling, 500 feet.

>Who decides to build that?

Howard Hughes?

https://en.wikipedia.org/wiki/Hughes_H-4_Hercules


>
> I still think a box like that will be useful for veth testing.

I put it through some paces last night, trying 4Gbit workloads.

>
> I’ve been really happy with the 2x APU2 you suggested, for their four cores
> and silence. I even think 2-3 more of them could be useful for different
> testing topologies as we discussed.

I like them a lot, too. We have to come up with a test that uses up 3
ethernet ports...

> Next time though, I won’t futz around
> when attaching the heat spreader, just fix the thermal adhesive, mount it to
> the board straight away and be careful not to twist anything. First attempt:
> apu2a. Got it right: apu2b. The second box is consistently 5C cooler, though
> both are well within spec, usually under 65C under full load.
>
> sysadmin@apu2a:~$ sensors
> k10temp-pci-00c3
> temp1:        +52.4°C  (high = +70.0°C)
>                        (crit = +105.0°C, hyst = +104.0°C)
>
> sysadmin@apu2b:~$ sensors
> k10temp-pci-00c3
> temp1:        +46.6°C  (high = +70.0°C)
>

I'll go install that. I have an old custom version of lede on two of 'em..

-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] new 12 core box

[Dave Taht]

On Sun, Dec 3, 2017 at 9:04 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Sun, Dec 3, 2017 at 3:37 AM, Pete Heist <peteheist@gmail.com> wrote:
>>
>> On Dec 2, 2017, at 11:10 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> Linux spaceheater 4.15.0-rc1-cake-5 #1 SMP PREEMPT Sat Dec 2 12:50:23
>> PST 2017 x86_64 x86_64 x86_64 GNU/Linux
>>
>> I had all kinds of trouble trying to install anything later than
>> ubuntu 16.04.3 on it.
>>
>> A kernel build still took an hour, and it turns out that time is
>> dominated by the single threaded debian packager, where less cores at
>> more clock would help. Still have time for naps, except that it's so
>> loud to require earplugs.
>>
>>
>> I like the name. “dornierdox” doesn’t have the same smooth ring:
>> https://en.wikipedia.org/wiki/Dornier_Do_X (Powerplant: 12× Curtiss
>> Conqueror water-cooled V12)
>
> Heh. I had not heard of that plane. Ceiling, 500 feet.

"As a result of its size, passengers were asked to crowd together on
one side or the other to help make turns. "

very apt analogy for these mailing lists!

"
"

>>Who decides to build that?
>
> Howard Hughes?
>
> https://en.wikipedia.org/wiki/Hughes_H-4_Hercules
>
>
>>
>> I still think a box like that will be useful for veth testing.
>
> I put it through some paces last night, trying 4Gbit workloads.
>
>>
>> I’ve been really happy with the 2x APU2 you suggested, for their four cores
>> and silence. I even think 2-3 more of them could be useful for different
>> testing topologies as we discussed.
>
> I like them a lot, too. We have to come up with a test that uses up 3
> ethernet ports...
>
>> Next time though, I won’t futz around
>> when attaching the heat spreader, just fix the thermal adhesive, mount it to
>> the board straight away and be careful not to twist anything. First attempt:
>> apu2a. Got it right: apu2b. The second box is consistently 5C cooler, though
>> both are well within spec, usually under 65C under full load.
>>
>> sysadmin@apu2a:~$ sensors
>> k10temp-pci-00c3
>> temp1:        +52.4°C  (high = +70.0°C)
>>                        (crit = +105.0°C, hyst = +104.0°C)
>>
>> sysadmin@apu2b:~$ sensors
>> k10temp-pci-00c3
>> temp1:        +46.6°C  (high = +70.0°C)
>>
>
> I'll go install that. I have an old custom version of lede on two of 'em..
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] new 12 core box

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]


> On Dec 3, 2017, at 6:07 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> On Sun, Dec 3, 2017 at 9:04 AM, Dave Taht <dave.taht@gmail.com <mailto:dave.taht@gmail.com>> wrote:
>> On Sun, Dec 3, 2017 at 3:37 AM, Pete Heist <peteheist@gmail.com> wrote:
>>> 
>>> I like the name. “dornierdox” doesn’t have the same smooth ring:
>>> https://en.wikipedia.org/wiki/Dornier_Do_X (Powerplant: 12× Curtiss
>>> Conqueror water-cooled V12)
>> 
>> Heh. I had not heard of that plane. Ceiling, 500 feet.
> 
> "As a result of its size, passengers were asked to crowd together on
> one side or the other to help make turns. "
> 
> very apt analogy for these mailing lists!

Indeed. "A successor, the Do-20, was envisioned by Dornier, but never advanced beyond the design study stage.”

Really? I’m shocked.

>>> Who decides to build that?
>> Howard Hughes?
>> https://en.wikipedia.org/wiki/Hughes_H-4_Hercules <https://en.wikipedia.org/wiki/Hughes_H-4_Hercules>

Another behemoth...

>>> I still think a box like that will be useful for veth testing.
>> 
>> I put it through some paces last night, trying 4Gbit workloads.
>> 
>>> 
>>> I’ve been really happy with the 2x APU2 you suggested, for their four cores
>>> and silence. I even think 2-3 more of them could be useful for different
>>> testing topologies as we discussed.
>> 
>> I like them a lot, too. We have to come up with a test that uses up 3
>> ethernet ports...

I have just such a test planned, as a middlebox for poor-man’s full-duplex p2p WiFi. One port in, one for egress and one for ingress, same on the other end of the link. I tried it last year with 2xAPUv1 but one link was still a cable because I didn’t have four NSM5s, so the results looked artificially good. Now I have them, so it’s just a matter of setting it up again...

[-- Attachment #2: Type: text/html, Size: 8024 bytes --]

Re: [Cake] new 12 core box

[Dave Taht]

On Sun, Dec 3, 2017 at 11:26 AM, Pete Heist <peteheist@gmail.com> wrote:
>
> On Dec 3, 2017, at 6:07 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
> On Sun, Dec 3, 2017 at 9:04 AM, Dave Taht <dave.taht@gmail.com> wrote:
>
> On Sun, Dec 3, 2017 at 3:37 AM, Pete Heist <peteheist@gmail.com> wrote:
>
> I like the name. “dornierdox” doesn’t have the same smooth ring:
> https://en.wikipedia.org/wiki/Dornier_Do_X (Powerplant: 12× Curtiss
> Conqueror water-cooled V12)
>
>
> Heh. I had not heard of that plane. Ceiling, 500 feet.
>
>
> "As a result of its size, passengers were asked to crowd together on
> one side or the other to help make turns. "
>
> very apt analogy for these mailing lists!
>
>
> Indeed. "A successor, the Do-20, was envisioned by Dornier, but never
> advanced beyond the design study stage.”
>
> Really? I’m shocked.
>
> Who decides to build that?
>
> Howard Hughes?
> https://en.wikipedia.org/wiki/Hughes_H-4_Hercules
>
>
> Another behemoth...
>
> I still think a box like that will be useful for veth testing.
>
>
> I put it through some paces last night, trying 4Gbit workloads.
>
>
> I’ve been really happy with the 2x APU2 you suggested, for their four cores
> and silence. I even think 2-3 more of them could be useful for different
> testing topologies as we discussed.
>
>
> I like them a lot, too. We have to come up with a test that uses up 3
> ethernet ports...
>
>
> I have just such a test planned, as a middlebox for poor-man’s full-duplex
> p2p WiFi. One port in, one for egress and one for ingress, same on the other
> end of the link. I tried it last year with 2xAPUv1 but one link was still a
> cable because I didn’t have four NSM5s, so the results looked artificially
> good. Now I have them, so it’s just a matter of setting it up again...

Cool. Another idea for a test is a real torrent or 5 of a really big
well distributed file on one etherport, while a set of hosts do
normal-home like things on the other, with cake unlimited vs fq-codel,
and bandwidth limited for both. We don't have good numbers for cake at
line rate in a scenario where we can dump 2Gbit into 1 in the real
world.

-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619