From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-f47.google.com (mail-wg0-f47.google.com [74.125.82.47]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 80B13201B20 for ; Sun, 29 Apr 2012 19:42:26 -0700 (PDT) Received: by wgbfa7 with SMTP id fa7so1856569wgb.28 for ; Sun, 29 Apr 2012 19:42:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=EIGrSWzwdNPUyQpkiw8X4lb0JTEaCqU5xCD+JztSsGg=; b=n0y5STUenfFwK9Wu4fMWZRIEubCTwOXGXOmwg9Fi5nBkOYN1Fc3lcfqrdBhtHyjgeP u6xobN85tz/V5oomHNu00NJ0tYEqHPIUc45E9xHy6qS/zS9okSx6mMTkJaVX5am8bgR8 7R6vp3Z/ZGBwqu5gMEDvhnkXiWqCXcENFxvLg+qtnnl7bzieFwTyHimUYPADL6kkh6iH 38wFuEVOupTr2zr35MHYAkk/bVlGg74X1i6Q+5EnkbWWKzRJaMouIivKMYw7uikJgjF3 c+JCKzOl7yaH9l8lYSQquXUOeUbPmnNYKg56RgoEfUOZaKGneaghyi0Me0PlaeyW3Kz6 qP/w== MIME-Version: 1.0 Received: by 10.216.138.17 with SMTP id z17mr11947547wei.18.1335753744191; Sun, 29 Apr 2012 19:42:24 -0700 (PDT) Received: by 10.223.112.66 with HTTP; Sun, 29 Apr 2012 19:42:24 -0700 (PDT) In-Reply-To: References: Date: Sun, 29 Apr 2012 19:42:24 -0700 Message-ID: From: Dave Taht To: Outback Dingo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] speeding up builds X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Apr 2012 02:42:27 -0000 On Sun, Apr 29, 2012 at 7:24 PM, Dave Taht wrote: > On Sun, Apr 29, 2012 at 6:59 PM, Dave Taht wrote: >> On Sun, Apr 29, 2012 at 6:42 PM, Outback Dingo = wrote: >>> On Sun, Apr 29, 2012 at 8:15 PM, Dave Taht wrote: >>>> I finally acquired a machine with 32GB of ram, an intel 3930k (6 >>>> cores), and an SSD. >>>> >>>> I put the build_dir, /tmp and /var/tmp on ramdisks, and... >>>> >>>> This cut a complete cerowrt build (including toolchain) down from > >>>> 3.5 hrs down to under 45 minutes. >>>> >>>> Without the toolchain rebuild, but after a make clean (to rebuild the >>>> packages and kernel), it's about 28 minutes. >>>> >>>> I can see that it is possible to parallelize things more to maybe chop >>>> another 30% of of things... >>>> ...but I'm glad to have 3 hrs of my life back, per build. >>>> >>>> I wanted to figure out to what extent modern hardware would enhance >>>> the existing buildbot system. >>>> Now I know... >>> >>> odd my laptop will do a full build with tool chain in about an >>> hour..... its only a core i3 with 6gb and an ssd >> >> The best box that I had was huchra, a dual quad-core xeon circa 2006, >> with 8GB of memory and mirrored drives. >> >> A 'full build' of cero is 578 packages, some of which are rather big, >> as well as building the sdk and cross development kit. >> >> For comparison purposes, I just built linux-3.3.4 for ubuntu (so this >> includes the kpkg overhead) >> >> real =A0 =A011m12.286s >> user =A0 =A067m11.076s >> sys =A0 =A0 7m19.955s >> >> I am puzzled. I end up with only 75MB for disk buffers, according to >> top, and I would assume that 25% of memory in this case would be good >> for disk buffers. >> >> I do like using ramdisks for this job, (why write to media unless you >> have to?) but it seems saner to have the disk cache, caching. > > Ah. I assume that 'cached' here means disk buffers. Maybe. > > =A0 =A0 =A0 =A0 =A0 =A0 total =A0 =A0 =A0 used =A0 =A0 =A0 free =A0 =A0 s= hared =A0 =A0buffers =A0 =A0 cached > Mem: =A0 =A0 =A032927452 =A0 28799604 =A0 =A04127848 =A0 =A0 =A0 =A0 =A00= =A0 =A0 =A075600 =A0 25122928 > -/+ buffers/cache: =A0 =A03601076 =A0 29326376 > Swap: =A0 =A0 33529852 =A0 =A01527668 =A0 32002184 > > Believe me, after doing the number of builds I've done this year, and > especially in the past two months, finding ways to shave even a few > minutes more off the build(s) would be a godsend. > > This particular box can do 64GB of ram, and doing that would add two > channels to the memory controller, assuming I plugged the ram in > wrong... > > anyway, a pure kernel build (no kpkg), > > time make -j 24 > > real =A0 =A07m33.494s > user =A0 =A073m3.146s > sys =A0 =A0 6m31.648s > > I see from the phoronix benchmarks that they claim a box of this > caliber can do a kernel build in under 60sec, but I doubt they are > using a kernel of this size. > > I've tossed the kernel .deb files, kernel config, script to make it a > deb, and patches here: > > http://huchra.bufferbloat.net/~d/debloat/ > > (note - TOTALLY untested on x86_64 as yet -) > > I'd gotten out of the habit of maintaining debloat-testing mostly > because doing a kernel build was taking so bloody long. And I just did a build right to the ssd, no ramdisk... real 7m41.516s user 71m6.395s sys 6m24.132s So it looks like, at least at present, with an SSD I/O is not the bottleneck... Now, from a buildbot perspective I'd really rather not light up a SSD but use up ram. Although I'm told they have got better. I still dream of 60s kernel builds tho... hah. the phoronix build test is available to all... /me has cpu cycles to burn and is working on something else --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://www.bufferbloat.net