[Cerowrt-devel] understanding_cerowrt_buildsystem

[Cerowrt-devel] understanding_cerowrt_buildsystem

[Oliver Niesner]

Hi!

After a zillion tries i managed to build a cerowrt image for myself.
It's not really clear what happened when it fails, but sometimes 
building a package again does the trick.
Here is another question to understand  better:

When building a package (eg. selecting a new one in menuconfig) does the 
buildsystem take care of
dependencies in other "parts" of the system? Let's say i do a package 
which requires something new
must be build in the linux kernel, but in the build before the kernel is 
already succesfully compiled. Will the
buildsystem automatically detect that it is neccessary to build the 
kernel again, or do i have to do it
manually, eg. do a "make clean"? My guess is that i have to do it manually.

Thanks, for clearing things up

Oliver

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Maciej Soltysiak]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

On Thu, Sep 20, 2012 at 9:10 AM, Oliver Niesner <oliver.niesner@gmail.com>wrote:

> Hi!
>
> After a zillion tries i managed to build a cerowrt image for myself.

I can only say I feel your pain as I don't think I've ever succeeded either.
Which sucks as it prevents me from supplying proper patches for Dave to
add minidlna, miniupnpd and minissdpd.

Also it makes it harder for Dave to delegate building to somebody else as
he hinted last time.

Could at least we make a list on the wiki of most common build problems and
questions?
- Like "build process stops because software package version X.Y.Z is no
longer available as it's been upgrader to X.Y.Z+1. where do I update the
version?"

Regards,
Maciej

[-- Attachment #2: Type: text/html, Size: 1132 bytes --]

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Mark Constable]

On 09/20/12 20:41, Maciej Soltysiak wrote:
> Also it makes it harder for Dave to delegate building to somebody else
> as he hinted last time. Could at least we make a list on the wiki of
> most common build problems and questions?

Wouldn't it be better to use the issue tracker and pull requests at Github?

The whole point of Github is to make is easier for folks to cooperate and
accelerate project development and it has excellent tools for doing exactly
that, as long as the direction of the project allows for it and takes
advantage of the available social facilities.

I'd like to get more involved too but it just seems all too hard atm so I
build stock AA OpenWrt uml images instead, for now, hoping I can get up to
speed with understanding the CeroWrt build process.

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Dave Taht]

Yes, getting cero (and openwrt) built right the first time is hard!

As for issue tracking, I'm not fond of redmine's system (particularly
as search on our implementation is busted and we have
100s of outstanding issues), but switching to github wholesale is not
particularly palatable at first glance. I have kept hoping that
something more compatable with me (something that interfaced to emacs
and didn't require the web to do *anything*) would arise.

My overall thought at the moment is to hold a "building
openwrt/cerowrt google hangout" with as many participants as possible,
get some your versions built, record the thing, see what barriers
there are to making it easier, and smash them. This can certainly
include trying to use github features harder. I do not currently
maintain or use the build_cero script...

I'm back in california now and migrating back to PDT, when would be a
good time to get everyone together? Saturday?

I note that I'm very free with commit access to the ceropackages repo
in particular. If, in order to make forward progress on the dlna, upnp
issues, we have to move those into ceropackages, go for it. The whole
point of ceropackages was to try to find ways to quickly spin up and
eventually push out, new, improved, interesting packages into openwrt
mainline.

Regardless pls send along your github ids, and I'll add you.

On Thu, Sep 20, 2012 at 6:05 AM, Mark Constable <markc@renta.net> wrote:
> On 09/20/12 20:41, Maciej Soltysiak wrote:
>> Also it makes it harder for Dave to delegate building to somebody else
>> as he hinted last time. Could at least we make a list on the wiki of
>> most common build problems and questions?
>
> Wouldn't it be better to use the issue tracker and pull requests at Github?
>
> The whole point of Github is to make is easier for folks to cooperate and
> accelerate project development and it has excellent tools for doing exactly
> that, as long as the direction of the project allows for it and takes
> advantage of the available social facilities.
>
> I'd like to get more involved too but it just seems all too hard atm so I
> build stock AA OpenWrt uml images instead, for now, hoping I can get up to
> speed with understanding the CeroWrt build process.
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel



-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
with fq_codel!"

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Mark Constable]

On 09/22/12 01:35, Dave Taht wrote:
> Yes, getting cero (and openwrt) built right the first time is hard!

As mentioned in a previous message, having an obvious "canonical"
meta repo would give the rest of us something to focus on. If I go
to https://github.com/dtaht it's not obvious where to start unless
I (or misc newbie) click around on the various cerowrt* projects or
go off site via googling for further directions.

Bufferbloat is an academic ideal whereas cerowrt is the actual
implementation that is of most interest to me, so I can eventually
get a handle on the deeper understanding of all the neat tools and
concepts (hands on first, rtfm later). As a newbie, something like
https://github.com/cerowrt and a https://github.com/cerowrt/build
project would make for the most intuitive starting points but just
https://github.com/dtaht/cerowrt would be the next most obvious...
even if it's only a documentation project with a single README.md

> Regardless pls send along your github ids, and I'll add you.

markc -> https://github.com/markc

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Maciej Soltysiak]

[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]

On Fri, Sep 21, 2012 at 5:35 PM, Dave Taht <dave.taht@gmail.com> wrote:

> My overall thought at the moment is to hold a "building
> openwrt/cerowrt google hangout" with as many participants as possible,
> get some your versions built, record the thing, see what barriers
> there are to making it easier, and smash them.
>
Sounds good! I don't know all the time zones people are in, but I'm in CEST
(GMT+2 at the moment).


> I'm back in california now and migrating back to PDT, when would be a
> good time to get everyone together? Saturday?
>
Short notice, but might be. I suggest you invite folks, see how many join,
how many reject but wanted to join.
And hold another session a week after.

For the agenda I'd suggest including these topics:
- quick walkthrough of build process
- resolving missing file issue
- adding new software to packages
- any advice on kernel config?

I note that I'm very free with commit access to the ceropackages repo
> in particular. If, in order to make forward progress on the dlna, upnp
> issues, we have to move those into ceropackages, go for it. The whole
> point of ceropackages was to try to find ways to quickly spin up and
> eventually push out, new, improved, interesting packages into openwrt
> mainline.
>


> Regardless pls send along your github ids, and I'll add you.
>
Although I've done my share of coding, patching, using cvs, I'm a github
newbe:
https://github.com/pysiak

Regards,
Maciej

[-- Attachment #2: Type: text/html, Size: 2260 bytes --]

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Dave Taht]

Well, let's stay speculative. Needent' be tomorrow.

What is a day - with at at least a 2 hour period, preferably more -
that works for the most people?

I'm pretty flexible as to what hours I keep, so pretend I'm not in PDT...

There was a time when I was available on irc 24/7 and #bufferbloat was
a hotbed of development, but of late that's been hard, and regardless,
the effort expended, diffuse, and worse, mostly unlogged.

I'm NOT huge on "meetings" per se', but having a regular get-together
*recorded*, seems like a very good idea at this point. Particularly on
this, focused topic, initially. And to not be a "meeting" per se', but
to get some actual work done, in concert, rather than asynchronously.

On Fri, Sep 21, 2012 at 2:08 PM, Maciej Soltysiak <maciej@soltysiak.com> wrote:
> On Fri, Sep 21, 2012 at 5:35 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> My overall thought at the moment is to hold a "building
>> openwrt/cerowrt google hangout" with as many participants as possible,
>> get some your versions built, record the thing, see what barriers
>> there are to making it easier, and smash them.
>
> Sounds good! I don't know all the time zones people are in, but I'm in CEST
> (GMT+2 at the moment).
>
>>
>> I'm back in california now and migrating back to PDT, when would be a
>> good time to get everyone together? Saturday?
>
> Short notice, but might be. I suggest you invite folks, see how many join,
> how many reject but wanted to join.
> And hold another session a week after.
>
> For the agenda I'd suggest including these topics:
> - quick walkthrough of build process
> - resolving missing file issue
> - adding new software to packages
> - any advice on kernel config?
>
>> I note that I'm very free with commit access to the ceropackages repo
>> in particular. If, in order to make forward progress on the dlna, upnp
>> issues, we have to move those into ceropackages, go for it. The whole
>> point of ceropackages was to try to find ways to quickly spin up and
>> eventually push out, new, improved, interesting packages into openwrt
>> mainline.
>
>
>>
>> Regardless pls send along your github ids, and I'll add you.
>
> Although I've done my share of coding, patching, using cvs, I'm a github
> newbe:
> https://github.com/pysiak
>
> Regards,
> Maciej
>



-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
with fq_codel!"

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Dave Taht]

On Thu, Sep 20, 2012 at 12:10 AM, Oliver Niesner
<oliver.niesner@gmail.com> wrote:
> Hi!
>
> After a zillion tries i managed to build a cerowrt image for myself.
> It's not really clear what happened when it fails, but sometimes building a
> package again does the trick.

When doing parallel builds, things can fail in random places as not
all the dependencies are captured in openwrt.

Also I've seen an issue when sometimes a file doesn't get built right
and fails a dependency later.

make package/{clean,compile,install}/the_package/name is usually of
help in both cases.

Sometimes merely doing a sequential build after a parallel build fails
is the only answer, too

make V=s


> Here is another question to understand  better:
>
> When building a package (eg. selecting a new one in menuconfig) does the
> buildsystem take care of
> dependencies in other "parts" of the system? Let's say i do a package which
> requires something new
> must be build in the linux kernel, but in the build before the kernel is
> already succesfully compiled. Will the
> buildsystem automatically detect that it is neccessary to build the kernel
> again, or do i have to do it
> manually, eg. do a "make clean"? My guess is that i have to do it manually.

Often it will "just work". But not often enough.

> Thanks, for clearing things up
>
> Oliver
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel



-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
with fq_codel!"

Re: [Cerowrt-devel] understanding_cerowrt_buildsystem

[Outback Dingo]

On Fri, Sep 21, 2012 at 6:16 PM, Dave Taht <dave.taht@gmail.com> wrote:
> On Thu, Sep 20, 2012 at 12:10 AM, Oliver Niesner
> <oliver.niesner@gmail.com> wrote:
>> Hi!
>>
>> After a zillion tries i managed to build a cerowrt image for myself.
>> It's not really clear what happened when it fails, but sometimes building a
>> package again does the trick.
>
> When doing parallel builds, things can fail in random places as not
> all the dependencies are captured in openwrt.
>
> Also I've seen an issue when sometimes a file doesn't get built right
> and fails a dependency later.
>
> make package/{clean,compile,install}/the_package/name is usually of
> help in both cases.
>
> Sometimes merely doing a sequential build after a parallel build fails
> is the only answer, too
>
> make V=s
>
>
>> Here is another question to understand  better:
>>
>> When building a package (eg. selecting a new one in menuconfig) does the
>> buildsystem take care of
>> dependencies in other "parts" of the system? Let's say i do a package which
>> requires something new
>> must be build in the linux kernel, but in the build before the kernel is
>> already succesfully compiled. Will the
>> buildsystem automatically detect that it is neccessary to build the kernel
>> again, or do i have to do it
>> manually, eg. do a "make clean"? My guess is that i have to do it manually.
>
> Often it will "just work". But not often enough.
>
>> Thanks, for clearing things up
>>
>> Oliver
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

Just so all are aware I spoke to Dave earlier and am taking a look at
the build system, Im
currently doing a build from scratch to see what potential gotchas,
pitfalls, fixes, or even
failures occur. Im also inclined to think, of possibly looking at a
logical integration, via feeds.
Theres alot of work here thats been done by Dave and it needs to be
easy for us who wish
to make our own builds / modifications / changes to be able to move
forward. It is also alot
easier to provide patches / fixes when you understand the whole build
system. So Im working
through this, and hopefully i can encourage my wife, whos a technical
writer to pitch in some
documentation time.



>
> --
> Dave Täht
> http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-17 is out
> with fq_codel!"
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel