keeping a better lab notebook, publishing results, etc?

[Dave Taht]

Over lunch the other day Julius (correctly) dinged me for not keeping
a better lab notebook than I have been.

In my defense, I'm working on a dozen machines on two continents, and
there are no tools common to them. And: one thing that drives me crazy
about redmine for everything is that I'm frequently offline... or
saturating the network!

I am hoping - after spending the last year basically getting a broad
overview of the bloat problem - to be doing some real science in the
next couple quarters.

I managed a lot of the early development of cerowrt (and
bufferbloat.net in the first 4 months, too!) in org-mode. I was much
more effective with coping with the (too) many details on my task list
that way.

I just spent the last week automating some tests in lua (as well as
rewriting the AQM driver in lua)

https://github.com/dtaht/deBloat

(I note that I cracked the manual on lua dec 19th, 2011, and I'm not
quite at co-routines and continuations in my grasp of it yet - but I'm
liking it, and it's the only decent language I can use on cerowrt.
Yes, I would still prefer to be coding in lisp...)

Once I got most of the wrappers around the tools done....

I had to think hard upon the best way to store the data. My first
inclination was postgres (vs the mark #1 eyeball I'd been using) but
the data set size is usually so small (60 seconds of pings) there is
no point. Also there's no easy way to get data directly into postgres
from a cerowrt box...

Packet captures, on the other hand, are so huge, as to also have not
much point to storing in a db...

And then there's the data interchange, where I figured json would be
the 'thing'. Then there's plotting the results - I'd like to find a
json plotting library, I have tenatively settled on using flot. (note
- again - I'm offline a lot, so google charts is right out). Other
suggestions welcomed.

So, my second inclination was json as an intermediate format.

So, I stewed about for a while, then looked at org-mode, again.

Turns out, it has a gnuplot interface that's pretty decent (see
screenshot), and org-mode has a great tables interface, and I can
merge data sets with a couple keystrokes (copy-rectangle) (on top of
managing my life, irc, code, etc in emacs), and it looks like I can
import/export from json fairly well from it...

So I'm leaning towards using org-mode for more stuff, although I
realize that few have the love or experience with emacs I do, and json
as an intermediate format I can reuse later...

Also: I fiddled a bit with ikiwiki again - it really is wonderful to
have a wiki .1ms away vs 170! It would take me, like a day, to redo
the existing redmine wiki in ikiwiki (which supports textile)  and I
have ikiwiki running, via rsync, on cerowrt, too.

I like redmine's bug tracker, but really need the bugs managable
offline, as well. Most of the work on distributed bug trackers seems
to have died, sigh.

All that said, using all these systems for managing my life is too
much. But redmine is not enough.

So I was curious as to what other means we could come up with to
manage the bloat infrastructure better, make it more usable, AND still
be able to share the work as it happens?

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gnuplotemacs.png
Type: image/png
Size: 95781 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/bloat-devel/attachments/20120101/4f7024ee/attachment-0002.png>