[Cerowrt-devel] Cerowrt 3.7.2-4 released, multicast (PIM and IGMP) fixed, kernel matches 3.3.8 again

[Cerowrt-devel] Cerowrt 3.7.2-4 released, multicast (PIM and IGMP) fixed, kernel matches 3.3.8 again

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

OK, well, I enabled pim, robert bradley killed more traps, and I bumped the
clock rate up,
so we basically now have the same kernel config that 3.3.8 had.

As well as had polipo try TFO by default, and update uftp to 3.7.1, and
tried to
config dhcpv6 appropriately for dhcpv6, and twiddle some firewall rules.

And booted it up only to see no /proc/whatever/net/igmp file once again,
and no 224.x.x.x in ip maddr...

It turned out the igmp entries in /proc ARE being  stripped out by a
openwrt patch... which I killed... so I built it again....

and walla! ip maddr shows multicast stuff and the igmp file exists.

Hopefully this will resolve the dlna issue and others.

Get it at:

http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.7.2-4/

BTW:

I had a little fun with uftp and uftpd with this (which are capable
of thoroughly exercising multicast and igmp)

http://www.tcnj.edu/~bush/uftp.html

Basically setting up a uftpd client on a couple of machines and
initiating uftp from the router, or vice versa, I was able to transfer
multiple files around at the same time to multiple machines
over wifi and wired.

I've long yearned to have time to truly benchmark multicast
at scale, and that's why uftp has been in there so long...

To setup uftpd on cero (the file receiving deamon)

uftpd -I se00,sw00,sw10,gw00,gw10 -D /tmp # or somewhere

To send a file from somewhere to a listening uftpd

uftp -I the_interface thefile

tons of options, and it's kind of inobvious that sending files
requires an interface to send on, and that the mesh interfaces
won't multicast...

happy multicasting. Have fun with high rates over wifi,
in particular....

-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 2211 bytes --]

Re: [Cerowrt-devel] Cerowrt 3.7.2-4 released, multicast (PIM and IGMP) fixed, kernel matches 3.3.8 again

[Maciej Soltysiak]

[-- Attachment #1: Type: text/plain, Size: 3320 bytes --]

On Thu, Jan 17, 2013 at 2:41 PM, Dave Taht <dave.taht@gmail.com> wrote:

> It turned out the igmp entries in /proc ARE being  stripped out by a
> openwrt patch... which I killed... so I built it again....
>
What was the purpose of the patch, do you know?


> Hopefully this will resolve the dlna issue and others.

It does, my wired TV and my wireless client work without a single change of
config!

However I got hit by the DNS / polipo bug with things like:
Jan 17 18:41:43 OpenWrt user.err polipo[2765]: Host
mail.soltysiak.comlookup failed: Timeout (131072).
Jan 17 18:41:43 OpenWrt user.info sysinit: Host mail.soltysiak.com lookup
failed: Timeout (131072).
Jan 17 18:41:44 OpenWrt user.err polipo[2765]: Host
notify5.dropbox.comlookup failed: Timeout (131072).
Jan 17 18:41:44 OpenWrt user.info sysinit: Host notify5.dropbox.com lookup
failed: Timeout (131072).
Jan 17 18:43:27 OpenWrt user.err polipo[2765]: Host
notify5.dropbox.comlookup failed: Timeout (131072).
Jan 17 18:43:27 OpenWrt user.info sysinit: Host notify5.dropbox.com lookup
failed: Timeout (131072).
Funny is that I didn't see them with 3.7.1-1.


> BTW:
>
> I had a little fun with uftp and uftpd with this (which are capable
> of thoroughly exercising multicast and igmp)
>
> http://www.tcnj.edu/~bush/uftp.html
>
> Basically setting up a uftpd client on a couple of machines and
> initiating uftp from the router, or vice versa, I was able to transfer
> multiple files around at the same time to multiple machines
> over wifi and wired.
>
> I've long yearned to have time to truly benchmark multicast
> at scale, and that's why uftp has been in there so long...
>
> To setup uftpd on cero (the file receiving deamon)
>
> uftpd -I se00,sw00,sw10,gw00,gw10 -D /tmp # or somewhere
>
uftp and uftpd don't seem to be in cero repository.
downloads.openwrt.org/snapshot/trunk/... has version 3.6.1, the latest
version is 3.7.1; the best thing would be to update upstream openwrt.


> To send a file from somewhere to a listening uftpd
>
> uftp -I the_interface thefile
>
To be super weird, I used a 3.7.1 win32 client and 3.6.1 openwrt
daemon. speed I got wasn't very high, example:

D:\TEMP>uftp DSC00160.JPG
UFTP version 3.7.1  Copyright (C) 2001-2012  Dennis A. Bush
Starting at Thu Jan 17 20:06:33 2013
Transfer rate: 1000 Kbps (125 KB/s)
Wait between packets: 11718 us
Using private multicast address 230.5.5.59  Group ID: 033D43BF
Initializing group
Sending ANNOUNCE 1
Received REGISTER from client 172.30.42.97
Received REGISTER+ from client 172.30.42.97
Sending REG_CONF 2.1
Sending ANNOUNCE 2
----- DSC00160.JPG -----
File ID: 0001  Name: DSC00160.JPG
  sending as: DSC00160.JPG
Bytes: 869125  Blocks: 602  Sections: 1
Sending FILEINFO 1.1
Received INFO_ACK from client 172.30.42.97
Maximum file transfer time: 20 seconds
Sending file...pass 1
Sending DONE 1.1
Got COMPLETE from client 172.30.42.97
Average wait time = 11738.89 us
Received 0 distinct NAKs for pass 1
Transfer status:
Host: 172.30.42.97     Status: Completed   time:   7.082 seconds    NAKs: 0
Total elapsed time: 7.082 seconds
Overall throughput: 119.84 KB/s
-----------------------------
Finishing group
Sending DONE 1.1
Got COMPLETE from client 172.30.42.97
Late completions:
Sending DONE_CONF 2.1
Group complete
uftp: Finishing at Thu Jan 17 20:06:56 2013
Regards,
Maciej

[-- Attachment #2: Type: text/html, Size: 5042 bytes --]

Re: [Cerowrt-devel] Cerowrt 3.7.2-4 released, multicast (PIM and IGMP) fixed, kernel matches 3.3.8 again

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 5963 bytes --]

On Thu, Jan 17, 2013 at 2:10 PM, Maciej Soltysiak <maciej@soltysiak.com>wrote:

> On Thu, Jan 17, 2013 at 2:41 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
>> It turned out the igmp entries in /proc ARE being  stripped out by a
>> openwrt patch... which I killed... so I built it again....
>>
> What was the purpose of the patch, do you know?
>

A somewhat misguided attempt at "saving space"


>
>
>> Hopefully this will resolve the dlna issue and others.
>
> It does, my wired TV and my wireless client work without a single change
> of config!
>

YEA!


>
> However I got hit by the DNS / polipo bug with things like:
> Jan 17 18:41:43 OpenWrt user.err polipo[2765]: Host mail.soltysiak.comlookup failed: Timeout (131072).
> Jan 17 18:41:43 OpenWrt user.info sysinit: Host mail.soltysiak.com lookup
> failed: Timeout (131072).
> Jan 17 18:41:44 OpenWrt user.err polipo[2765]: Host notify5.dropbox.comlookup failed: Timeout (131072).
> Jan 17 18:41:44 OpenWrt user.info sysinit: Host notify5.dropbox.comlookup failed: Timeout (131072).
> Jan 17 18:43:27 OpenWrt user.err polipo[2765]: Host notify5.dropbox.comlookup failed: Timeout (131072).
> Jan 17 18:43:27 OpenWrt user.info sysinit: Host notify5.dropbox.comlookup failed: Timeout (131072).
> Funny is that I didn't see them with 3.7.1-1.
>

As I keep noting, it seems to take time to occur. Try usednsgethostbyname =
true for a while. (can't remember the capitalization)

I'd love to nail this bug one one day because proxies really help on wifi
in particular.... but next up on my todo list this week
is nailing the last few traps and then thoroughly exercising ipv6. (in
addition to a bunch of ns2 based sim work,
which is actually my primary task)

Some full packet captures from localhost when it starts failing would be
useful. Also try it talking to localhost via ::1.


>
>> BTW:
>>
>> I had a little fun with uftp and uftpd with this (which are capable
>> of thoroughly exercising multicast and igmp)
>>
>> http://www.tcnj.edu/~bush/uftp.html
>>
>> Basically setting up a uftpd client on a couple of machines and
>> initiating uftp from the router, or vice versa, I was able to transfer
>> multiple files around at the same time to multiple machines
>> over wifi and wired.
>>
>> I've long yearned to have time to truly benchmark multicast
>> at scale, and that's why uftp has been in there so long...
>>
>> To setup uftpd on cero (the file receiving deamon)
>>
>> uftpd -I se00,sw00,sw10,gw00,gw10 -D /tmp # or somewhere
>>
> uftp and uftpd don't seem to be in cero repository.
> downloads.openwrt.org/snapshot/trunk/... has version 3.6.1, the latest
> version is 3.7.1; the best thing would be to update upstream openwrt.
>

It's there.

opkg update
opkg install uftpd uftp

I uypdated my version to 3.7.1, will push the patches out to openwrt as
soon as I get a few more cleanups.


>
>> To send a file from somewhere to a listening uftpd
>>
>> uftp -I the_interface thefile
>>
> To be super weird, I used a 3.7.1 win32 client and 3.6.1 openwrt
> daemon. speed I got wasn't very high, example:
>
> D:\TEMP>uftp DSC00160.JPG
> UFTP version 3.7.1  Copyright (C) 2001-2012  Dennis A. Bush
> Starting at Thu Jan 17 20:06:33 2013
> Transfer rate: 1000 Kbps (125 KB/s)
> Wait between packets: 11718 us
> Using private multicast address 230.5.5.59  Group ID: 033D43BF
> Initializing group
> Sending ANNOUNCE 1
> Received REGISTER from client 172.30.42.97
> Received REGISTER+ from client 172.30.42.97
> Sending REG_CONF 2.1
> Sending ANNOUNCE 2
> ----- DSC00160.JPG -----
> File ID: 0001  Name: DSC00160.JPG
>   sending as: DSC00160.JPG
> Bytes: 869125  Blocks: 602  Sections: 1
> Sending FILEINFO 1.1
> Received INFO_ACK from client 172.30.42.97
> Maximum file transfer time: 20 seconds
> Sending file...pass 1
> Sending DONE 1.1
> Got COMPLETE from client 172.30.42.97
> Average wait time = 11738.89 us
> Received 0 distinct NAKs for pass 1
> Transfer status:
> Host: 172.30.42.97     Status: Completed   time:   7.082 seconds    NAKs: 0
> Total elapsed time: 7.082 seconds
> Overall throughput: 119.84 KB/s
> -----------------------------
> Finishing group
> Sending DONE 1.1
> Got COMPLETE from client 172.30.42.97
> Late completions:
> Sending DONE_CONF 2.1
> Group complete
> uftp: Finishing at Thu Jan 17 20:06:56 2013
> Regards,
> Maciej
>
>

uftp is a VERY interesting protocol. For starters it defaults to a very low
rate, which you can change,
which is what I think you just saw here. If you aren't getting NAKs you
aren't saturating your network.
Try a gigabit. :)

if you want to blow up your wifi, feel free to set it high. I have high
hopes that fq_codel will
still do something of the right thing with it, but as multicast packets
have a high "weight",
am not particularly hopeful. Need to script a test or three with it, now
that it works.

Secondly, it has a user configurable congestion control algorithm which is
also very interesting,
and yet very primitive.

Thirdly, it can multicast encrypted it's content. That's a pretty unique
facility.

The web page describes its primary use case (distributing articles over
satellite), I have a ton of
others:

1) Keeping core files in sync across a network (think dns)
2) generating varying loads to route across wired and wifi with the
uftp-proxy
to test stuff with
3) blowing up igmp on bridges and elsewhere
4) sharing dns caches
5) co-existing with ccnx
6) video content (well, this is better done via vlc)
7) synchronized distributed audio content (linux's sound daemons can do
this)

I always liked multicast's promise, going back 2 decades now. I'll try not
to fool with it too much,
but I have to admit it can be like wet paint once you get it working right.

-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 8852 bytes --]