[Cerowrt-devel] lede integration issues remaining from the detrius of cerowrt

[Cerowrt-devel] lede integration issues remaining from the detrius of cerowrt

[Dave Taht]

happy to see cake working today! thx all!

In https://github.com/dtaht/ceropackages-3.10:

A) We have flent's tc-iterate.c as a fast, lightweight means of
polling fq_codel, pie, cake stats (as the shell in openwrt doesn't
support a subsecond sleep) for openwrt.

(However this package and code is in need of iteration to support
polling the new stats presented in the fq_codel for wifi sysfs code.)

Having something like this working is essential to analyze the real
performance and correctness of the new implementations on the low end
hardware.

B) I am probably the only user of the gnugol package. :(

C) I really need to sync the isochronous code with what avery has upstream

D) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/nanom5poe

I don't know what landed upstream to control poe for the nano-m5
radios, if anything?

E) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/gdisk

The principal problem with fdisk nowadays is that very large (> 2TB, I
think) devices are not supported by it, and require a GPT capable
tool. Is there a replacement in lede that handles GPT? If not - this
is an old gdisk port to openwrt that I used to use.

F) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/cerowrt-scripts
was a tool we used to measure latency under load directly on the
router...

some other packages that might be worth upstreaming, or putting in
another repo or updating in ceropackages

owamp: lets us measure ping times very precisely. (requires good ntp)
ntpsec - a vastly hardened full fledged ntp fork https://www.ntpsec.org/
scamper - two other measurement tools
shaperprobe

?

G) We used lighttpd in cerowrt as it was tons faster than uhttpd and
flexible enough to also do local web serving. can it (or ngnix) be
substituted for uhttpd? Or has uhttpd got faster?

H) Anyone working on go or rust for lede? hugo and ipfs and all the
other blockchain related distributed web bits looks like fun...

It looks like the static linking in go is a deal-killer

https://github.com/GeertJohan/openwrt-go/issues/2

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Rosen Penev]

[-- Attachment #1: Type: text/plain, Size: 2570 bytes --]

E: current fdisk versions support GPT. g must be used instead of o in order
to create it. from what i've seen, both tools work the same(2048 sector
padding, etc...) for GPT.

On Sat, Jun 11, 2016 at 10:46 AM Dave Taht <dave.taht@gmail.com> wrote:

> happy to see cake working today! thx all!
>
> In https://github.com/dtaht/ceropackages-3.10:
>
> A) We have flent's tc-iterate.c as a fast, lightweight means of
> polling fq_codel, pie, cake stats (as the shell in openwrt doesn't
> support a subsecond sleep) for openwrt.
>
> (However this package and code is in need of iteration to support
> polling the new stats presented in the fq_codel for wifi sysfs code.)
>
> Having something like this working is essential to analyze the real
> performance and correctness of the new implementations on the low end
> hardware.
>
> B) I am probably the only user of the gnugol package. :(
>
> C) I really need to sync the isochronous code with what avery has upstream
>
> D) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/nanom5poe
>
> I don't know what landed upstream to control poe for the nano-m5
> radios, if anything?
>
> E) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/gdisk
>
> The principal problem with fdisk nowadays is that very large (> 2TB, I
> think) devices are not supported by it, and require a GPT capable
> tool. Is there a replacement in lede that handles GPT? If not - this
> is an old gdisk port to openwrt that I used to use.
>
> F)
> https://github.com/dtaht/ceropackages-3.10/tree/master/utils/cerowrt-scripts
> was a tool we used to measure latency under load directly on the
> router...
>
> some other packages that might be worth upstreaming, or putting in
> another repo or updating in ceropackages
>
> owamp: lets us measure ping times very precisely. (requires good ntp)
> ntpsec - a vastly hardened full fledged ntp fork https://www.ntpsec.org/
> scamper - two other measurement tools
> shaperprobe
>
> ?
>
> G) We used lighttpd in cerowrt as it was tons faster than uhttpd and
> flexible enough to also do local web serving. can it (or ngnix) be
> substituted for uhttpd? Or has uhttpd got faster?
>
> H) Anyone working on go or rust for lede? hugo and ipfs and all the
> other blockchain related distributed web bits looks like fun...
>
> It looks like the static linking in go is a deal-killer
>
> https://github.com/GeertJohan/openwrt-go/issues/2
>
> _______________________________________________
> Lede-dev mailing list
> Lede-dev@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev
>

[-- Attachment #2: Type: text/html, Size: 3805 bytes --]

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Lars Kruse]

Hi Dave,

> D) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/nanom5poe
> 
> I don't know what landed upstream to control poe for the nano-m5
> radios, if anything?

I submitted a patch (that was accepted) for GPIO-based POE control - at least
for Nanostations XM/XW and for TP-Link CPE210 and CPE510:
 https://git.lede-project.org/?p=source.git;a=commitdiff;h=d65916047b44d6d157d88d15e8e3d92555c5e6f8

Only the luci interface is missing ...

Cheers,
Lars

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Daniel Curran-Dickinson]

Hi Dave,

I don't speak for the LEDE team, but it looks to me a lot of your
problem is that you are using LEDE/openwrt for far bigger iron than the
primary target (standard routers, including pre-AC non-NAND ones, which
are really quite small and low powered).  2 TB+ storage for example, or
using lighttpd instead of uhttpd are really things that don't affect the
primary use case and if you want to support this, you need to find a way
to do that does not negatively affect your typical router (without
external storage).

Regards,

Daniel

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[David Lang]

On Sat, 11 Jun 2016, Daniel Curran-Dickinson wrote:

> Hi Dave,
>
> I don't speak for the LEDE team, but it looks to me a lot of your
> problem is that you are using LEDE/openwrt for far bigger iron than the
> primary target (standard routers, including pre-AC non-NAND ones, which
> are really quite small and low powered).  2 TB+ storage for example, or
> using lighttpd instead of uhttpd are really things that don't affect the
> primary use case and if you want to support this, you need to find a way
> to do that does not negatively affect your typical router (without
> external storage).

While CeroWRT has expanded it's aim to be able to support today's faster network 
connections (up to and including the 1G connections now available), that's not 
really the issue here.

Even low-end devices now include a USB port, and it's really easy to plugi in an 
external USB drive that's >2TB. 3TB drives are now <$100

Now, if support for larger drives really does add a lot to the system footprint, 
it should be optional. But how much space are we talking about here? It should 
at least be an easy-to-select option.

David Lang

Re: [Cerowrt-devel] lede integration issues remaining from the detrius of cerowrt

[Alan Jenkins]

On 11/06/16 18:44, Dave Taht wrote:
> happy to see cake working today! thx all!
>
> In https://github.com/dtaht/ceropackages-3.10:
> The principal problem with fdisk nowadays is that very large (> 2TB, I
> think) devices are not supported by it, and require a GPT capable
> tool. Is there a replacement in lede that handles GPT? If not - this
> is an old gdisk port to openwrt that I used to use.
>

fdisk has support for creating GPT now (like all the other formats it 
can do).  Available in util-linux versions used on the desktop. I just 
checked Chaos Calmer, it's there in the fdisk package.

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Dirk Neukirchen]

On 11.06.2016 19:44, Dave Taht wrote:
> E) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/gdisk
> 
> The principal problem with fdisk nowadays is that very large (> 2TB, I
> think) devices are not supported by it, and require a GPT capable
> tool. Is there a replacement in lede that handles GPT? If not - this
> is an old gdisk port to openwrt that I used to use.

util-linux w. fdisk/cfdisk etc. should work - i just tested your case in a malta VM w. virtual 3TB qemu disk:

qemu-img create test.disk 3T
Formatting 'test.disk', fmt=raw size=3298534883328

qemu-system-mipsel -M malta -hda lede-malta-le-root.ext4 -hdb test.disk -kernel lede-malta-le-vmlinux.elf -nographic -append "root=/dev/sda console=ttyS0 debug ignore_loglevel loglevel=7 verbose "

root@(none):/# lsblk
[    9.429150] random: nonblocking pool is initialized
NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda         8:0    0   48M  0 disk /
sdb         8:16   0    3T  0 disk 
mtdblock0  31:0    0    1M  1 disk 
mtdblock1  31:1    0  2.9M  0 disk 
mtdblock2  31:2    0  128K  1 disk 

cfdisk /dev/sdb
i create 3 partitions there

root@lede:/# fdisk -l /dev/sdb
Disk /dev/sdb: 3 TiB, 3298534883328 bytes, 6442450944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5F5FA543-8EEB-47B1-A6D0-F0E24A4C544D

Device          Start        End    Sectors  Size Type
/dev/sdb1        2048 2147485695 2147483648    1T Linux filesystem
/dev/sdb2  2147485696 4294969343 2147483648    1T Linux filesystem
/dev/sdb3  4294969344 6442450910 2147481567 1024G Linux filesystem
root@lede:/# lsblk
NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda         8:0    0   48M  0 disk /
sdb         8:16   0    3T  0 disk 
├─sdb1      8:17   0    1T  0 part 
├─sdb2      8:18   0    1T  0 part 
└─sdb3      8:19   0 1024G  0 part 

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Dave Taht]

On Sat, Jun 11, 2016 at 3:07 PM, Lars Kruse <lists@sumpfralle.de> wrote:
> Hi Dave,
>
>> D) https://github.com/dtaht/ceropackages-3.10/tree/master/utils/nanom5poe
>>
>> I don't know what landed upstream to control poe for the nano-m5
>> radios, if anything?
>
> I submitted a patch (that was accepted) for GPIO-based POE control - at least
> for Nanostations XM/XW and for TP-Link CPE210 and CPE510:
>  https://git.lede-project.org/?p=source.git;a=commitdiff;h=d65916047b44d6d157d88d15e8e3d92555c5e6f8

Groovy. There are quite a few other devices that have POE (edgerouter
is the first that comes to mind), perhaps this can be built on
generically?

> Only the luci interface is missing ...

noted....

>
> Cheers,
> Lars
>
> _______________________________________________
> Lede-dev mailing list
> Lede-dev@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] lede integration issues remaining from the detrius of cerowrt

[Dave Taht]

On Sun, Jun 12, 2016 at 1:06 AM, Alan Jenkins
<alan.christopher.jenkins@gmail.com> wrote:
> On 11/06/16 18:44, Dave Taht wrote:
>>
>> happy to see cake working today! thx all!
>>
>> In https://github.com/dtaht/ceropackages-3.10:
>> The principal problem with fdisk nowadays is that very large (> 2TB, I
>> think) devices are not supported by it, and require a GPT capable
>> tool. Is there a replacement in lede that handles GPT? If not - this
>> is an old gdisk port to openwrt that I used to use.
>>
>
> fdisk has support for creating GPT now (like all the other formats it can
> do).  Available in util-linux versions used on the desktop. I just checked
> Chaos Calmer, it's there in the fdisk package.

Thank you for testing. It sounds like the
"https://wiki.openwrt.org/doc/howto/storage" is out of date if this is
true, and further, in the context of my question, the gdisk package
we'd had in cerowrt is no longer needed.

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Lars Kruse]

Hi Dave,


Am Sun, 12 Jun 2016 08:23:04 -0700
schrieb Dave Taht <dave.taht@gmail.com>:

> Groovy. There are quite a few other devices that have POE (edgerouter
> is the first that comes to mind), perhaps this can be built on
> generically?

here should be the proper place (it was changed after my patch):
 https://git.lede-project.org/?p=source.git;a=blob;f=target/linux/ar71xx/base-files/etc/board.d/03_gpio_switches

Cheers,
Lars

Re: [Cerowrt-devel] [LEDE-DEV] lede integration issues remaining from the detrius of cerowrt

[Dave Taht]

On Sat, Jun 11, 2016 at 11:11 AM, Daniel Curran-Dickinson
<daniel@daniel.thecshore.com> wrote:
> Hi Dave,
>
> I don't speak for the LEDE team, but it looks to me a lot of your
> problem is that you are using LEDE/openwrt for far bigger iron than the
> primary target (standard routers, including pre-AC non-NAND ones, which
> are really quite small and low powered).  2 TB+ storage for example, or
> using lighttpd instead of uhttpd are really things that don't affect the
> primary use case and if you want to support this, you need to find a way
> to do that does not negatively affect your typical router (without
> external storage).

The context of the original question was more about being able to
access large amounts of external storage easily at all and if an
optional package needed to go upstream or not. Subsequent emails
resolved that.

The meta questions are: defining what lede's use cases will be 5-10
years in the future.

...

CeroWrt targetted routers, starting 5 years ago, with 8-16MB flash and
wedged quite a lot into that. One core feature was shipping a regular
build with an operable gui, which I hope lede starts doing, in
addition to the basic build, for devices with sufficient flash, as it
lowers the basic barriers to entry for new users considerably.

I think that lede sits atop a shrinking space, where - while very
small routers will continue to exist -  it is getting hard to get
flash chips as small as 4Mbytes these days for new products, and there
are other spaces growing at a rapid rate, notably IoT.

Most of the new routers ship with tons more flash than that.

Also, the latest generations of hackerboards, the getchip.com one, for
example, costs 9 dollars and comes with 512MB of flash, with a default
OS of debian. Nearly all the hackerboards (rpi, odroids, etc) have
been debian based, which while that does have the innate advantage of
local compilation, lacks the industrial strength characteristics of
lede (smaller footprint, well integrated key/value store in uci and
the gui, almost never writing to local flash, a good firewall).

Ledes willingness to be on the bleeding edge means it will continue to
gain valuable new features more rapidly than anything else (the
original bufferbloat and ongoing make-wifi-fast work being my own
cases in point), but to some extent that's also a disadvantage as long
term stability has been tough to get.

Another area of growth (I hope) will be in the "distributed web" space
where we start sharing data across the edges of the internet better,
and where the most "always on" box - like a lede router - would be one
of the best places to host and route such replicated content. The
tools that are creating that world have thus far tended to be written
in higher level languages (ipfs is written in go, for example), and
yet the added cpu horsepower required to manipulate a blockchain or do
DNS via a DHT is somewhat trivial for the next generation of embedded
devices.

http://www.decentralizedweb.net/schedule/ was very inspiring.

http://www.nytimes.com/2016/06/08/technology/the-webs-creator-looks-to-reinvent-it.html?_r=0

I personally don't make much distinction between "host" and "router"
anymore - a lot of the cerowrt related work on hncp (hnetd), babeld,
and source specific routing tried to make (primarily ipv6) devices
reachable no matter where or what network (or vpn) they were on, and
everyone in the above spaces is doing it with DHTs, and trying to go
the next mile past Tor. The CCN folk even go so far as to redefine a
router always having a ton of local storage.

There will always be a need for small cost effective, extremely
reliable and long term stable, routing devices, with good (and soon to
be *great*)  wifi, admittedly.

> Regards,
>
> Daniel
>



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org