[Cerowrt-devel] packet capture hardware

[Cerowrt-devel] packet capture hardware

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 11765 bytes --]

Changing the subject line to reflect this line of discourse.

On Mon, Feb 4, 2013 at 8:41 AM, <dpreed@reed.com> wrote:

> I hadn't researched the HPC FMC requirement for 10 GigE one yet.
>
>
>
> The 1 GigE one is expensive, but not because of parts cost.  This is the
> usual huge markup that goes with stuff sold to "Design Engineers" in
> companies - because they can charge, they do.
>

Well, it is also a function of volume. as a counter example, we can
probably leverage an upcoming manufacturing run of one of atheros's newer
chipsets, designed close to a cerowrt-able, debloatable spec, for about 30
bucks in 10k qtys. This still sort of implies a change in cerowrt's focus
from "fixing hardware you can get off the shelf" to *making something*
arduino-raspberri pi like, but has a great deal of appeal for me.
(inspiration: meraki) I am sufficiently annoyed at the entire industry at
this point. I am insufficiently wealthy.

Anyway, that chipset probably isn't fast enough to do packet captures at
line rate, so to continue on the thread of "designing a good box for packet
captures" but sort of half retaining the cerowrt concept and wandering
around others, in this email....

I think there is a real market need for something in the SFP form factor
that can do high rate packet captures and other sorts of analysis. I
imagine a SFP in, and Esata out going into a router would be a useful
diagnostic tool (and also something the NSA would love, which I have
ambiguous feelings about)

It could also be priced appropriately and maybe make some money.

I think there is also a market need for something that can be an analysis
box/home router that can also do captures at typical rates in the home
(20-30Mbit), but that's still just above what a wndr3800 can do when last I
tried. (it's mostly bound by the usb interface actually)

The dreamplug hw can do that, as best as I recall (getting one shortly)


>
> The zedboard PMOD interface seems to be more marketing appropriate for
> "cheap" stuff.  There is a PMOD for 100baseT, so you could throw a few of
> those on your system very cheaply.   Since the interface to PMODs is 8-bit
> parallel, all you might need is the magnetics and PHY for GigE, and you
> could make a soft GigE controller in the programmable logic part of the
> Zynq-7020.
>

I'd certainly like to make an eth controller capable of handling TSO/UFO
and breaking them up with fq/codel at the lowest possible level. On the
other hand I'm pretty sure a dual core a9 box is fast enough to drive gigE
with minimal buffering (but haven't played with the zedboard enough to
know. I do know the driver isn't bql'd. It's on my todo list)

One of the things I'm vague about is the path to making silicon, starting
with a FPGA design like this. Say we solve the universe:

* Build a better wifi interface (and other forms of wireless interface)
* Do gigE switching/routing/rate limiting with fq/codel in hw
* Has adsl and/or cable modem functionality
* Earthquake detector (just throwing that in there! :) )

What's the path to cost reducing that to, say, 15 bucks a chip in 3 years?


> I'd have to check that the signalling rates would be sustainable across
> the PMOD connector.
>

100Mbit is enough for the "home gateway" scenario.


>
>
> To make an FMC board, populate it with whatever GigE chip you like, etc.
> is trivial.  It should cost no more to fabricate than one of these little
> single chip GigE PCIe cards you can buy.   What chip would you like to
> use?   I (or others) could design the board and BOM, kit it up for
> manufacturing (by, say, Sunstone or other places that do PC boards and
> kitted assembly in small runs).
>

I like the idea of a soft chip on the fpga myself, actually. I'd like to
get smarter logic inside the tx ring. I don't care for any of the current
generation of ethernet chips very much. The ar71xx in cero has the
advantage of being rather simple, the e1000e is a very common chip, too.
The realtek is terrible with tons of errata.

So to just use a phy... well, broadcom's common phys need a nda to look at,
so do marvel's. It would be interesting to pursue making a switch/router
actually out of a sufficient number of phys, if there is sufficient I/Os
available on the fpga. Something like the vyatta...

and with a soft eth design it could scale up to 10GigE or higher.


>
> Trivial stuff - maybe one could even convince Digilent and/or Avnet to do
> the design/mfring.
>

I would like to think that the latency advantage of making a debloated box
would convince some people, like wall street, and large scale buyers to get
involved. That said, I look at the hits on things like the water videos at
modena and the uphill battle with multiple manufacturers thus far and get
discouraged...


>
>
> Wouldn't it be a lot better to have a pluggable and completely flexible
> highly scalable monitoring unit that could go down the wire level as
> needed, with the base cost being the $300 that a Zedboard goes from?
>

It looks like the fpga chip itself is 220 presently. I am not sure how
rapidly that will drop with time or volume.

ooh, I see they have a milspec version (my hobby is space stuff)


>
>
> And it would be completely "open hardware" and :"open source".
>

I would so totally dig that. The number of VCs in my rolodex is rather
small.

I agree with you that the zedboard is "the raspberri pi of high speed
digital logic" and that a zillion things can/will be done with it. However
it's at a painful price point presently for most "normal" people. This is
an advantage, actually, given some of the target markets...

(I kind of hate it when I wear my business hat rather than my engineering
one)

I think the scope of designing a full fledged standalone zedboard-like
board,
one that fits into the home router role, or a packet capture role, or a SFP
slot,
is rather large, and would need a payoff at the end...

Even something on the scale of the netfpga project over at stanford (which
only saw about 2000 manufactured and huge uni support), will take time and
money. It would be very fun, and potentially profitable at the end, but as
a hobby project... the learning curve is steep, the skills required very
diverse. (yes, fun, yes needs a community to form around it)

(And cero as it stands eats way too much of my time and I really would like
to get someone else(s) building it so I can focus on more nagging issues up
the stack)

As for designing an add-on 100Mbit board to the zedboard, much easier. I'm
not huge on the PMOD connectors (fragile. Worse, the SD card sticks out the
side, and I already broke one zedboard's SD connector off), and a big
unknown is how fast they can be driven....




>
> -----Original Message-----
> From: "Dave Taht" <dave.taht@gmail.com>
> Sent: Sunday, February 3, 2013 8:47pm
> To: dpreed@reed.com
> Cc: "Mark Constable" <markc@renta.net>,
> cerowrt-devel@lists.bufferbloat.net
> Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab
>
>  Darn I wish I'd made it to that show today.
>
> On Sun, Feb 3, 2013 at 5:11 PM, <dpreed@reed.com> wrote:
>
>> http://www.prweb.com/releases/2012/2/prweb9154394.htm (10 GigE FMC card)
>>
>>
>>
>
> impressive. Seems to require a hpc (high pin count) board, which zed isn't.
>
>
>>
>> http://www.xilinx.com/products/boards-and-kits/1-2AJPAV.htm (1 GiGE FMC
>> card)
>>
>
> 625 eu. While I am painfully aware of how much it costs to step ahead of
> the bleeding edge, I think the odds are pointing harder and harder at doing
> a non-fpga design that does what I want...
>
> I may go back to looking at octeons or ti's new octeon killer.
>
> And/or leveraging a newer atheros reference board.
>
>
>>
>>
>> -----Original Message-----
>> From: "Dave Taht" <dave.taht@gmail.com>
>>  Sent: Sunday, February 3, 2013 1:39pm
>> To: dpreed@reed.com
>> Cc: "Mark Constable" <markc@renta.net>,
>> cerowrt-devel@lists.bufferbloat.net
>> Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab
>>
>>
>>
>> On Sun, Feb 3, 2013 at 10:26 AM, <dpreed@reed.com> wrote:
>>
>>> It would be trivial to do this with a Zedboard.
>>>
>>
>> Well, need two network ports. Haven't figured out much on interfacing the
>> thing to offboard gear (I'd have liked it if it had a pci interface). So is
>> interfacing up a second network card "trivial" on the I/Os provided?
>>
>> And wanted esata, or some high speed disk I/O interface for captures.
>>
>> I'd rather like to continue forward on the zedboard front. The prospect
>> of designing an ethernet chip that actually could incorporate fq_codel etc
>> is very exciting. The RGII interface is available to access directly, in
>> particular.
>>
>>
>>
>>
>>
>>>
>>> -----Original Message-----
>>> From: "Dave Taht" <dave.taht@gmail.com>
>>> Sent: Sunday, February 3, 2013 1:17pm
>>> To: "Mark Constable" <markc@renta.net>
>>> Cc: cerowrt-devel@lists.bufferbloat.net
>>> Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab
>>>
>>>  Well, I see it for 320. Then you need to add a SSD, and a decent
>>> network card, and I suppose it could be made to work. Awful big, tho, in an
>>> era where I can get 1/2TB on an 2.5 inch SSD.
>>>
>>> What I'd wanted was closer to a dreamplug - 160 bucks, two network
>>> ports, but with an internal SSD. bonus points if it fit into a 1U rack and
>>> ate as little power as possible.
>>>
>>> Principal use case here is to be a "network monitor" with enough oomph
>>> to run stuff like cacti/mrtg/snmp tools, as well as do captures off of a
>>> mirrored switch port.
>>>
>>>
>>>
>>> On Sun, Feb 3, 2013 at 10:10 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>>>
>>>>
>>>>  On Sun, Feb 3, 2013 at 10:03 AM, Mark Constable <markc@renta.net>wrote:
>>>>
>>>>> On 2013-02-03 09:18am, Dave Taht wrote:
>>>>> > I'm grumpy, as it doesn't have an esata interface internally,
>>>>> apparently.
>>>>>
>>>>> https://www.google.com?q=HP+N40L+MicroServer
>>>>>
>>>>> I know this is no where near an embedded device but I just got one of
>>>>> these
>>>>> on sale (new model out) for $220 and I think it's the most useful
>>>>> all-round
>>>>> cheap server box I've ever seen. Some people have it running 16 GB ram
>>>>> and
>>>>> I've got mine booting off an SSD via external eSATA. Very well built
>>>>> with 2
>>>>> x half height PCI slots (4 x eth port card?). Only missing USB3 ports
>>>>> and
>>>>> hot-swap drive space. And, very quiet with just an SSD.
>>>>>
>>>>
>>>> I'd be very interested to know how fast it could do packet header
>>>> captures.
>>>>
>>>> Line rate (gigE) would be good.
>>>>
>>>> Does it do BQL? (what is the onboard ethernet chips)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> Cerowrt-devel mailing list
>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Dave Täht
>>>>
>>>> Fixing bufferbloat with cerowrt:
>>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>>
>>>
>>>
>>>
>>> --
>>> Dave Täht
>>>
>>> Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>
>>
>>
>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html
>>
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 17808 bytes --]

Re: [Cerowrt-devel] packet capture hardware

[Michael Richardson]

>>>>> "Dave" == Dave Taht <dave.taht@gmail.com> writes:
    Dave> I think there is a real market need for something in the SFP
    Dave> form factor that can do high rate packet captures and other
    Dave> sorts of analysis. I imagine a SFP in, and Esata out going
    Dave> into a router would be a useful diagnostic tool (and also
    Dave> something the NSA would love, which I have ambiguous feelings
    Dave> about)

    Dave> It could also be priced appropriately and maybe make some
    Dave> money.

    Dave> I think there is also a market need for something that can be
    Dave> an analysis box/home router that can also do captures at
    Dave> typical rates in the home (20-30Mbit), but that's still just
    Dave> above what a wndr3800 can do when last I tried. (it's mostly
    Dave> bound by the usb interface actually)

For people doing *testing* rather than people doing long-haul packet
captures (e.g. the NSA, or
http://www.caida.org/projects/network_telescope/), you don't need to
capture for very long.

What I've wanted to put together, but I never get around to it, is a set
of bootable live CDs/TFTP images that you basically just run on a
machine with a pair of Gb/ethernet and 16-32Gbyte ram.  The NIC driver is
replaced with one that basically just uses all of available ram, and
when it's full, you stop capturing, and start either crunching or saving
to disk.

Stupid simple arithmetic you can capture 1Gb/s traffic for 32*8=256s
with 32G ram.  "Server" motherboards that go up to 48G ram are
relatively easy to acquire, and desktop ($99) motherboards systems that
can up to 16 or 24G are common.    I'd leave the machine on someone's
desk that travelled a lot if I was in an office...

The same system could play back traffic at speed.

-- 
]               Never tell me the odds!                 | ipv6 mesh networks [ 
]   Michael Richardson, Sandelman Software Works        | network architect  [ 
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [ 
	

Re: [Cerowrt-devel] packet capture hardware

[David Lang]

[-- Attachment #1: Type: TEXT/Plain, Size: 638 bytes --]

On Mon, 4 Feb 2013, Dave Taht wrote:

> I think there is also a market need for something that can be an analysis
> box/home router that can also do captures at typical rates in the home
> (20-30Mbit), but that's still just above what a wndr3800 can do when last I
> tried. (it's mostly bound by the usb interface actually)

the switch in the 3800 is pretty sophisticated, it can mirror ports as well as 
doign VLANs (although I haven't looked into how to do this through openwrt)

consider sending the data out through a dedicated port on the switch, then let 
people use a throw-away server/laptop to do the traffic capture.

David Lang

[-- Attachment #2: Type: TEXT/PLAIN, Size: 164 bytes --]

_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] packet capture hardware

[Guillaume Fortaine]

>> Wouldn't it be a lot better to have a pluggable and completely flexible
>> highly scalable monitoring unit that could go down the wire level as needed,
>> with the base cost being the $300 that a Zedboard goes from?
>
>
> It looks like the fpga chip itself is 220 presently. I am not sure how
> rapidly that will drop with time or volume.

There is also the Parallella Kit form Adapteva starting at $99 (with a
Zynq-7010 Dual-core ARM A9 CPU and an Epiphany Multicore Accelerator)
:

http://www.adapteva.com/products/eval-kits/parallella/


> Even something on the scale of the netfpga project over at stanford

There is also the Liberouter project at CESNET :

http://www.liberouter.org/




On Mon, Feb 4, 2013 at 6:48 PM, Dave Taht <dave.taht@gmail.com> wrote:
> Changing the subject line to reflect this line of discourse.
>
> On Mon, Feb 4, 2013 at 8:41 AM, <dpreed@reed.com> wrote:
>>
>> I hadn't researched the HPC FMC requirement for 10 GigE one yet.
>>
>>
>>
>> The 1 GigE one is expensive, but not because of parts cost.  This is the
>> usual huge markup that goes with stuff sold to "Design Engineers" in
>> companies - because they can charge, they do.
>
>
> Well, it is also a function of volume. as a counter example, we can probably
> leverage an upcoming manufacturing run of one of atheros's newer chipsets,
> designed close to a cerowrt-able, debloatable spec, for about 30 bucks in
> 10k qtys. This still sort of implies a change in cerowrt's focus from
> "fixing hardware you can get off the shelf" to *making something*
> arduino-raspberri pi like, but has a great deal of appeal for me.
> (inspiration: meraki) I am sufficiently annoyed at the entire industry at
> this point. I am insufficiently wealthy.
>
> Anyway, that chipset probably isn't fast enough to do packet captures at
> line rate, so to continue on the thread of "designing a good box for packet
> captures" but sort of half retaining the cerowrt concept and wandering
> around others, in this email....
>
> I think there is a real market need for something in the SFP form factor
> that can do high rate packet captures and other sorts of analysis. I imagine
> a SFP in, and Esata out going into a router would be a useful diagnostic
> tool (and also something the NSA would love, which I have ambiguous feelings
> about)
>
> It could also be priced appropriately and maybe make some money.
>
> I think there is also a market need for something that can be an analysis
> box/home router that can also do captures at typical rates in the home
> (20-30Mbit), but that's still just above what a wndr3800 can do when last I
> tried. (it's mostly bound by the usb interface actually)
>
> The dreamplug hw can do that, as best as I recall (getting one shortly)
>
>>
>>
>> The zedboard PMOD interface seems to be more marketing appropriate for
>> "cheap" stuff.  There is a PMOD for 100baseT, so you could throw a few of
>> those on your system very cheaply.   Since the interface to PMODs is 8-bit
>> parallel, all you might need is the magnetics and PHY for GigE, and you
>> could make a soft GigE controller in the programmable logic part of the
>> Zynq-7020.
>
>
> I'd certainly like to make an eth controller capable of handling TSO/UFO and
> breaking them up with fq/codel at the lowest possible level. On the other
> hand I'm pretty sure a dual core a9 box is fast enough to drive gigE with
> minimal buffering (but haven't played with the zedboard enough to know. I do
> know the driver isn't bql'd. It's on my todo list)
>
> One of the things I'm vague about is the path to making silicon, starting
> with a FPGA design like this. Say we solve the universe:
>
> * Build a better wifi interface (and other forms of wireless interface)
> * Do gigE switching/routing/rate limiting with fq/codel in hw
> * Has adsl and/or cable modem functionality
> * Earthquake detector (just throwing that in there! :) )
>
> What's the path to cost reducing that to, say, 15 bucks a chip in 3 years?
>
>>
>> I'd have to check that the signalling rates would be sustainable across
>> the PMOD connector.
>
>
> 100Mbit is enough for the "home gateway" scenario.
>
>>
>>
>>
>> To make an FMC board, populate it with whatever GigE chip you like, etc.
>> is trivial.  It should cost no more to fabricate than one of these little
>> single chip GigE PCIe cards you can buy.   What chip would you like to use?
>> I (or others) could design the board and BOM, kit it up for manufacturing
>> (by, say, Sunstone or other places that do PC boards and kitted assembly in
>> small runs).
>
>
> I like the idea of a soft chip on the fpga myself, actually. I'd like to get
> smarter logic inside the tx ring. I don't care for any of the current
> generation of ethernet chips very much. The ar71xx in cero has the advantage
> of being rather simple, the e1000e is a very common chip, too. The realtek
> is terrible with tons of errata.
>
> So to just use a phy... well, broadcom's common phys need a nda to look at,
> so do marvel's. It would be interesting to pursue making a switch/router
> actually out of a sufficient number of phys, if there is sufficient I/Os
> available on the fpga. Something like the vyatta...
>
> and with a soft eth design it could scale up to 10GigE or higher.
>
>>
>>
>> Trivial stuff - maybe one could even convince Digilent and/or Avnet to do
>> the design/mfring.
>
>
> I would like to think that the latency advantage of making a debloated box
> would convince some people, like wall street, and large scale buyers to get
> involved. That said, I look at the hits on things like the water videos at
> modena and the uphill battle with multiple manufacturers thus far and get
> discouraged...
>
>>
>>
>>
>> Wouldn't it be a lot better to have a pluggable and completely flexible
>> highly scalable monitoring unit that could go down the wire level as needed,
>> with the base cost being the $300 that a Zedboard goes from?
>
>
> It looks like the fpga chip itself is 220 presently. I am not sure how
> rapidly that will drop with time or volume.
>
> ooh, I see they have a milspec version (my hobby is space stuff)
>
>>
>>
>>
>> And it would be completely "open hardware" and :"open source".
>
>
> I would so totally dig that. The number of VCs in my rolodex is rather
> small.
>
> I agree with you that the zedboard is "the raspberri pi of high speed
> digital logic" and that a zillion things can/will be done with it. However
> it's at a painful price point presently for most "normal" people. This is an
> advantage, actually, given some of the target markets...
>
> (I kind of hate it when I wear my business hat rather than my engineering
> one)
>
> I think the scope of designing a full fledged standalone zedboard-like
> board,
> one that fits into the home router role, or a packet capture role, or a SFP
> slot,
> is rather large, and would need a payoff at the end...
>
> Even something on the scale of the netfpga project over at stanford (which
> only saw about 2000 manufactured and huge uni support), will take time and
> money. It would be very fun, and potentially profitable at the end, but as a
> hobby project... the learning curve is steep, the skills required very
> diverse. (yes, fun, yes needs a community to form around it)
>
> (And cero as it stands eats way too much of my time and I really would like
> to get someone else(s) building it so I can focus on more nagging issues up
> the stack)
>
> As for designing an add-on 100Mbit board to the zedboard, much easier. I'm
> not huge on the PMOD connectors (fragile. Worse, the SD card sticks out the
> side, and I already broke one zedboard's SD connector off), and a big
> unknown is how fast they can be driven....
>
>
>
>>
>>
>> -----Original Message-----
>> From: "Dave Taht" <dave.taht@gmail.com>
>> Sent: Sunday, February 3, 2013 8:47pm
>> To: dpreed@reed.com
>> Cc: "Mark Constable" <markc@renta.net>,
>> cerowrt-devel@lists.bufferbloat.net
>> Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab
>>
>> Darn I wish I'd made it to that show today.
>>
>> On Sun, Feb 3, 2013 at 5:11 PM, <dpreed@reed.com> wrote:
>>>
>>> http://www.prweb.com/releases/2012/2/prweb9154394.htm (10 GigE FMC card)
>>>
>>>
>>
>>
>> impressive. Seems to require a hpc (high pin count) board, which zed
>> isn't.
>>
>>>
>>>
>>> http://www.xilinx.com/products/boards-and-kits/1-2AJPAV.htm (1 GiGE FMC
>>> card)
>>
>>
>> 625 eu. While I am painfully aware of how much it costs to step ahead of
>> the bleeding edge, I think the odds are pointing harder and harder at doing
>> a non-fpga design that does what I want...
>>
>> I may go back to looking at octeons or ti's new octeon killer.
>>
>> And/or leveraging a newer atheros reference board.
>>
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: "Dave Taht" <dave.taht@gmail.com>
>>> Sent: Sunday, February 3, 2013 1:39pm
>>> To: dpreed@reed.com
>>> Cc: "Mark Constable" <markc@renta.net>,
>>> cerowrt-devel@lists.bufferbloat.net
>>> Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab
>>>
>>>
>>>
>>> On Sun, Feb 3, 2013 at 10:26 AM, <dpreed@reed.com> wrote:
>>>>
>>>> It would be trivial to do this with a Zedboard.
>>>
>>>
>>> Well, need two network ports. Haven't figured out much on interfacing the
>>> thing to offboard gear (I'd have liked it if it had a pci interface). So is
>>> interfacing up a second network card "trivial" on the I/Os provided?
>>>
>>> And wanted esata, or some high speed disk I/O interface for captures.
>>>
>>> I'd rather like to continue forward on the zedboard front. The prospect
>>> of designing an ethernet chip that actually could incorporate fq_codel etc
>>> is very exciting. The RGII interface is available to access directly, in
>>> particular.
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: "Dave Taht" <dave.taht@gmail.com>
>>>> Sent: Sunday, February 3, 2013 1:17pm
>>>> To: "Mark Constable" <markc@renta.net>
>>>> Cc: cerowrt-devel@lists.bufferbloat.net
>>>> Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab
>>>>
>>>> Well, I see it for 320. Then you need to add a SSD, and a decent network
>>>> card, and I suppose it could be made to work. Awful big, tho, in an era
>>>> where I can get 1/2TB on an 2.5 inch SSD.
>>>>
>>>> What I'd wanted was closer to a dreamplug - 160 bucks, two network
>>>> ports, but with an internal SSD. bonus points if it fit into a 1U rack and
>>>> ate as little power as possible.
>>>>
>>>> Principal use case here is to be a "network monitor" with enough oomph
>>>> to run stuff like cacti/mrtg/snmp tools, as well as do captures off of a
>>>> mirrored switch port.
>>>>
>>>>
>>>>
>>>> On Sun, Feb 3, 2013 at 10:10 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Feb 3, 2013 at 10:03 AM, Mark Constable <markc@renta.net>
>>>>> wrote:
>>>>>>
>>>>>> On 2013-02-03 09:18am, Dave Taht wrote:
>>>>>> > I'm grumpy, as it doesn't have an esata interface internally,
>>>>>> > apparently.
>>>>>>
>>>>>> https://www.google.com?q=HP+N40L+MicroServer
>>>>>>
>>>>>> I know this is no where near an embedded device but I just got one of
>>>>>> these
>>>>>> on sale (new model out) for $220 and I think it's the most useful
>>>>>> all-round
>>>>>> cheap server box I've ever seen. Some people have it running 16 GB ram
>>>>>> and
>>>>>> I've got mine booting off an SSD via external eSATA. Very well built
>>>>>> with 2
>>>>>> x half height PCI slots (4 x eth port card?). Only missing USB3 ports
>>>>>> and
>>>>>> hot-swap drive space. And, very quiet with just an SSD.
>>>>>
>>>>>
>>>>> I'd be very interested to know how fast it could do packet header
>>>>> captures.
>>>>>
>>>>> Line rate (gigE) would be good.
>>>>>
>>>>> Does it do BQL? (what is the onboard ethernet chips)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Cerowrt-devel mailing list
>>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Täht
>>>>>
>>>>> Fixing bufferbloat with cerowrt:
>>>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Dave Täht
>>>>
>>>> Fixing bufferbloat with cerowrt:
>>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>
>>>
>>>
>>>
>>> --
>>> Dave Täht
>>>
>>> Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>
>>
>>
>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html
>
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

Re: [Cerowrt-devel] packet capture hardware

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 14537 bytes --]


You can buy add-on 10/100 Pmods for $30 that would work on the zedboard from Digilent.  No need to design one.
 
Also, I just dug into the Zynq-7000 Tech Ref Manuals, and the Zedboard documentation.   The Zynq7020 on the Zedboard has two 10/100/1000 (GigE) controllers.  The board only has one external PHY (Broadcom).  But the "pins" of the other GigE controller are connected to the PL (Programmable Logic) and can be routed as RGMII or GMII signals (even tapped by PL along the way) to SelectI/O pins on Pmod or FMC interfaces.   So if you want just one more 1 GigE port, you just have to make a tiny board that holds one PHY chip of your choice. I can probably arrange to have a couple hundred made in Ireland for almost nothing per board.   One of my buddies here in Boston does a lot of small hardware boards for medical electronics, and is partnered with a PCB maker that is very inexpensive for small runs of simple boards.
 
For prototyping for a small group, one could make a "single board" a few inches across and slice it into maybe 10 boards with a single FMC connector, a GigE PHY and RJ45 as the only pieces.  The FMC would be on one side, and the other side would be the PHY and RJ45.  the "single board" would be maybe $125 bucks plus kit, quantity one.  That would give you 10 adapters for under $20 each, all in.
 
Now the idea of going from FPGA to ASIC is not really that interesting - I'm much more interested in hobbyist or prosumer network debugging stuff.   Yeah, the quantity one cost of the Zynq7020 is high (as are most FPGAs).  I've never talked to Avnet or Digilent about whether they'd be interested in this sort of thing.   At the Media Lab and CSAIL, all the gear needed to assemble short runs with tape-and-reel parts and reflow soldering are pretty available if its for a good cause.
 
I also probably could interest Vanu Bose (his company does design/manufacturing in India for his SDR products) in maybe helping, if the project involves perhaps extensibility to debugging various cellular networking deployments, etc.  He is selling a lot of cellular data gear for the Indian rural market, at very low costs compared to the high cost of non-SDR stuff.
 
 
-----Original Message-----
From: "Dave Taht" <dave.taht@gmail.com>
Sent: Monday, February 4, 2013 12:48pm
To: dpreed@reed.com
Cc: "Mark Constable" <markc@renta.net>, cerowrt-devel@lists.bufferbloat.net
Subject: packet capture hardware



Changing the subject line to reflect this line of discourse.


On Mon, Feb 4, 2013 at 8:41 AM,  <[mailto:dpreed@reed.com] dpreed@reed.com> wrote:

I hadn't researched the HPC FMC requirement for 10 GigE one yet.
 
The 1 GigE one is expensive, but not because of parts cost.  This is the usual huge markup that goes with stuff sold to "Design Engineers" in companies - because they can charge, they do.

Well, it is also a function of volume. as a counter example, we can probably leverage an upcoming manufacturing run of one of atheros's newer chipsets, designed close to a cerowrt-able, debloatable spec, for about 30 bucks in 10k qtys. This still sort of implies a change in cerowrt's focus from "fixing hardware you can get off the shelf" to *making something* arduino-raspberri pi like, but has a great deal of appeal for me. (inspiration: meraki) I am sufficiently annoyed at the entire industry at this point. I am insufficiently wealthy. 

Anyway, that chipset probably isn't fast enough to do packet captures at line rate, so to continue on the thread of "designing a good box for packet captures" but sort of half retaining the cerowrt concept and wandering around others, in this email....

I think there is a real market need for something in the SFP form factor that can do high rate packet captures and other sorts of analysis. I imagine a SFP in, and Esata out going into a router would be a useful diagnostic tool (and also something the NSA would love, which I have ambiguous feelings about)

It could also be priced appropriately and maybe make some money.
 
I think there is also a market need for something that can be an analysis box/home router that can also do captures at typical rates in the home (20-30Mbit), but that's still just above what a wndr3800 can do when last I tried. (it's mostly bound by the usb interface actually)

The dreamplug hw can do that, as best as I recall (getting one shortly)


 
The zedboard PMOD interface seems to be more marketing appropriate for "cheap" stuff.  There is a PMOD for 100baseT, so you could throw a few of those on your system very cheaply.   Since the interface to PMODs is 8-bit parallel, all you might need is the magnetics and PHY for GigE, and you could make a soft GigE controller in the programmable logic part of the Zynq-7020.

I'd certainly like to make an eth controller capable of handling TSO/UFO and breaking them up with fq/codel at the lowest possible level. On the other hand I'm pretty sure a dual core a9 box is fast enough to drive gigE with minimal buffering (but haven't played with the zedboard enough to know. I do know the driver isn't bql'd. It's on my todo list)

One of the things I'm vague about is the path to making silicon, starting with a FPGA design like this. Say we solve the universe:

* Build a better wifi interface (and other forms of wireless interface)
 * Do gigE switching/routing/rate limiting with fq/codel in hw
* Has adsl and/or cable modem functionality
* Earthquake detector (just throwing that in there! :) )

What's the path to cost reducing that to, say, 15 bucks a chip in 3 years?
I'd have to check that the signalling rates would be sustainable across the PMOD connector.

100Mbit is enough for the "home gateway" scenario.
 

 
To make an FMC board, populate it with whatever GigE chip you like, etc. is trivial.  It should cost no more to fabricate than one of these little single chip GigE PCIe cards you can buy.   What chip would you like to use?   I (or others) could design the board and BOM, kit it up for manufacturing (by, say, Sunstone or other places that do PC boards and kitted assembly in small runs).

I like the idea of a soft chip on the fpga myself, actually. I'd like to get smarter logic inside the tx ring. I don't care for any of the current generation of ethernet chips very much. The ar71xx in cero has the advantage of being rather simple, the e1000e is a very common chip, too. The realtek is terrible with tons of errata.

So to just use a phy... well, broadcom's common phys need a nda to look at, so do marvel's. It would be interesting to pursue making a switch/router actually out of a sufficient number of phys, if there is sufficient I/Os available on the fpga. Something like the vyatta...

and with a soft eth design it could scale up to 10GigE or higher.


 
Trivial stuff - maybe one could even convince Digilent and/or Avnet to do the design/mfring.

I would like to think that the latency advantage of making a debloated box would convince some people, like wall street, and large scale buyers to get involved. That said, I look at the hits on things like the water videos at modena and the uphill battle with multiple manufacturers thus far and get discouraged... 
 

 
Wouldn't it be a lot better to have a pluggable and completely flexible highly scalable monitoring unit that could go down the wire level as needed, with the base cost being the $300 that a Zedboard goes from?

It looks like the fpga chip itself is 220 presently. I am not sure how rapidly that will drop with time or volume.

ooh, I see they have a milspec version (my hobby is space stuff)
 

 
And it would be completely "open hardware" and :"open source".

I would so totally dig that. The number of VCs in my rolodex is rather small. 

I agree with you that the zedboard is "the raspberri pi of high speed digital logic" and that a zillion things can/will be done with it. However it's at a painful price point presently for most "normal" people. This is an advantage, actually, given some of the target markets...

(I kind of hate it when I wear my business hat rather than my engineering one)

I think the scope of designing a full fledged standalone zedboard-like board, 
one that fits into the home router role, or a packet capture role, or a SFP slot,
 is rather large, and would need a payoff at the end...

Even something on the scale of the netfpga project over at stanford (which only saw about 2000 manufactured and huge uni support), will take time and money. It would be very fun, and potentially profitable at the end, but as a hobby project... the learning curve is steep, the skills required very diverse. (yes, fun, yes needs a community to form around it)

(And cero as it stands eats way too much of my time and I really would like to get someone else(s) building it so I can focus on more nagging issues up the stack)

As for designing an add-on 100Mbit board to the zedboard, much easier. I'm not huge on the PMOD connectors (fragile. Worse, the SD card sticks out the side, and I already broke one zedboard's SD connector off), and a big unknown is how fast they can be driven....



 
 
-----Original Message-----
From: "Dave Taht" <[mailto:dave.taht@gmail.com] dave.taht@gmail.com>

Sent: Sunday, February 3, 2013 8:47pm
To: [mailto:dpreed@reed.com] dpreed@reed.com
Cc: "Mark Constable" <[mailto:markc@renta.net] markc@renta.net>, [mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net
 Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab





Darn I wish I'd made it to that show today.


On Sun, Feb 3, 2013 at 5:11 PM,  <[mailto:dpreed@reed.com] dpreed@reed.com> wrote:

[http://www.prweb.com/releases/2012/2/prweb9154394.htm] http://www.prweb.com/releases/2012/2/prweb9154394.htm (10 GigE FMC card)
 

 impressive. Seems to require a hpc (high pin count) board, which zed isn't.


 
[http://www.xilinx.com/products/boards-and-kits/1-2AJPAV.htm] http://www.xilinx.com/products/boards-and-kits/1-2AJPAV.htm (1 GiGE FMC card)

625 eu. While I am painfully aware of how much it costs to step ahead of the bleeding edge, I think the odds are pointing harder and harder at doing a non-fpga design that does what I want...

I may go back to looking at octeons or ti's new octeon killer.

And/or leveraging a newer atheros reference board.


 
 
-----Original Message-----
From: "Dave Taht" <[mailto:dave.taht@gmail.com] dave.taht@gmail.com>

Sent: Sunday, February 3, 2013 1:39pm
To: [mailto:dpreed@reed.com] dpreed@reed.com
Cc: "Mark Constable" <[mailto:markc@renta.net] markc@renta.net>, [mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net
 Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab







On Sun, Feb 3, 2013 at 10:26 AM,  <[mailto:dpreed@reed.com] dpreed@reed.com> wrote:

It would be trivial to do this with a Zedboard.

Well, need two network ports. Haven't figured out much on interfacing the thing to offboard gear (I'd have liked it if it had a pci interface). So is interfacing up a second network card "trivial" on the I/Os provided?

And wanted esata, or some high speed disk I/O interface for captures.

I'd rather like to continue forward on the zedboard front. The prospect of designing an ethernet chip that actually could incorporate fq_codel etc is very exciting. The RGII interface is available to access directly, in particular.







 
-----Original Message-----
From: "Dave Taht" <[mailto:dave.taht@gmail.com] dave.taht@gmail.com>
Sent: Sunday, February 3, 2013 1:17pm
 To: "Mark Constable" <[mailto:markc@renta.net] markc@renta.net>
 Cc: [mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net
 Subject: Re: [Cerowrt-devel] stanford talk/deluged in hardware/yurtlab



Well, I see it for 320. Then you need to add a SSD, and a decent network card, and I suppose it could be made to work. Awful big, tho, in an era where I can get 1/2TB on an 2.5 inch SSD.

What I'd wanted was closer to a dreamplug - 160 bucks, two network ports, but with an internal SSD. bonus points if it fit into a 1U rack and ate as little power as possible.

Principal use case here is to be a "network monitor" with enough oomph to run stuff like cacti/mrtg/snmp tools, as well as do captures off of a mirrored switch port.




On Sun, Feb 3, 2013 at 10:10 AM, Dave Taht <[mailto:dave.taht@gmail.com] dave.taht@gmail.com> wrote:




On Sun, Feb 3, 2013 at 10:03 AM, Mark Constable <[mailto:markc@renta.net] markc@renta.net> wrote:

On 2013-02-03 09:18am, Dave Taht wrote:
 > I'm grumpy, as it doesn't have an esata interface internally, apparently.

[https://www.google.com?q=HP+N40L+MicroServer] https://www.google.com?q=HP+N40L+MicroServer

 I know this is no where near an embedded device but I just got one of these
 on sale (new model out) for $220 and I think it's the most useful all-round
 cheap server box I've ever seen. Some people have it running 16 GB ram and
 I've got mine booting off an SSD via external eSATA. Very well built with 2
 x half height PCI slots (4 x eth port card?). Only missing USB3 ports and
 hot-swap drive space. And, very quiet with just an SSD.


I'd be very interested to know how fast it could do packet header captures.

Line rate (gigE) would be good. 

Does it do BQL? (what is the onboard ethernet chips)






 _______________________________________________
 Cerowrt-devel mailing list
[mailto:Cerowrt-devel@lists.bufferbloat.net] Cerowrt-devel@lists.bufferbloat.net
[https://lists.bufferbloat.net/listinfo/cerowrt-devel] https://lists.bufferbloat.net/listinfo/cerowrt-devel





-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html


-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html


-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html


-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html


-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 21725 bytes --]

Re: [Cerowrt-devel] packet capture hardware

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]


Cool.  I like this cheap little switch that can mirror ports.  Can one acquire it independent of the 3800?
 
And is there support in OpenWRT's driver set?
 
-----Original Message-----
From: "David Lang" <david@lang.hm>
Sent: Monday, February 4, 2013 4:20pm
To: "Dave Taht" <dave.taht@gmail.com>
Cc: dpreed@reed.com, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] packet capture hardware



On Mon, 4 Feb 2013, Dave Taht wrote:

> I think there is also a market need for something that can be an analysis
> box/home router that can also do captures at typical rates in the home
> (20-30Mbit), but that's still just above what a wndr3800 can do when last I
> tried. (it's mostly bound by the usb interface actually)

the switch in the 3800 is pretty sophisticated, it can mirror ports as well as 
doign VLANs (although I haven't looked into how to do this through openwrt)

consider sending the data out through a dedicated port on the switch, then let 
people use a throw-away server/laptop to do the traffic capture.

David Lang_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel

[-- Attachment #2: Type: text/html, Size: 1668 bytes --]