hardware hacking on fq_codel in FPGA form at 10GigE

hardware hacking on fq_codel in FPGA form at 10GigE

[Dave Taht]

see: http://netfpga.org/

The quad gig-E version looks like it works well (in that there are
advanced things like DRR available for it) but the underlying FPGA is
obsolete. The quad 10GigE version looks sexy but not very functional
as yet. In either case, implementing {n,e,s}fq_codel onboard looks
very feasible, although something of a larger effort than I'd like to
spend personally. (it's been about 6 years since I last touched
verilog)

I would be interested however in getting a bunch of hardware hackers
together (and getting the academic discount on this board) to try and
do an implementation of a fq_codel derivative on the quad 10GigE
board. (or I would be happy to learn of some group already doing it)

The only thing that is seriously serial about fq_codel is shooting the
biggest flow when the queue limit is exceeded, and that could be made
embarrassingly parallel with enough gates.There are no doubt other
tricky issues.

I would like it if we could match the ns2 model entirely... down to
the last packet... in the real world. I have few contacts at stanford,
presently.

https://mailman.stanford.edu/pipermail/netfpga-announce/2012-October/000094.html

There are a couple of other alternatives in the GigE space and below
on openers....

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: hardware hacking on fq_codel in FPGA form at 10GigE

[Hal Murray]

If I was going to do something like that, I'd build a small/simple CPU and do 
the work in microcode.

> implementing {n,e,s}fq_codel onboard looks very feasible

How many lines of assembler code would it take?

How many registers do you need?  Do you need any memory other than queues?  
Maybe counters?


> The only thing that is seriously serial about fq_codel is shooting the
> biggest flow when the queue limit is exceeded, and that could be made
> embarrassingly parallel with enough gates.There are no doubt other tricky
> issues. 

Would it be better to do the fq work in the main CPU and let the FPGA grab 
packets from some shared  data structure in memory?  Can you work out a 
memory structure that doesn't need locks?


-- 
These are my opinions.  I hate spam.

Re: hardware hacking on fq_codel in FPGA form at 10GigE

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

A small CPU can be made in perhaps 35K gates - something like an ARM7TDMI
or a Cortex-M0. It is common to stick one of those in a special purpose
chip to help with control logic.

But that would operate at a few hundred MHz, which leaves only a few cycles
per packet for small packets. That's not enough to run even a relatively
simple algorithm like codel.

Dedicated logic that *is* fast enough to run the algorithm on each packet
shouldn't be any bigger than such a CPU.

- Jonathan Morton
 On Dec 20, 2012 10:17 AM, "Hal Murray" <hmurray@megapathdsl.net> wrote:

>
> If I was going to do something like that, I'd build a small/simple CPU and
> do
> the work in microcode.
>
> > implementing {n,e,s}fq_codel onboard looks very feasible
>
> How many lines of assembler code would it take?
>
> How many registers do you need?  Do you need any memory other than queues?
> Maybe counters?
>
>
> > The only thing that is seriously serial about fq_codel is shooting the
> > biggest flow when the queue limit is exceeded, and that could be made
> > embarrassingly parallel with enough gates.There are no doubt other tricky
> > issues.
>
> Would it be better to do the fq work in the main CPU and let the FPGA grab
> packets from some shared  data structure in memory?  Can you work out a
> memory structure that doesn't need locks?
>
>
> --
> These are my opinions.  I hate spam.
>
>
>
> _______________________________________________
> Bloat-devel mailing list
> Bloat-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat-devel
>

[-- Attachment #2: Type: text/html, Size: 2087 bytes --]

Re: hardware hacking on fq_codel in FPGA form at 10GigE

[Dave Taht]

On Thu, Dec 20, 2012 at 3:17 AM, Hal Murray <hmurray@megapathdsl.net> wrote:
>
> If I was going to do something like that, I'd build a small/simple CPU and do
> the work in microcode.

There are two ppc 440 cpus already onboard the 10GigE device, I think.
It's a REALLY NICE fpga.

http://netfpga.org/10G_specs.html

http://www.xilinx.com/support/documentation/data_sheets/ds100.pdf

If we really wanted to get a jump on the high end:

http://www.hitechglobal.com/boards/100gig.htm

>
>> implementing {n,e,s}fq_codel onboard looks very feasible
>
> How many lines of assembler code would it take?

I could do a dump of the current code into any given assembly
language. It's not a lot, but there are a lot of out of band
functions.

> How many registers do you need?  Do you need any memory other than queues?
> Maybe counters?

The total overhead for fq_codel is presently 1024*64 bytes for 1024
flows, and 4-8k of pointer overhead (32 or 64 bit). I would argue for
such a device to hash to 64k flows, or heck, higher. And the per-flow
overhead can be reduced a lot in a dedicated device.

As to what of that needs to be on-board the fpga or off-board, is a
fairly good question. The sfq/codel queue management stuff sits nicely
in parallel with getting the packets so that's an obvious second
bus/cache arch...

>> The only thing that is seriously serial about fq_codel is shooting the
>> biggest flow when the queue limit is exceeded, and that could be made
>> embarrassingly parallel with enough gates.There are no doubt other tricky
>> issues.
>
> Would it be better to do the fq work in the main CPU and let the FPGA grab

Well there are a few things that would benefit from moving directly
into hardware - the 5 tuple hash, for example.

> packets from some shared  data structure in memory?

The problem that I would like to beat is that TSO/GSO seem to be
necessary on the host processor to reduce the interrupt count to
sanity at 10GigE. A goal here would be to allow for TSO generation
(and GRO receive) to hand off to the board, but for the board to
interleave and aqm packets from there to the wire. Rather than a tx
descriptor ring you'd have a tx descriptor list and tx completion ring
so that you could send streams out of order.

> Can you work out a
> memory structure that doesn't need locks?

The enqueue and dequeue algorithms are entirely decoupled, with the
exception of this error handling phase of (out of queue space) One
thought would be to track packet count on enqueue (this is more
"sfq"-like than fq_codel-like) which still has a tiny lock...
:grumble:

>
>
> --
> These are my opinions.  I hate spam.
>
>
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: hardware hacking on fq_codel in FPGA form at 10GigE

[Dave Taht]

On Thu, Dec 20, 2012 at 4:13 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Thu, Dec 20, 2012 at 3:17 AM, Hal Murray <hmurray@megapathdsl.net> wrote:
>>
>> If I was going to do something like that, I'd build a small/simple CPU and do
>> the work in microcode.

I'd also looked at the octeon and the latest arm chipset from TI which
I can't remember the codename for at the moment...

http://www.cavium.com/OCTEON-II_CN68XX.html

Looks like cavium is going ARM with their next chip.

I'd certainly like to see an IP block for arm socs for this stuff too.
Given the huge number of offloads in the network path on most upcoming
SOCs, it's going to be hard to wedge queue management on top in
software. See, for example, the mindspeed C2000. So it seems like
finding an arm design house to get an IP block for fq_codel put
together is in order.

There are a couple interesting hybrid fpga/arm chips from places like st micro.

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: hardware hacking on fq_codel in FPGA form at 10GigE

[Hal Murray]

chromatix99@gmail.com said:
> A small CPU can be made in perhaps 35K gates - something like an ARM7TDMI or
> a Cortex-M0. It is common to stick one of those in a special purpose chip to
> help with control logic. 

I was thinking of a small/simple CPU with a wide instruction word to simplify 
instruction decoding.  We used to call it microcode rather than software.

My straw man would be an ALU that can add and compare and whatever else you 
need, an instruction field for loading various hardware control registers and 
another field for writing control registers, and other fields for 
setting/testing flags and such.  We used to put a next-instruction-address 
field so there was no ALU for the PC.  Branching was done by ORing bits into 
the PC.


> But that would operate at a few hundred MHz, which leaves only a few cycles
> per packet for small packets. That's not enough to run even a relatively
> simple algorithm like codel.

Sounds about right.

> Dedicated logic that *is* fast enough to run the algorithm on each packet
> shouldn't be any bigger than such a CPU.

You still need to describe the algorithm.  That's going to be some sort of 
FSM.  How many steps (clock cycles) will that take?

I think the key idea behind what I was trying to say is that as soon as the 
algorithm gets reasonably complicated, you probably want to think of it as 
software rather than hardware.

The sort of microcode I'm thinking of should be a reasonable way to describe 
that type of algorithm.  If it can't come close to what raw hardware could do 
then the design of the instruction set should be fixed.





-- 
These are my opinions.  I hate spam.

Re: hardware hacking on fq_codel in FPGA form at 10GigE

[Hal Murray]

dave.taht@gmail.com said:
>> If I was going to do something like that, I'd build a small/simple CPU
>> the work in microcode.

> There are two ppc 440 cpus already onboard the 10GigE device, I think. It's
> a REALLY NICE fpga. 

> I'd also looked at the octeon and the latest arm chipset from TI which I
> can't remember the codename for at the moment... 

I wasn't thinking of a traditional general purpose CPU but rather something 
special for this problem.



>> How many lines of assembler code would it take?

> I could do a dump of the current code into any given assembly language. It's
> not a lot, but there are a lot of out of band functions.

I didn't mean lines of traditional assembly code.  If we want to pursue this, 
pick a chunk of c code (not too big) and break it into "lines" where 
everything on a line can be executed at the same time.  I'll try to sketch a 
"CPU" and write the microcode.


> The enqueue and dequeue algorithms are entirely decoupled, with the
> exception of this error handling phase of (out of queue space) One thought
> would be to track packet count on enqueue (this is more "sfq"-like than
> fq_codel-like) which still has a tiny lock... 

Stuff that can be reasonably done in the driver should probably be done there 
if it saves a lot of work for the microcode.  Avoiding out-of-queue-space 
might be a good example.



> Well there are a few things that would benefit from moving directly into
> hardware - the 5 tuple hash, for example. 

I'm probably missing the big picture.  Are you building a router or a server?

A server has socket control blocks.  Can the hash be precomputed and stored 
there?
That doesn't help with UDP sendto, but I think it would work with TCP.


If you are building a router, does the routing as well as fq-ing have to fit 
in the FPGA?


-- 
These are my opinions.  I hate spam.

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 3876 bytes --]


I have lately been using (for my very wideband software defined radio amateur radio transceiver project) the brand new, very nice device called the Zynq 7000 series of Platform FPGA's from Xilinx.  It's a complete system on a chip, with a dual core ARM Cortex A9 and an enormous amount of programmable logic that has "cache coherent access" to the memory system.
 
The chip is fabricated in 28 nm form, has a "zillion" SelectIO programmable pins, but more importantly has a bunch of hard logic I/O paths.
 
Since it comes packaged "cheap" (full  6 inch square evalboard with 512 MB DRAM and full standard "PC type" interconnects - GigE, VGA, HDMI, USB), as a $299 board with free FPGA tool chain) and runs Linux out of the box, I highly recommend the board called Zedboard (just google that).
 
Easy to attach 10 GigE hardware if you can do simple PCB design and soldering.
 
You can be up and running with a development system for under $500 in a weekend, building FPGA acceleration, or if you want to add hardware that connects to the zillions of I/O pins to the PLL and memory system, that might take a week or more, depending on your hardware design and hacking skills. I've connected "eval boards" of various sorts using a breakout board you can buy from Xilinx quicker than that.
 
In some ways, this is the Raspberry Pi of high speed digital logic hacking.
 
 
 
 
-----Original Message-----
From: "Hal Murray" <hmurray@megapathdsl.net>
Sent: Thursday, December 20, 2012 5:32am
To: "Dave Taht" <dave.taht@gmail.com>
Cc: "bloat-devel" <bloat-devel@lists.bufferbloat.net>, codel@lists.bufferbloat.net, "Hal Murray" <hmurray@megapathdsl.net>, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE



dave.taht@gmail.com said:
>> If I was going to do something like that, I'd build a small/simple CPU
>> the work in microcode.

> There are two ppc 440 cpus already onboard the 10GigE device, I think. It's
> a REALLY NICE fpga. 

> I'd also looked at the octeon and the latest arm chipset from TI which I
> can't remember the codename for at the moment... 

I wasn't thinking of a traditional general purpose CPU but rather something 
special for this problem.



>> How many lines of assembler code would it take?

> I could do a dump of the current code into any given assembly language. It's
> not a lot, but there are a lot of out of band functions.

I didn't mean lines of traditional assembly code.  If we want to pursue this, 
pick a chunk of c code (not too big) and break it into "lines" where 
everything on a line can be executed at the same time.  I'll try to sketch a 
"CPU" and write the microcode.


> The enqueue and dequeue algorithms are entirely decoupled, with the
> exception of this error handling phase of (out of queue space) One thought
> would be to track packet count on enqueue (this is more "sfq"-like than
> fq_codel-like) which still has a tiny lock... 

Stuff that can be reasonably done in the driver should probably be done there 
if it saves a lot of work for the microcode.  Avoiding out-of-queue-space 
might be a good example.



> Well there are a few things that would benefit from moving directly into
> hardware - the 5 tuple hash, for example. 

I'm probably missing the big picture.  Are you building a router or a server?

A server has socket control blocks.  Can the hash be precomputed and stored 
there?
That doesn't help with UDP sendto, but I think it would work with TCP.


If you are building a router, does the routing as well as fq-ing have to fit 
in the FPGA?


-- 
These are my opinions.  I hate spam.



_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel

[-- Attachment #2: Type: text/html, Size: 4917 bytes --]

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[Dave Taht]

On Thu, Dec 20, 2012 at 8:53 AM,  <dpreed@reed.com> wrote:
> I have lately been using (for my very wideband software defined radio
> amateur radio transceiver project) the brand new, very nice device called
> the Zynq 7000 series of Platform FPGA's from Xilinx.  It's a complete system
> on a chip, with a dual core ARM Cortex A9 and an enormous amount of
> programmable logic that has "cache coherent access" to the memory system.
>
>
>
> The chip is fabricated in 28 nm form, has a "zillion" SelectIO programmable
> pins, but more importantly has a bunch of hard logic I/O paths.
>
>
>
> Since it comes packaged "cheap" (full  6 inch square evalboard with 512 MB
> DRAM and full standard "PC type" interconnects - GigE, VGA, HDMI, USB), as a
> $299 board with free FPGA tool chain) and runs Linux out of the box, I
> highly recommend the board called Zedboard (just google that).

Their documentation is first rate!

http://www.zedboard.org/sites/default/files/documentations/ZedBoard_HW_UG_v1_6.pdf

> Easy to attach 10 GigE hardware if you can do simple PCB design and
> soldering.

The 10GigE thought was mostly because that's what's driving most of
the decision making behind doing these huge network offloads on the
host processor. Which is so problematic when it migrates out of the
data center/talks to stuff outside of the datacenter

and/or the software concepts end up in devices that need to run well
at 100Mbit and below.

So thoroughly solving the fq/codel problem in hardware at that level
will head off the next worldwide problem when these devices become
more affordable. And (IMHO) make them work better.

> You can be up and running with a development system for under $500 in a
> weekend, building FPGA acceleration, or if you want to add hardware that
> connects to the zillions of I/O pins to the PLL and memory system, that
> might take a week or more, depending on your hardware design and hacking
> skills. I've connected "eval boards" of various sorts using a breakout board
> you can buy from Xilinx quicker than that.
>
>
>
> In some ways, this is the Raspberry Pi of high speed digital logic hacking.

You should give them that quote.

And I'll order one. Heck, maybe two. Real find! Thanks!

How far along is your SDR project?

>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: "Hal Murray" <hmurray@megapathdsl.net>
> Sent: Thursday, December 20, 2012 5:32am
> To: "Dave Taht" <dave.taht@gmail.com>
> Cc: "bloat-devel" <bloat-devel@lists.bufferbloat.net>,
> codel@lists.bufferbloat.net, "Hal Murray" <hmurray@megapathdsl.net>,
> cerowrt-devel@lists.bufferbloat.net
> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at
> 10GigE
>
>
> dave.taht@gmail.com said:
>>> If I was going to do something like that, I'd build a small/simple CPU
>>> the work in microcode.
>
>> There are two ppc 440 cpus already onboard the 10GigE device, I think.
>> It's
>> a REALLY NICE fpga.
>
>> I'd also looked at the octeon and the latest arm chipset from TI which I
>> can't remember the codename for at the moment...
>
> I wasn't thinking of a traditional general purpose CPU but rather something
> special for this problem.
>
>
>
>>> How many lines of assembler code would it take?
>
>> I could do a dump of the current code into any given assembly language.
>> It's
>> not a lot, but there are a lot of out of band functions.
>
> I didn't mean lines of traditional assembly code. If we want to pursue this,
> pick a chunk of c code (not too big) and break it into "lines" where
> everything on a line can be executed at the same time. I'll try to sketch a
> "CPU" and write the microcode.
>
>
>> The enqueue and dequeue algorithms are entirely decoupled, with the
>> exception of this error handling phase of (out of queue space) One thought
>> would be to track packet count on enqueue (this is more "sfq"-like than
>> fq_codel-like) which still has a tiny lock...
>
> Stuff that can be reasonably done in the driver should probably be done
> there
> if it saves a lot of work for the microcode. Avoiding out-of-queue-space
> might be a good example.
>
>
>
>> Well there are a few things that would benefit from moving directly into
>> hardware - the 5 tuple hash, for example.
>
> I'm probably missing the big picture. Are you building a router or a server?
>
> A server has socket control blocks. Can the hash be precomputed and stored
> there?
> That doesn't help with UDP sendto, but I think it would work with TCP.
>
>
> If you are building a router, does the routing as well as fq-ing have to fit
> in the FPGA?
>
>
> --
> These are my opinions. I hate spam.
>
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[Dave Taht]

OK, I ordered 2. It's not clear how the ethernet is fully implemented
(The marvel phy is documented, the actual ethernet interface is not so
far as I can see, so some source code  reading is going to be
required. )

I note that me doing this is also kind of driven by supplies of the
wndr3800 drying up. The prospect of having to do yet another port to
some other low-end hardware for cerowrt is depressing and mildly
intimidating. I'd rather do anything else than bring up yet another
board that will be obsolete in 6 months, when I'd rather be working at
a level far, far, above the hardware at this point. If I gotta do a
new board bring up, I'd rather switch to arm as I view MIPs as
basically dead (it's not dead, it's resting) at this point.

I also picked up a dreamplug today to BQL it. Not fond of that arm
implementation.

As for cero's future, well...

The wndr3700v4 is out, and appears to be a good hardware upgrade from
the 3800 series, but it's not supported by openwrt yet.

I took a look at their GPL source distribution. And yea! it's openwrt.
And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
2.64), and their kernel is 2.6.31.

I think the cpu and ethernet chips tho look a lot better: Atheros
AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
better.

I have looked at similar products from buffalo and others, would like
external antennas and a good switch this time around. Thoughts?

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[Dave Taht]

It really is astounding what is going on in the arm world:

http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451

> The wndr3700v4 is out, and appears to be a good hardware upgrade from
> the 3800 series, but it's not supported by openwrt yet.
>
> I took a look at their GPL source distribution. And yea! it's openwrt.
> And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
> 2.64), and their kernel is 2.6.31.
>
> I think the cpu and ethernet chips tho look a lot better: Atheros
> AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
> better.
>
> I have looked at similar products from buffalo and others, would like
> external antennas and a good switch this time around. Thoughts?
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]


And if you want an open tablet the ODROID U2 there is pretty nice for <$900 all in, including an HSPA+ modem w/GPS.   I can only imagine what one can do with this....
 
-----Original Message-----
From: "Dave Taht" <dave.taht@gmail.com>
Sent: Friday, December 21, 2012 7:34am
To: dpreed@reed.com
Cc: "Hal Murray" <hmurray@megapathdsl.net>, "bloat-devel" <bloat-devel@lists.bufferbloat.net>, codel@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE



It really is astounding what is going on in the arm world:

http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451

> The wndr3700v4 is out, and appears to be a good hardware upgrade from
> the 3800 series, but it's not supported by openwrt yet.
>
> I took a look at their GPL source distribution. And yea! it's openwrt.
> And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
> 2.64), and their kernel is 2.6.31.
>
> I think the cpu and ethernet chips tho look a lot better: Atheros
> AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
> better.
>
> I have looked at similar products from buffalo and others, would like
> external antennas and a good switch this time around. Thoughts?
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 1979 bytes --]

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 1989 bytes --]

On Fri, Dec 21, 2012 at 12:48 PM, <dpreed@reed.com> wrote:

> And if you want an open tablet the ODROID U2 there is pretty nice for
> <$900 all in, including an HSPA+ modem w/GPS.   I can only imagine what one
> can do with this....
>

Often the modem bits are binary blobs.  Any information on this front?
                    - Jim


>
>
> -----Original Message-----
> From: "Dave Taht" <dave.taht@gmail.com>
> Sent: Friday, December 21, 2012 7:34am
> To: dpreed@reed.com
> Cc: "Hal Murray" <hmurray@megapathdsl.net>, "bloat-devel" <
> bloat-devel@lists.bufferbloat.net>, codel@lists.bufferbloat.net,
> cerowrt-devel@lists.bufferbloat.net
> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at
> 10GigE
>
>  It really is astounding what is going on in the arm world:
>
>
> http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451
>
>
>
> > The wndr3700v4 is out, and appears to be a good hardware upgrade from
> > the 3800 series, but it's not supported by openwrt yet.
> >
> > I took a look at their GPL source distribution. And yea! it's openwrt.
> > And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
> > 2.64), and their kernel is 2.6.31.
> >
> > I think the cpu and ethernet chips tho look a lot better: Atheros
> > AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
> > better.
> >
> > I have looked at similar products from buffalo and others, would like
> > external antennas and a good switch this time around. Thoughts?
> >
> > --
> > Dave Täht
> >
> > Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
>
>
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
>
>
>
> _______________________________________________
> Bloat-devel mailing list
> Bloat-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat-devel
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 3871 bytes --]

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 3219 bytes --]


I'm sure the HSPA+ modem is a binary blob.  For regulatory reasons, if nothing else...
 
When I said open, I did not mean the ultimate ideal.  Hardware devices with firmware are often sold as black boxes.
 
Fixing the bloat in HSPA+ and LTE systems will require organizations like Ericsson, ALU and Cisco and Huawei to get their acts together and fix their own bugs.
 
-----Original Message-----
From: "Jim Gettys" <jg@freedesktop.org>
Sent: Friday, December 21, 2012 1:14pm
To: "David P Reed" <dpreed@reed.com>
Cc: "Dave Taht" <dave.taht@gmail.com>, "bloat-devel" <bloat-devel@lists.bufferbloat.net>, "cerowrt-devel@lists.bufferbloat.net" <cerowrt-devel@lists.bufferbloat.net>, "codel@lists.bufferbloat.net" <codel@lists.bufferbloat.net>
Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE








On Fri, Dec 21, 2012 at 12:48 PM,  <[mailto:dpreed@reed.com] dpreed@reed.com> wrote:

And if you want an open tablet the ODROID U2 there is pretty nice for <$900 all in, including an HSPA+ modem w/GPS.   I can only imagine what one can do with this....
Often the modem bits are binary blobs.  Any information on this front?
- Jim


 
-----Original Message-----
From: "Dave Taht" <[mailto:dave.taht@gmail.com] dave.taht@gmail.com>
Sent: Friday, December 21, 2012 7:34am
To: [mailto:dpreed@reed.com] dpreed@reed.com
 Cc: "Hal Murray" <[mailto:hmurray@megapathdsl.net] hmurray@megapathdsl.net>, "bloat-devel" <[mailto:bloat-devel@lists.bufferbloat.net] bloat-devel@lists.bufferbloat.net>, [mailto:codel@lists.bufferbloat.net] codel@lists.bufferbloat.net, [mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net
 Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE




It really is astounding what is going on in the arm world:

[http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451] http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451


> The wndr3700v4 is out, and appears to be a good hardware upgrade from
> the 3800 series, but it's not supported by openwrt yet.
>
 > I took a look at their GPL source distribution. And yea! it's openwrt.
> And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
> 2.64), and their kernel is 2.6.31.
>
> I think the cpu and ethernet chips tho look a lot better: Atheros
 > AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
> better.
>
> I have looked at similar products from buffalo and others, would like
> external antennas and a good switch this time around. Thoughts?
 >
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html




-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html

_______________________________________________
 Bloat-devel mailing list
[mailto:Bloat-devel@lists.bufferbloat.net] Bloat-devel@lists.bufferbloat.net
[https://lists.bufferbloat.net/listinfo/bloat-devel] https://lists.bufferbloat.net/listinfo/bloat-devel



[-- Attachment #2: Type: text/html, Size: 5304 bytes --]

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 4754 bytes --]

On Fri, Dec 21, 2012 at 1:25 PM, <dpreed@reed.com> wrote:

> I'm sure the HSPA+ modem is a binary blob.  For regulatory reasons, if
> nothing else...
>
>
>
> When I said open, I did not mean the ultimate ideal.  Hardware devices
> with firmware are often sold as black boxes.
>
>
>
> Fixing the bloat in HSPA+ and LTE systems will require organizations like
> Ericsson, ALU and Cisco and Huawei to get their acts together and fix their
> own bugs.
>

Actually, this needs probably/certainly needs help from the chip vendors as
well as the system vendors.  The system vendors (such as ALU) often get
their chips and drivers from those chip vendors.

I haven't looked into the HSPA+ or LTE systems; I am looking into DSL.  In
the DSL concrete example, Broadcom controls the bulk of the DSL chip
market, and (at least on the client home router end) provides a binary blob
Linux driver based on an antique Linux release (2.6.26, IIRC). The hardware
itself did not look like it had bad buffering, but the drivers....

I've been working with blogic (John Crispin) on debloating the Lantiq
driver, which we are trying to test; since it is entirely open source.  In
the Lantiq DSL driver case, the hardware sprouted exactly the same sort of
transmit/receive rings found in ethernet drivers, with the corresponding
bufferbloat due to device drivers being totally stupid.  Unfortunately, BQL
is ethernet only, and right now doesn't work properly at low bandwidths.

When John cut the transmit ring down to 2 packets, the latency dropped (at
the bandwidth he was testing at, .5Mbps) to around 60ms running fq_codel,
about what you would expect given the remaining buffers; we have to
automate the control of the transmit ring properly (since we probably do
need some help when running VDSL at 100Mbps).

I suspect/expect we'll find similar "features" when we dig into LTE and
HSPA+, though that is not certain.

Now, back to my saga of my recalcitrant water well, after which I can go
back to testing DSL the next trip to New Jersey (and flush the toilets and
have heat; I have geothermal heat...).
                         Sigh,
                                        - Jim



>
> -----Original Message-----
> From: "Jim Gettys" <jg@freedesktop.org>
> Sent: Friday, December 21, 2012 1:14pm
> To: "David P Reed" <dpreed@reed.com>
> Cc: "Dave Taht" <dave.taht@gmail.com>, "bloat-devel" <
> bloat-devel@lists.bufferbloat.net>, "cerowrt-devel@lists.bufferbloat.net"
> <cerowrt-devel@lists.bufferbloat.net>, "codel@lists.bufferbloat.net" <
> codel@lists.bufferbloat.net>
> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at
> 10GigE
>
>
>
>
> On Fri, Dec 21, 2012 at 12:48 PM, <dpreed@reed.com> wrote:
>
>> And if you want an open tablet the ODROID U2 there is pretty nice for
>> <$900 all in, including an HSPA+ modem w/GPS.   I can only imagine what one
>> can do with this....
>>
> Often the modem bits are binary blobs.  Any information on this front?
> - Jim
>
>>
>>
>> -----Original Message-----
>> From: "Dave Taht" <dave.taht@gmail.com>
>> Sent: Friday, December 21, 2012 7:34am
>> To: dpreed@reed.com
>> Cc: "Hal Murray" <hmurray@megapathdsl.net>, "bloat-devel" <
>> bloat-devel@lists.bufferbloat.net>, codel@lists.bufferbloat.net,
>> cerowrt-devel@lists.bufferbloat.net
>> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at
>> 10GigE
>>
>>   It really is astounding what is going on in the arm world:
>>
>>
>> http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451
>>
>>
>>
>>
>> > The wndr3700v4 is out, and appears to be a good hardware upgrade from
>> > the 3800 series, but it's not supported by openwrt yet.
>> >
>> > I took a look at their GPL source distribution. And yea! it's openwrt.
>> > And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
>> > 2.64), and their kernel is 2.6.31.
>> >
>> > I think the cpu and ethernet chips tho look a lot better: Atheros
>> > AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
>> > better.
>> >
>> > I have looked at similar products from buffalo and others, would like
>> > external antennas and a good switch this time around. Thoughts?
>> >
>> > --
>> > Dave Täht
>> >
>> > Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html
>>
>>
>>
>>
>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt:
>> http://www.teklibre.com/cerowrt/subscribe.html
>>
>>
>> _______________________________________________
>> Bloat-devel mailing list
>> Bloat-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 8558 bytes --]

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 6126 bytes --]


Understand about the chip vendors.  It is the case, though, that system vendors often have NDA's with the chip vendors ...  and they are the immediate customer of the chip vendors, so have some leverage to get their help resolving the issues.
 
I think the first step is doing the right measurements, to show that the chip is a bottleneck (as you are doing with Lantiq).
 
-----Original Message-----
From: "Jim Gettys" <jg@freedesktop.org>
Sent: Friday, December 21, 2012 1:45pm
To: "David P Reed" <dpreed@reed.com>
Cc: "Dave Taht" <dave.taht@gmail.com>, "bloat-devel" <bloat-devel@lists.bufferbloat.net>, "cerowrt-devel@lists.bufferbloat.net" <cerowrt-devel@lists.bufferbloat.net>, "codel@lists.bufferbloat.net" <codel@lists.bufferbloat.net>, blogic@openwrt.org
Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE








On Fri, Dec 21, 2012 at 1:25 PM,  <[mailto:dpreed@reed.com] dpreed@reed.com> wrote:

I'm sure the HSPA+ modem is a binary blob.  For regulatory reasons, if nothing else...
 
When I said open, I did not mean the ultimate ideal.  Hardware devices with firmware are often sold as black boxes.
 
Fixing the bloat in HSPA+ and LTE systems will require organizations like Ericsson, ALU and Cisco and Huawei to get their acts together and fix their own bugs.
Actually, this needs probably/certainly needs help from the chip vendors as well as the system vendors.  The system vendors (such as ALU) often get their chips and drivers from those chip vendors.

I haven't looked into the HSPA+ or LTE systems; I am looking into DSL.  In the DSL concrete example, Broadcom controls the bulk of the DSL chip market, and (at least on the client home router end) provides a binary blob Linux driver based on an antique Linux release (2.6.26, IIRC). The hardware itself did not look like it had bad buffering, but the drivers....
I've been working with blogic (John Crispin) on debloating the Lantiq driver, which we are trying to test; since it is entirely open source.  In the Lantiq DSL driver case, the hardware sprouted exactly the same sort of transmit/receive rings found in ethernet drivers, with the corresponding bufferbloat due to device drivers being totally stupid.  Unfortunately, BQL is ethernet only, and right now doesn't work properly at low bandwidths.
When John cut the transmit ring down to 2 packets, the latency dropped (at the bandwidth he was testing at, .5Mbps) to around 60ms running fq_codel, about what you would expect given the remaining buffers; we have to automate the control of the transmit ring properly (since we probably do need some help when running VDSL at 100Mbps).
I suspect/expect we'll find similar "features" when we dig into LTE and HSPA+, though that is not certain.
Now, back to my saga of my recalcitrant water well, after which I can go back to testing DSL the next trip to New Jersey (and flush the toilets and have heat; I have geothermal heat...).
Sigh,
- Jim


 
-----Original Message-----
From: "Jim Gettys" <[mailto:jg@freedesktop.org] jg@freedesktop.org>
Sent: Friday, December 21, 2012 1:14pm
To: "David P Reed" <[mailto:dpreed@reed.com] dpreed@reed.com>
 Cc: "Dave Taht" <[mailto:dave.taht@gmail.com] dave.taht@gmail.com>, "bloat-devel" <[mailto:bloat-devel@lists.bufferbloat.net] bloat-devel@lists.bufferbloat.net>, "[mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net" <[mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net>, "[mailto:codel@lists.bufferbloat.net] codel@lists.bufferbloat.net" <[mailto:codel@lists.bufferbloat.net] codel@lists.bufferbloat.net>
 Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE








On Fri, Dec 21, 2012 at 12:48 PM,  <[mailto:dpreed@reed.com] dpreed@reed.com> wrote:

And if you want an open tablet the ODROID U2 there is pretty nice for <$900 all in, including an HSPA+ modem w/GPS.   I can only imagine what one can do with this....
Often the modem bits are binary blobs.  Any information on this front?
- Jim

 
-----Original Message-----
From: "Dave Taht" <[mailto:dave.taht@gmail.com] dave.taht@gmail.com>
Sent: Friday, December 21, 2012 7:34am
 To: [mailto:dpreed@reed.com] dpreed@reed.com
 Cc: "Hal Murray" <[mailto:hmurray@megapathdsl.net] hmurray@megapathdsl.net>, "bloat-devel" <[mailto:bloat-devel@lists.bufferbloat.net] bloat-devel@lists.bufferbloat.net>, [mailto:codel@lists.bufferbloat.net] codel@lists.bufferbloat.net, [mailto:cerowrt-devel@lists.bufferbloat.net] cerowrt-devel@lists.bufferbloat.net
 Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE




It really is astounding what is going on in the arm world:

[http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451] http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451


> The wndr3700v4 is out, and appears to be a good hardware upgrade from
> the 3800 series, but it's not supported by openwrt yet.
>
 > I took a look at their GPL source distribution. And yea! it's openwrt.
 > And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
> 2.64), and their kernel is 2.6.31.
>
> I think the cpu and ethernet chips tho look a lot better: Atheros
 > AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
 > better.
>
> I have looked at similar products from buffalo and others, would like
> external antennas and a good switch this time around. Thoughts?
 >
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html




-- 
Dave Täht

Fixing bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklibre.com/cerowrt/subscribe.html

_______________________________________________
 Bloat-devel mailing list
[mailto:Bloat-devel@lists.bufferbloat.net] Bloat-devel@lists.bufferbloat.net
[https://lists.bufferbloat.net/listinfo/bloat-devel] https://lists.bufferbloat.net/listinfo/bloat-devel



[-- Attachment #2: Type: text/html, Size: 9514 bytes --]

Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 6272 bytes --]

On Fri, Dec 21, 2012 at 2:34 PM, <dpreed@reed.com> wrote:

> Understand about the chip vendors.  It is the case, though, that system
> vendors often have NDA's with the chip vendors ...  and they are the
> immediate customer of the chip vendors, so have some leverage to get their
> help resolving the issues.
>

Yup.  But it's hard when there are very few chip vendors supplying
alternatives, and when people do not understand bufferbloat in the first
place.

>
>
> I think the first step is doing the right measurements, to show that the
> chip is a bottleneck (as you are doing with Lantiq).
>

Exactly (thanks to John Chrispin!).  Data will help in that conversation
greatly.

The other motivation is to put fear of competition into moving the market
in the right direction...

Now back to seeing how the well people are doing with my well pipe and
pump....
                     - Jim


>
>
> -----Original Message-----
> From: "Jim Gettys" <jg@freedesktop.org>
> Sent: Friday, December 21, 2012 1:45pm
> To: "David P Reed" <dpreed@reed.com>
> Cc: "Dave Taht" <dave.taht@gmail.com>, "bloat-devel" <
> bloat-devel@lists.bufferbloat.net>, "cerowrt-devel@lists.bufferbloat.net"
> <cerowrt-devel@lists.bufferbloat.net>, "codel@lists.bufferbloat.net" <
> codel@lists.bufferbloat.net>, blogic@openwrt.org
> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at
> 10GigE
>
>
>
>
> On Fri, Dec 21, 2012 at 1:25 PM, <dpreed@reed.com> wrote:
>
>> I'm sure the HSPA+ modem is a binary blob.  For regulatory reasons, if
>> nothing else...
>>
>>
>>
>> When I said open, I did not mean the ultimate ideal.  Hardware devices
>> with firmware are often sold as black boxes.
>>
>>
>>
>> Fixing the bloat in HSPA+ and LTE systems will require organizations like
>> Ericsson, ALU and Cisco and Huawei to get their acts together and fix their
>> own bugs.
>>
> Actually, this needs probably/certainly needs help from the chip vendors
> as well as the system vendors.  The system vendors (such as ALU) often get
> their chips and drivers from those chip vendors.
>  I haven't looked into the HSPA+ or LTE systems; I am looking into DSL.
>  In the DSL concrete example, Broadcom controls the bulk of the DSL chip
> market, and (at least on the client home router end) provides a binary blob
> Linux driver based on an antique Linux release (2.6.26, IIRC). The hardware
> itself did not look like it had bad buffering, but the drivers....
> I've been working with blogic (John Crispin) on debloating the Lantiq
> driver, which we are trying to test; since it is entirely open source.  In
> the Lantiq DSL driver case, the hardware sprouted exactly the same sort of
> transmit/receive rings found in ethernet drivers, with the corresponding
> bufferbloat due to device drivers being totally stupid.  Unfortunately, BQL
> is ethernet only, and right now doesn't work properly at low bandwidths.
> When John cut the transmit ring down to 2 packets, the latency dropped (at
> the bandwidth he was testing at, .5Mbps) to around 60ms running fq_codel,
> about what you would expect given the remaining buffers; we have to
> automate the control of the transmit ring properly (since we probably do
> need some help when running VDSL at 100Mbps).
> I suspect/expect we'll find similar "features" when we dig into LTE and
> HSPA+, though that is not certain.
> Now, back to my saga of my recalcitrant water well, after which I can go
> back to testing DSL the next trip to New Jersey (and flush the toilets and
> have heat; I have geothermal heat...).
> Sigh,
> - Jim
>
>>
>>
>> -----Original Message-----
>> From: "Jim Gettys" <jg@freedesktop.org>
>> Sent: Friday, December 21, 2012 1:14pm
>> To: "David P Reed" <dpreed@reed.com>
>> Cc: "Dave Taht" <dave.taht@gmail.com>, "bloat-devel" <
>> bloat-devel@lists.bufferbloat.net>, "cerowrt-devel@lists.bufferbloat.net"
>> <cerowrt-devel@lists.bufferbloat.net>, "codel@lists.bufferbloat.net" <
>> codel@lists.bufferbloat.net>
>> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at
>> 10GigE
>>
>>
>>
>>
>> On Fri, Dec 21, 2012 at 12:48 PM, <dpreed@reed.com> wrote:
>>
>>> And if you want an open tablet the ODROID U2 there is pretty nice for
>>> <$900 all in, including an HSPA+ modem w/GPS.   I can only imagine what one
>>> can do with this....
>>>
>> Often the modem bits are binary blobs.  Any information on this front?
>> - Jim
>>
>>>
>>>
>>> -----Original Message-----
>>> From: "Dave Taht" <dave.taht@gmail.com>
>>> Sent: Friday, December 21, 2012 7:34am
>>> To: dpreed@reed.com
>>> Cc: "Hal Murray" <hmurray@megapathdsl.net>, "bloat-devel" <
>>> bloat-devel@lists.bufferbloat.net>, codel@lists.bufferbloat.net,
>>> cerowrt-devel@lists.bufferbloat.net
>>> Subject: Re: [Cerowrt-devel] hardware hacking on fq_codel in FPGA form
>>> at 10GigE
>>>
>>>   It really is astounding what is going on in the arm world:
>>>
>>>
>>> http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451
>>>
>>>
>>>
>>>
>>> > The wndr3700v4 is out, and appears to be a good hardware upgrade from
>>> > the 3800 series, but it's not supported by openwrt yet.
>>> >
>>> > I took a look at their GPL source distribution. And yea! it's openwrt.
>>> > And boo! it's ancient openwrt, for example dnsmasq is 2.39 (current is
>>> > 2.64), and their kernel is 2.6.31.
>>> >
>>> > I think the cpu and ethernet chips tho look a lot better: Atheros
>>> > AR9344+ AR9580(5GHz)+AR9344(2.4GHz). It's my hope these do ipv6
>>> > better.
>>> >
>>> > I have looked at similar products from buffalo and others, would like
>>> > external antennas and a good switch this time around. Thoughts?
>>> >
>>> > --
>>> > Dave Täht
>>> >
>>> > Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Dave Täht
>>>
>>> Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>>
>>>
>>> _______________________________________________
>>> Bloat-devel mailing list
>>> Bloat-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>>
>>>
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 11632 bytes --]

Re: [Codel] [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

On Fri, 2012-12-21 at 13:45 -0500, Jim Gettys wrote:
> 
> I've been working with blogic (John Crispin) on debloating the Lantiq
> driver, which we are trying to test; since it is entirely open source.  In
> the Lantiq DSL driver case, the hardware sprouted exactly the same sort of
> transmit/receive rings found in ethernet drivers, with the corresponding
> bufferbloat due to device drivers being totally stupid.  Unfortunately, BQL
> is ethernet only, and right now doesn't work properly at low bandwidths.

Hmmm. With the code that's currently in OpenWRT, we shouldn't *let* the
ATM driver have a huge queue of packets unless it's doing something
egregiously stupid like calling the vcc->pop() function on the skb
before it's actually sent it.

PPPoATM for a long time has limited the queue at the hardware driver to
2 packets, instead of a full sock_wmem's worth of buffers like it used
to in the past. Since a couple of weeks ago the BR2684 driver
(Ether-over-ATM, as used with PPPoE on top) also does the same thing.

So the ATM driver would have to go out of its way to screw you over¹.
John?

Btw, I'm looking at ways to *make* BQL on the PPP netdev work properly,
tracking packets all the way to the wire even when it's PPPoA,
PPPoE(oA), L2TP or whatever.

-- 
dwmw2

¹ Unless you're using IP over ATM directly. I haven't looked at that one.


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]

Re: [Codel] [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[John Crispin]

 > So the ATM driver would have to go out of its way to screw you over¹. 
John?

the lantiq atm/dsl drivers use a dma ring to push packets from the atm 
layer to the dsl modem core. this ring needs to be 2 packets big as a 
minimum to have the hardware work. however, using software we can make 
sure that only 1 packet at a tim eis inside the ring. this is what my 
current dirty hack does.

 > Btw, I'm looking at ways to *make* BQL on the PPP netdev work 
properly, tracking packets all the way to the wire even when it's PPPoA, 
PPPoE(oA), L2TP or whatever.

yay :)

Re: [Codel] [Cerowrt-devel] hardware hacking on fq_codel in FPGA form at 10GigE

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Fri, 2012-12-21 at 22:56 +0100, John Crispin wrote:
>  > So the ATM driver would have to go out of its way to screw you over¹. 
> John?
> 
> the lantiq atm/dsl drivers use a dma ring to push packets from the atm 
> layer to the dsl modem core. this ring needs to be 2 packets big as a 
> minimum to have the hardware work. however, using software we can make 
> sure that only 1 packet at a tim eis inside the ring. this is what my 
> current dirty hack does.

You shouldn't need that hack any more. With the current core ATM code
you should only have two packets in the queue anyway — the one the
hardware is *currently* working on, and the next one ready to be sent
(which is necessary to avoid introducing too much latency between
packets. Your ADSL uplink is *very* limited, and you really want to be
sure you're using it *all*).

-- 
dwmw2


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]