[Bloat] sch_fq pacing rate: Xbps, but at which layer in the network stack?

[Bloat] sch_fq pacing rate: Xbps, but at which layer in the network stack?

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 689 bytes --]

When I was testing with my iPerf changes, I realized that the sch_fq pacing
(which in iperf is set via setsockopt()), is pacing at a bandwidth that's
set at a pretty low level in the stack (which makes sense).  This is
different from the application pacing that iperf does (which is pacing the
goodput).

But it's not clear to me where the X bps determination is being made.  My
current guess is that it's at the interface level (since that's where
sch_fq is), and so it's approximately "bytes on the wire", minus preambles
and inter-packet spacing, and whatnot.  And so it's including all the 802.x
headers involved (vlan tags, qos tags, source/dest macs, etc).  Is this
correct?

-Aaron

[-- Attachment #2: Type: text/html, Size: 797 bytes --]

Re: [Bloat] sch_fq pacing rate: Xbps, but at which layer in the network stack?

[Eric Dumazet]

On Thu, 2017-04-13 at 20:12 -0700, Aaron Wood wrote:
> When I was testing with my iPerf changes, I realized that the sch_fq
> pacing (which in iperf is set via setsockopt()), is pacing at a
> bandwidth that's set at a pretty low level in the stack (which makes
> sense).  This is different from the application pacing that iperf does
> (which is pacing the goodput).
> 
> 
> But it's not clear to me where the X bps determination is being made.
> My current guess is that it's at the interface level (since that's
> where sch_fq is), and so it's approximately "bytes on the wire", minus
> preambles and inter-packet spacing, and whatnot.  And so it's
> including all the 802.x headers involved (vlan tags, qos tags,
> source/dest macs, etc).  Is this correct?
> 
Like other qdisc having rate limits (TBF, HTB ....), FQ sees packets
with all headers (including Ethernet one)

This is why the default quantum is 3028, which is exactly 2 regular
Ethernet frames (MTU=1500 + 14 bytes of Ethernet header)

If you have VLAN tag, it is generally not included in the calculation,
as many devices provide 'hardware tagging'.

Re: [Bloat] sch_fq pacing rate: Xbps, but at which layer in the network stack?

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 1571 bytes --]

Eric,

Thanks for the confirmation of that.

So application pacing of say 8Mbps (1000 packets at 1000 bytes each), is
equivalent to 8.3-8.4Mbps at the interface (depending on ip header options)

So anyone planning on calling setsockopt() needs to keep that in mind? Or
is that calculated correctly for an application?

-Aaron
On Fri, Apr 14, 2017 at 11:00 Eric Dumazet <eric.dumazet@gmail.com> wrote:

> On Thu, 2017-04-13 at 20:12 -0700, Aaron Wood wrote:
> > When I was testing with my iPerf changes, I realized that the sch_fq
> > pacing (which in iperf is set via setsockopt()), is pacing at a
> > bandwidth that's set at a pretty low level in the stack (which makes
> > sense).  This is different from the application pacing that iperf does
> > (which is pacing the goodput).
> >
> >
> > But it's not clear to me where the X bps determination is being made.
> > My current guess is that it's at the interface level (since that's
> > where sch_fq is), and so it's approximately "bytes on the wire", minus
> > preambles and inter-packet spacing, and whatnot.  And so it's
> > including all the 802.x headers involved (vlan tags, qos tags,
> > source/dest macs, etc).  Is this correct?
> >
> Like other qdisc having rate limits (TBF, HTB ....), FQ sees packets
> with all headers (including Ethernet one)
>
> This is why the default quantum is 3028, which is exactly 2 regular
> Ethernet frames (MTU=1500 + 14 bytes of Ethernet header)
>
> If you have VLAN tag, it is generally not included in the calculation,
> as many devices provide 'hardware tagging'.
>
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1996 bytes --]