Re: [Bloat] sch_fq and sk->sk_pacing_rate

Re: [Bloat] sch_fq and sk->sk_pacing_rate

[Bill Ver Steeg (versteb)]

[-- Attachment #1: Type: text/plain, Size: 2355 bytes --]

Oops- I seem to have hit send too quickly. I think setsockopt(sockfd, SOL_SOCKET, SO_MAX_PACING_RATE, &val, sizeof(val)) is the call I am looking for. I was looking on an older kernel when I wrote the first note.

Comments still welcome on it use, though.

Bill VerSteeg


From: Bill Ver Steeg (versteb)
Sent: Monday, April 27, 2015 10:51 AM
To: bloat@lists.bufferbloat.net
Subject: sch_fq and sk->sk_pacing_rate

Bloaters (and particularly Eric)

I will try to not confuse the list, as this question is about host behaviors rather than middlebox behaviors. Having said that......

I am playing around with some application code that would benefit from pacing. The example I have in mind is MPEG-DASH style vide, but any type of ABR flow would probably benefit from the scheme I have in mind. I have done some work with client based pacing via RWND, but it is very clunky.

I am thinking of having the server tell the kernel the rate at which it wants data delivered to the client. (The client would probably tell the server via an HTTP pragma, but that is a separate can of worms....) So, the HTTP server would tell the kernel that it wants to deliver the TCP data at or below a certain max rate. There are currently hooks in sch_fq for the TCP layer to set sk_pacing_rate. It seems that I would need to add another pacing variable and have the data sent at the min of q->flow_max_rate, skb->sk->sk_pacing_rate, and this new app_pacing_rate. In other words, the old code


rate<http://lxr.free-electrons.com/ident?i=rate> = q->flow_max_rate;

if (skb<http://lxr.free-electrons.com/ident?i=skb>->sk)

   rate<http://lxr.free-electrons.com/ident?i=rate> = min<http://lxr.free-electrons.com/ident?i=min>(skb<http://lxr.free-electrons.com/ident?i=skb>->sk->sk_pacing_rate, rate<http://lxr.free-electrons.com/ident?i=rate>);

would become


rate<http://lxr.free-electrons.com/ident?i=rate> = q->flow_max_rate;

if (skb<http://lxr.free-electrons.com/ident?i=skb>->sk)

{

    rate<http://lxr.free-electrons.com/ident?i=rate> = min<http://lxr.free-electrons.com/ident?i=min>(skb<http://lxr.free-electrons.com/ident?i=skb>->sk->sk_pacing_rate, rate<http://lxr.free-electrons.com/ident?i=rate>);

    if (skb->sk->app_pacing_rate)

        rate = (min(skb->sk->app_pacing_rate, rate);

}


Thoughts?

Bill VerSteeg

[-- Attachment #2: Type: text/html, Size: 6835 bytes --]

Re: [Bloat] sch_fq and sk->sk_pacing_rate

[Steinar H. Gunderson]

On Mon, Apr 27, 2015 at 02:56:38PM +0000, Bill Ver Steeg (versteb) wrote:
> Oops- I seem to have hit send too quickly. I think setsockopt(sockfd,
> SOL_SOCKET, SO_MAX_PACING_RATE, &val, sizeof(val)) is the call I am looking
> for. I was looking on an older kernel when I wrote the first note.
> 
> Comments still welcome on it use, though.

cubemap uses it. It works great.

/* Steinar */
-- 
Homepage: http://www.sesse.net/

[Bloat] sch_fq and sk->sk_pacing_rate

[Bill Ver Steeg (versteb)]

[-- Attachment #1: Type: text/plain, Size: 1929 bytes --]

Bloaters (and particularly Eric)

I will try to not confuse the list, as this question is about host behaviors rather than middlebox behaviors. Having said that......

I am playing around with some application code that would benefit from pacing. The example I have in mind is MPEG-DASH style vide, but any type of ABR flow would probably benefit from the scheme I have in mind. I have done some work with client based pacing via RWND, but it is very clunky.

I am thinking of having the server tell the kernel the rate at which it wants data delivered to the client. (The client would probably tell the server via an HTTP pragma, but that is a separate can of worms....) So, the HTTP server would tell the kernel that it wants to deliver the TCP data at or below a certain max rate. There are currently hooks in sch_fq for the TCP layer to set sk_pacing_rate. It seems that I would need to add another pacing variable and have the data sent at the min of q->flow_max_rate, skb->sk->sk_pacing_rate, and this new app_pacing_rate. In other words, the old code


rate<http://lxr.free-electrons.com/ident?i=rate> = q->flow_max_rate;

if (skb<http://lxr.free-electrons.com/ident?i=skb>->sk)

   rate<http://lxr.free-electrons.com/ident?i=rate> = min<http://lxr.free-electrons.com/ident?i=min>(skb<http://lxr.free-electrons.com/ident?i=skb>->sk->sk_pacing_rate, rate<http://lxr.free-electrons.com/ident?i=rate>);

would become


rate<http://lxr.free-electrons.com/ident?i=rate> = q->flow_max_rate;

if (skb<http://lxr.free-electrons.com/ident?i=skb>->sk)

{

    rate<http://lxr.free-electrons.com/ident?i=rate> = min<http://lxr.free-electrons.com/ident?i=min>(skb<http://lxr.free-electrons.com/ident?i=skb>->sk->sk_pacing_rate, rate<http://lxr.free-electrons.com/ident?i=rate>);

    if (skb->sk->app_pacing_rate)

        rate = (min(skb->sk->app_pacing_rate, rate);

}


Thoughts?

Bill VerSteeg

[-- Attachment #2: Type: text/html, Size: 5183 bytes --]

Re: [Bloat] sch_fq and sk->sk_pacing_rate

[Eric Dumazet]

Hi Bill


I am confused, because it looks like this is already there in the kernel

Look for SO_MAX_PACING_RATE  setsockopt() ?

It works for all sockets, not only TCP ;)

http://git.kernel.org/cgit/linux/kernel/git/davem/net.git/commit/?id=62748f32d501f5d3712a7c372bbb92abc7c62bc7


Instead of doing the test in sch_fq (under potential qdisc lock),
its done in TCP (under socket spinlock, which is more scalable)


On Mon, 2015-04-27 at 14:49 +0000, Bill Ver Steeg (versteb) wrote:
> Bloaters (and particularly Eric)
> 
>  
> 
> I will try to not confuse the list, as this question is about host
> behaviors rather than middlebox behaviors. Having said that……
> 
>  
> 
> I am playing around with some application code that would benefit from
> pacing. The example I have in mind is MPEG-DASH style vide, but any
> type of ABR flow would probably benefit from the scheme I have in
> mind. I have done some work with client based pacing via RWND, but it
> is very clunky.
> 
>  
> 
> I am thinking of having the server tell the kernel the rate at which
> it wants data delivered to the client. (The client would probably tell
> the server via an HTTP pragma, but that is a separate can of worms….)
> So, the HTTP server would tell the kernel that it wants to deliver the
> TCP data at or below a certain max rate. There are currently hooks in
> sch_fq for the TCP layer to set sk_pacing_rate. It seems that I would
> need to add another pacing variable and have the data sent at the min
> of q->flow_max_rate, skb->sk->sk_pacing_rate, and this new
> app_pacing_rate. In other words, the old code
> 
>  
> 
> rate = q->flow_max_rate;
> if (skb->sk)
>    rate = min(skb->sk->sk_pacing_rate, rate);
> 
>  
> 
> would become
> 
>  
> 
> rate = q->flow_max_rate;
> if (skb->sk)
> {
>     rate = min(skb->sk->sk_pacing_rate, rate);
>     if (skb->sk->app_pacing_rate)
>         rate = (min(skb->sk->app_pacing_rate, rate);
> }       
> 
>  
> 
>  
> 
> Thoughts?
> 
>  
> 
> Bill VerSteeg
> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat