From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vb0-f43.google.com (mail-vb0-f43.google.com [209.85.212.43]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id A14C921F115 for ; Fri, 28 Dec 2012 06:36:22 -0800 (PST) Received: by mail-vb0-f43.google.com with SMTP id fs19so10695678vbb.16 for ; Fri, 28 Dec 2012 06:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=Tqn1y0aMcvJfpZ1uDFQQU977MGvwRp6zEgmo1iQNuz8=; b=LiK1QwOB00UD1kwyqnl0vCOoYLEw8atajxo8zamid2UHKm/Egz1Pf8JnVfCDpjDVr7 3rZAObSBCCxmBpTysapoXeezAydJq8p+8Iha7HGMVDmQK7lSaVtFj9IXP1ax+B4h6wnx IRl1q0ZRiHvGB42+Adx5NGkAJCxHbhPyAg9Pv8Q1jvxdjCRjyahIeYlT3EDQdtnVQp5v bw/kd0WFG6VetEvyLf850N9Khv524rPqxaNy61RsBiG3UUinIXCKDcuJpGk/34hYN6/i PI2sc7viQuLSw2rplKPc0zy/gHVOXaHPKNY0TjiPzMTAAqrwXqBjKXilDt5rk3JI89OB N2pw== Received: by 10.52.89.106 with SMTP id bn10mr45200291vdb.68.1356705381087; Fri, 28 Dec 2012 06:36:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.59.8.130 with HTTP; Fri, 28 Dec 2012 06:36:04 -0800 (PST) From: Jesper Louis Andersen Date: Fri, 28 Dec 2012 15:36:04 +0100 Message-ID: To: codel@lists.bufferbloat.net Content-Type: multipart/alternative; boundary=20cf307f3a564dfb1d04d1ea9683 Subject: [Codel] Another use of CoDel in an Erlang queue system. X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2012 14:36:23 -0000 --20cf307f3a564dfb1d04d1ea9683 Content-Type: text/plain; charset=UTF-8 Hi CoDel list. I have implemented the CoDel algorithm in Erlang for a subsystem I am writing. It is one of the "abuse one algorithm in another setting" cases. Here is the implementation, all 200 lines including comments: https://github.com/jlouis/safetyvalve/blob/master/src/sv_codel.erl One advantage of Erlang is that a lot of the if-maze in the ns2 codel patch can be reduced to a single pattern match here. The "safetyvalve" application has a queueing purpose in an Erlang application. Requests arrive at the Erlang system in a rather controlled fashion of Poissonesque proportions (but it isn't Poisson in practice though). In Erlang, we spawn a new (ultra-light-weight) process per incoming request. The problem is that we have limited resources in the system and we want to protect base service. As soon as a small process begin doing work, it may use a lot of resources, especially memory. We may know the system can at most handle 800 simultaneous requests of a given type. Or that the total amount of connections can't go above 5000. You could just stop providing service when the service ceiling was hit, but this would not allow sudden "spikes" of traffic. This is exactly why we need a queue of incoming requests to "smooth" out request spikes and handle them over time. The problem, however, is that the queue I use is a tail-drop queue. And I have no idea of the sojourn time through the queue. My fear is that the tail-drop behavior will just incur more latency on the tasks that do get through the queue. Hence the attempt at using CoDel for this. Preliminary tests show that it does look like it performs the task it should. When the Target is not met for Interval, we begin dropping tasks out of the queue, like we should. More experiments are needed however, before I know if this works well. One thing which is an important difference is that there are no TCP-behaviour where the window will close down upon dropped packets. But CoDel *does* allow me to provide early feedback that the Target can't really be met. For real-time requests, this is actualy the desired behavior. The next plan is to provide a TCP-like window feature for batch processing of jobs. The idea is that you can have a large list of jobs that needs to be run and have the system be self-tuning. TCP congestion control works, so why not apply it in a new setting :) -- J. --20cf307f3a564dfb1d04d1ea9683 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi CoDel list.

I have implemented the C= oDel algorithm in Erlang for a subsystem I am writing. It is one of the &qu= ot;abuse one algorithm in another setting" cases. Here is the implemen= tation, all 200 lines including comments:


One advantage of Erlang = is that a lot of the if-maze in the ns2 codel patch can be reduced to a sin= gle pattern match here.

The "safetyvalve" application has a que= ueing purpose in an Erlang application. Requests arrive at the Erlang syste= m in a rather controlled fashion of Poissonesque proportions (but it isn= 9;t Poisson in practice though). In Erlang, we spawn a new (ultra-light-wei= ght) process per incoming request. The problem is that we have limited reso= urces in the system
and we want to protect base service. As soon as a small process begin = doing work, it may use a lot of resources, especially memory. We may know t= he system can at most handle 800 simultaneous requests of a given type. Or = that the total amount of connections can't go above 5000. You could jus= t stop providing service when the service ceiling was hit, but this would n= ot allow sudden "spikes" of traffic. This is exactly why we need = a queue of incoming requests to "smooth" out request spikes and h= andle them over time.

The problem, however, is that the queue I use is a tail= -drop queue. And I have no idea of the sojourn time through the queue. My f= ear is that the tail-drop behavior will just incur more latency on the task= s that do get through the queue. Hence the attempt at using CoDel for this.=

Preliminary tests show that it does look like it perfor= ms the task it should. When the Target is not met for Interval, we begin dr= opping tasks out of the queue, like we should. More experiments are needed = however, before I know if this works well. One thing which is an important = difference is that there are no TCP-behaviour where the window will close d= own upon dropped packets. But CoDel *does* allow me to provide early feedba= ck that the Target can't really be met. For real-time requests, this is= actualy the desired behavior.

The next plan is to provide a TCP-like window feature f= or batch processing of jobs. The idea is that you can have a large list of = jobs that needs to be run and have the system be self-tuning. TCP congestio= n control works, so why not apply it in a new setting :)


--
J.
--20cf307f3a564dfb1d04d1ea9683--