From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp131.iad.emailsrvr.com (smtp131.iad.emailsrvr.com [207.97.245.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id D24A121F1BB for ; Sat, 23 Feb 2013 11:04:35 -0800 (PST) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp33.relay.iad1a.emailsrvr.com (SMTP Server) with ESMTP id 6A9D63027F; Sat, 23 Feb 2013 14:04:34 -0500 (EST) X-Virus-Scanned: OK Received: from legacy19.wa-web.iad1a (legacy19.wa-web.iad1a.rsapps.net [192.168.2.205]) by smtp33.relay.iad1a.emailsrvr.com (SMTP Server) with ESMTP id 477BA3024F; Sat, 23 Feb 2013 14:04:34 -0500 (EST) Received: from reed.com (localhost.localdomain [127.0.0.1]) by legacy19.wa-web.iad1a (Postfix) with ESMTP id 35ECA2D5806B; Sat, 23 Feb 2013 14:04:34 -0500 (EST) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Sat, 23 Feb 2013 14:04:34 -0500 (EST) Date: Sat, 23 Feb 2013 14:04:34 -0500 (EST) From: dpreed@reed.com To: "Dave Taht" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20130223140434000000_82169" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: Message-ID: <1361646274.21926765@apps.rackspace.com> X-Mailer: webmail7.0 Cc: cerowrt-devel@lists.bufferbloat.net, bloat Subject: Re: [Cerowrt-devel] Fwd: High Speed WAN Rsync now possible via UDT X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2013 19:04:36 -0000 ------=_20130223140434000000_82169 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AI have not investigated UDT in detail. My sense is that it is pretty m= uch like TCP but assumes that the bottleneck is actually a fixed rate link,= rather than one with significant variability in the capacity-per-flow avai= lable.=0A =0AI don't think it uses the "biggest gun" that can improve thing= s when the end-to-end latency gets long: non-ordered delivery acknowledgeme= nt coupled with Fountain Codes (or some other rateless erasure code).=0A = =0AFor file transfer (where the only goal is complete delivery of the entir= e file), sending every file block independently and using rateless erasure = coding to ensure that any sufficiently large subset of the packets will giv= e the contents of the file, one can allow the bottleneck link to use packet= drops to signal congestion, while not requiring sequential retransmission.= =0A =0ASo if I were starting with fq_codel in the network, I would use a ra= te-controlled UDP with Fountain Coding on the file contents, per-file-block= acknowledgment (using a run-length-coded bitstring for blocks correctly re= ceived), and a rate-controller that uses the number of blocks received vs. = blocks sent to detect drops due to buffer overflow to set window size to li= mit the number of blocks in transit.=0A =0AThat would maximize FTP, while s= till allowing "mice" and "ants" and other FTPs to get great, low-latency, s= ervice.=0A =0A-----Original Message-----=0AFrom: "Dave Taht" =0ASent: Friday, February 22, 2013 11:48am=0ATo: cerowrt-devel@lists= .bufferbloat.net, "bloat" =0ASubject: [Cerowrt= -devel] Fwd: High Speed WAN Rsync now possible via UDT=0A=0A=0A=0ASince we = are essentially observing "wan" latencies anyway I am curious as to how the= UDT protocol functions... has anyone tried it?=0A=0AIf useful (or scary) I= 'll try to find the time to package it up for cero. =0A=0AI liked how mosh = solved the terminal emulation problem over lousy links (in fact, reflecting= on it, had it existed in 1998 when I was mucking with the strip protocol, = I'd have not bothered with getting tcp to work at all. :) ) I also get a ki= ck out of people using ssh to "authenticate" and then dropping to something= else....=0A=0A=0A=0A---------- Forwarded message ----------=0AFrom: Erich = Weiler <[mailto:weiler@soe.ucsc.edu] weiler@soe.ucsc.edu>=0A Date: Fri, Feb= 22, 2013 at 10:54 AM=0ASubject: High Speed WAN Rsync now possible!!!=0ATo:= [mailto:rsync@lists.samba.org] rsync@lists.samba.org=0A=0A=0AFolks-=0A=0A = Just wanted to plug a totally awesome software package from a group I know:= UDR (UDT Enabled Rsync).=0A=0A For those not familiar with UDT, it is a lo= w level network protocol based on UDP that allows for high speed transfers = over high latency WAN networks:=0A=0A[http://udt.sourceforge.net/] http://u= dt.sourceforge.net/=0A=0A For a while the UDT API was available, developed = by the Laboratory for Advanced Computing at the University of Chicago, but = there were little development around it for actually developing a software = suite to allow for high speed WAN transfers, such as can be achieved by Asp= era, GridFTP, FDT, and a couple others. The problem with those often is:= =0A=0A Aspera: Great but *crazy* expensive=0A GridFTP: Not bad but non-triv= ial to set up=0A FDT: Java (Eh...)=0A=0A But the awesome thing here is that= UDR is a lightweight wrapper for rsync that allows for rsync functionality= , but uses UDT as the underlying protocol for transfer (no rsh or ssh). It= authenticates over ssh and then transfers the data over UDT streams. And = supports encryption.=0A=0A Right now it is kind of in beta but we've been u= sing it for a while and it's very stable. It has been tested on Linux, BSD= and OSX. It may compile on other platforms but not much testing has been = done on those. Written in C++.=0A=0A[https://github.com/LabAdvComp/UDR] ht= tps://github.com/LabAdvComp/ UDR=0A=0A You can clone the repo with 'git clo= ne' and build the code. It compiles very easily and only requires the Open= SSL library as a dependency.=0A=0A As an example: We have been trying to t= ransfer data from California to our servers in Germany for a while, and hav= e only be getting 10Mb/s. With UDR we get 700Mb/s. Not bad.=0A=0A There ar= e details on the GitHub page. Check it out!!=0A=0A -erich=0A -- =0A Please= use reply-all for most replies to avoid omitting the mailing list.=0A To u= nsubscribe or change options: [https://lists.samba.org/mailman/listinfo/rsy= nc] https://lists.samba.org/ mailman/listinfo/rsync=0A Before posting, read= : [http://www.catb.org/~esr/faqs/smart-questions.html] http://www.catb.org/= ~esr/faqs/smart-questions.html=0A=0A=0A=0A-- =0ADave T=C3=A4ht=0A=0AFixing = bufferbloat with cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] = http://www.teklibre.com/cerowrt/subscribe.html ------=_20130223140434000000_82169 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

= I have not investigated UDT in detail.   My sense is that it is p= retty much like TCP but assumes that the bottleneck is actually a fixed rat= e link, rather than one with significant variability in the capacity-per-fl= ow available.

=0A

 

=0A

I don't think it uses the "biggest gun" that can i= mprove things when the end-to-end latency gets long: non-ordered delivery a= cknowledgement coupled with Fountain Codes (or some other rateless erasure = code).

=0A

 

=0A

For file transfer (where the only goal is complete deliver= y of the entire file), sending every file block independently and using rat= eless erasure coding to ensure that any sufficiently large subset of the pa= ckets will give the contents of the file, one can allow the bottleneck link= to use packet drops to signal congestion, while not requiring sequential r= etransmission.

=0A

 

=0A

So if I were starting with fq_codel in the network= , I would use a rate-controlled UDP with Fountain Coding on the file conten= ts, per-file-block acknowledgment (using a run-length-coded bitstring for b= locks correctly received), and a rate-controller that uses the number of bl= ocks received vs. blocks sent to detect drops due to buffer overflow to set= window size to limit the number of blocks in transit.

=0A

 

=0A

That would= maximize FTP, while still allowing "mice" and "ants" and other FTPs to get= great, low-latency, service.

=0A

 =

=0A

-----Original Message-----
Fro= m: "Dave Taht" <dave.taht@gmail.com>
Sent: Friday, February 22, = 2013 11:48am
To: cerowrt-devel@lists.bufferbloat.net, "bloat" <bloa= t@lists.bufferbloat.net>
Subject: [Cerowrt-devel] Fwd: High Speed W= AN Rsync now possible via UDT

=0A
=0A

Since we are essentially observin= g "wan" latencies anyway I am curious as to how the UDT protocol functions.= .. has anyone tried it?

If useful (or scary) I'll try to find th= e time to package it up for cero.

I liked how mosh solved the t= erminal emulation problem over lousy links (in fact, reflecting on it, had = it existed in 1998 when I was mucking with the strip protocol, I'd have not= bothered with getting tcp to work at all. :) ) I also get a kick out of pe= ople using ssh to "authenticate" and then dropping to something else....

=0A
---------- Forwarded messa= ge ----------
From: Erich Weiler <weiler@= soe.ucsc.edu>
Date: Fri, Feb 22, 2013 at 10:54 AM
= Subject: High Speed WAN Rsync now possible!!!
To: rsync@lists.samba.org


Folks-

Just wanted to plug a totally awesome software package from a group= I know: UDR (UDT Enabled Rsync).

For those not familiar with U= DT, it is a low level network protocol based on UDP that allows for high sp= eed transfers over high latency WAN networks:

http://udt.sourceforge.net/

For a while the UDT API was available, developed by the Laboratory= for Advanced Computing at the University of Chicago, but there were little= development around it for actually developing a software suite to allow fo= r high speed WAN transfers, such as can be achieved by Aspera, GridFTP, FDT= , and a couple others.  The problem with those often is:

A= spera: Great but *crazy* expensive
GridFTP: Not bad but non-trivial t= o set up
FDT: Java (Eh...)

But the awesome thing here is = that UDR is a lightweight wrapper for rsync that allows for rsync functiona= lity, but uses UDT as the underlying protocol for transfer (no rsh or ssh).=  It authenticates over ssh and then transfers the data over UDT strea= ms.  And supports encryption.

Right now it is kind of in b= eta but we've been using it for a while and it's very stable.  It has = been tested on Linux, BSD and OSX.  It may compile on other platforms = but not much testing has been done on those.  Written in C++.
https://= github.com/LabAdvComp/ U= DR

You can clone the repo with 'git clone' and build the co= de.  It compiles very easily and only requires the OpenSSL library as = a dependency.

As an example:  We have been trying to trans= fer data from California to our servers in Germany for a while, and have on= ly be getting 10Mb/s. With UDR we get 700Mb/s.  Not bad.

T= here are details on the GitHub page.  Check it out!!

-erich
--
Pl= ease use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/ mailman/listinfo/rsync
B= efore posting, read: http://www.catb.org/~esr/faqs/smart-questions.ht= ml
=0A


--
Dave T=C3=A4h= t

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cer= owrt/subscribe.html
 ------=_20130223140434000000_82169--