From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp129.dfw.emailsrvr.com (smtp129.dfw.emailsrvr.com [67.192.241.129]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 75A9921F4E9 for ; Thu, 4 Dec 2014 07:31:04 -0800 (PST) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp29.relay.dfw1a.emailsrvr.com (SMTP Server) with ESMTP id 83C781801B1; Thu, 4 Dec 2014 10:31:02 -0500 (EST) X-Virus-Scanned: OK Received: by smtp29.relay.dfw1a.emailsrvr.com (Authenticated sender: dpreed-AT-reed.com) with ESMTPSA id A84DC1801DD; Thu, 4 Dec 2014 10:31:01 -0500 (EST) X-Sender-Id: dpreed@reed.com Received: from [100.71.199.201] (251.sub-70-197-13.myvzw.com [70.197.13.251]) (using TLSv1.2 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:465 (trex/5.4.1); Thu, 04 Dec 2014 15:31:02 GMT User-Agent: K-@ Mail for Android X-Priority: 3 In-Reply-To: References: <20141203120246.GO10533@sliepen.org> <892513fe-8e57-4ee9-be7d-423a3afb4fba@reed.com> <1417653909.838517290@apps.rackspace.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----S0BX1XE97Q2HGGOATQ7A1GQSLH62KW" Content-Transfer-Encoding: 7bit From: "David P. Reed" Date: Thu, 04 Dec 2014 07:30:58 -0800 To: Sebastian Moeller Message-ID: <05c3d7d2-0f61-47a7-8f77-0033dd4a3230@reed.com> Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] tinc vpn: adding dscp passthrough (priorityinherit), ecn, and fq_codel support X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2014 15:31:33 -0000 ------S0BX1XE97Q2HGGOATQ7A1GQSLH62KW Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I'd be more likely to agree if I thought that the network level technologie= s could work=2E The problem is that I've been in the system security busine= ss long enough (starting in 1973 in a professional role) that I know how us= eless the network level techniques are and how little is gained by tinkerin= g with them to improve them=2E Scada systems cannot be secured by addons, p= eriod=2E Even airgap didn't protect against Stuxnet=2E It's approximately = the same as security theater to think that home nets can be secured by a fa= ncy vpn=2E That only deals with a single threat model and the solution real= ly does not scale at all=2E It just lets designers of home systems off the = hook so they can promote inherently bad designs by saying they don't need t= o fix their designs=2E As soon as a hacker can control most of the stuff i= n a rich person's home because of the IoT craze, we will see for profit rin= gs promoting ransom attacks=2E Vpn's aren't likely to fix that at the netw= ork level=2E So my point is a little subtle=2E Put effort where it pays of= f=2E At the end to end authentication interoperability level rather than fa= ntasy based solutions that just break the network=2E At the creation of sys= tems for attribution and policing and prosecution of the motivated conspira= tors=2E On Dec 4, 2014, Sebastian Moeller wrote: >Hi, = > >on the danger of going off on a tangent=2E=2E=2E > >On Dec 4, 2014, at 0= 1:45 , dpreed@reed=2Ecom wrote: > >> Awesome start on the issue, in your no= te, Dave=2E Tor needs to change >for several reasons - not that it isn't g= reat, but with IPv6 and other >things coming on line, plus the understandin= g of fq_codel's rationale, >plus =2E=2E=2E - the world can do much better= =2E Same with VPNs=2E >> >> I hope we can set our sights on a convergent= target that doesn't get >bogged down in the tradeoffs that were made when = VPNs were originally >proposed=2E The world is no longer a bunch of discon= nected networks >protected by Cheswick firewalls=2E Cheswick said they wer= e only >temporary, and they've outlived their usefulness - they actually cr= eate >security risks more than they fix them (centralizing security creates= >points of failure and attack that exponentially decrease the attackers' >= work factor)=2E To some extent that is also true for Tor after these >many= years=2E > > But trying to keep all computers on the end/edge secure also = does not >work/scale well, so both ends of the continuum have their issues;= I >would not be amazed if realistically we need to keep doing both=E2=80= =A6 >securing the end devices as well as intermediary devices=2E > >> >> B= y putting the intelligence about security in the network, you >basically do= all the bad things that the end-to-end argument encourages >you to avoid= =2E > > I might misinterpret your point here, but given the devices peopl= e >connect to their own networks full e2e without added layers of security = >seems not very practical=2E There is an ever growing class of devices >orp= haned by their makers (either explicitly like old ipods, or >implicitly by = lack of timely security fixes like Siemens SCADA systems, >plus old but use= ful hardware requiring obsolete operating systems like >windows XP, the lis= t goes on=2E=2E=2E) that still can be used to good effect >in a secured net= work but can not be trusted to access the wider >internet, let alone be con= tacted by the wider internet=2E So unless we >want to retire all those devi= ces of dubious =E2=80=9Csecurity=E2=80=9D we need a layer >in the network t= hat can preempt traffic to and from specific devices=2E >In the old IPv4 da= ys the for =E2=80=9Cend-users=E2=80=9D ubiquitous NAT tool care of >the =E2= =80=9Ctraffic to specific devices=E2=80=9D to some degree=2E I would be hap= py if >even in the brave new IPv6 world we could keep such >gatekeepers/bou= ncers around, ideally also controlling which devices can >send packets to t= he internet=2E > I do not propose to put these things into the core of the = network, but >the boundary between a implicitly trusted home-network and th= e internet >seems like a decent compromise to me=2E (I would also like such= a device >to default to "no implicit connectivity=E2=80=9D, so that each d= evice needs to >be manually declared fit for the internet, so that the user= s are aware >of this system)=2E Since the number of connections between the= home-net >and the internet often is smaller than the number of connected d= evices >in such a network, the transfer points/routers seem like ideal >can= didates to implement the =E2=80=9Caccess control=E2=80=9D=2E =2E (This does= not mean >that keeping end systems not secured and patched is a good idea,= but at >least should greatly diminish the risk imposed by sub-optimally se= cured >end points, I think/hope)=2E > Being a biologist I like to think abo= ut this as maintaining a special >niche for hard/impossible to secure devic= es in my home, avoiding their >extinction/pawning by keeping the predators = away; as fitness is >relative=2E Might not work perfectly, but =E2=80=9Cgoo= d enough=E2=80=9D would do ;) > To cite the russians: Dowerjai, no prowerja= i, "Trust, but verify=E2=80=9D=E2=80=A6 > > >> We could also put congestion= control in the network by re-creating >admission control and requiring con= tractual agreements to carry traffic >across every intermediary=2E But I t= hink that basically destroys almost >all the value of an "inter" net=2E It= makes it a balkanized proprietary >set of subnets that have dozens of reas= ons why you can't connect with >anyone else, and no way to be free to conne= ct=2E >> >> >> >> >> >> On Wednesday, December 3, 2014 2:44pm, "Dav= e Taht" > said: >> >> > On Wed, Dec 3, 2014 at 6:= 17 AM, David P=2E Reed >wrote: >> > > Tor needs this st= uff very badly=2E >> > >> > Tor has many, many problematic behaviors relev= ant to congestion >control >> > in general=2E Let me paste a bit of private= discussion I'd had on it >in a second, >> > but a very good paper that tou= ched upon it all was: >> > >> > DefenestraTor: Throwing out Windows in Tor= >> > http://www=2Ecypherpunks=2Eca/~iang/pubs/defenestrator=2Epdf >> > >>= > Honestly tor needs to move to udp, and hide in all the upcoming >> > web= rtc traffic=2E=2E=2E=2E >> > >> > >http://blog=2Emozilla=2Eorg/futurerelea= ses/2014/10/16/test-the-new-firefox-hello-webrtc-feature-in-firefox-beta/ >= > > >> > webrtc needs some sort of non-centralized rendezvous mechanism, b= ut >I am REALLY >> > happy to see calls and video stay entirely inside my n= etwork when >they can be >> > negotiated as such=2E >> > >> > https://plus= =2Egoogle=2Ecom/u/0/107942175615993706558/posts/M4xUtpCKJ4P >> > >> > And = of course, people are busily reinventing torrent in webrtc >without >> > pa= ying attention to congestion control at all=2E >> > >> > https://github=2E= com/feross/webtorrent/issues/39 >> > >> > Giving access to udp to javascri= pt programmers=2E=2E=2E what could go >wrong? >> > :/ >> > >> > > I do won= der whether we should focus on vpn's rather than end to >end >> > > encrypt= ion that does not leak secure information through from >inside as the >> > = > plan seems to do=2E >> > >> > "plan"? >> > >> > I like e2e encryption= =2E I also like overlay networks=2E And meshes=2E >> > And working dns and = service discovery=2E And low latency=2E >> > >> > vpns are useful abstract= ions for sharing an address space you >> > may not want to share more widel= y=2E >> > >> > and: I've taken a lot of flack about how fq doesn't help on= >conventional >> > vpns, and well, just came up with an unconventional vpn= idea, >> > that might have some legs here=2E=2E=2E (certainly in my case t= inc >> > as constructed already, no patches, solves hooking together the >>= > 12 networks I have around the globe, mostly) >> > >> > As for "leaking = information", packet size and frequency is >generally >> > an obvious indic= ator of a given traffic type, some padding added or >> > no=2E There is one= piece of plaintext >> > in tinc (the seqno), also=2E It also uses a fixed = port number for >both >> > sides of the connection (perhaps it shouldn't) >= > > >> > So I don't necessarily see a difference between sending a whole l= ot >of >> > varying data on one tuple >> > >> > 2001:db8::1 <-> 2001:db8:1= ::1 on port 655 >> > >> > vs >> > >> > 2001:db8::1 <-> 2001:db8:1::1 port= 655 >> > 2001:db8::2 <-> 2001:db8:1::1 port 655 >> > 2001:db8::3 <-> 2001:= db8:1::1 port 655 >> > 2001:db8::4 <-> 2001:db8:1::1 port 655 >> > =2E=2E= =2E=2E >> > >> > which solves the fq problem on a vpn like tinc neatly=2E = A security >feature >> > could be source specific routing where we send stu= ff over different >paths >> > from different ipv6 source addresses=2E=2E=2E= and mixing up the src/dest >ports >> > more but that complexifies the fq p= ortion of the algo=2E=2E=2E=2E my >thought >> > for an initial implementati= on is to just hard code the ipv6 address >range=2E >> > >> > I think howev= er that adding tons and tons of ipv6 addresses to a >given >> > interface i= s probably slow, >> > and might break things like nd and/or multicast=2E=2E= =2E >> > >> > what would be cooler would be if you could allocate an entir= e /64 >(or >> > /118) to the vpn daemon >> > >> > bindtoaddress(2001:db8::= /118) (give me all the data for 1024 ips) >> > >> > but I am not sure how = to go about doing that=2E=2E >> > >> > =2E=2E=2Emoving back to a formerly = private discussion about tors woes=2E=2E=2E >> > >> > >> > "This conversa= tion is a bit separate from #11197 (which is an >> > implementation issue i= n obfsproxy), so separate discussion >somewhere >> > would probably be requ= ired=2E >> > >> > So, there appears to be a slight misconception on how to= r traffic >> > travels across the Internet that I will attempt to clarify, = and >> > hopefully not get too terribly wrong=2E >> > >> > Each step of a = given connection over tor involves multiple TCP/IP >> > connections=2E To u= se a standard example of someone trying to watch >Cat >> > Videos on the "r= eal internet", it will look approximately like >thus: >> > >> > Client <->= Guard <-> Relay <-> Exit <-> Cat Videos >> > >> > Each step is a separate= TCP/IP connection, authenticated and >encrypted >> > via TLS (TLS is likew= ise terminated at each hop)=2E Using a pluggable >> > transport encapsulate= s the first hop's TLS session with a different >> > protocol be it obfs2, o= bfs3, or something else=2E >> > >> > The cat videos are passed through thi= s path of many TCP/IP >connections >> > across things called Circuits that = are created/extended by the >Client >> > one hop at a time (So the example = above, the kitty cats travel >across >> > 4 TCP/IP connections, relaying da= ta across a Circuit that spans >from >> > the Client to the Exit=2E If my a= rt skills were up to it, I would >draw a >> > diagram=2E)=2E >> > >> > Cir= cuits are currently required to provide reliable, in-order >delivery=2E >> = > >> > In addition to the standard congestion control provided by TCP/IP >= on a >> > per-hop basis, there is Circuit level flow control *and* "end to = >end" >> > flow control in the form of RELAY_SENDME cells, but given that >= multiple >> > circuits can end up being multiplexed over a singlular TCP/IP= >> > connection, propagation of these RELAY_SENDME cells can get delayed >= due >> > to HOL issues=2E >> > >> > So, with that quick and dirty overview= out of the way: >> > >> > * "Ah so if ecn is enabled it can be used?" >> = > >> > ECN will be used if it is enabled, *but* the congestion information= >> > will not get propaged to the source/destination of a given stream=2E = >> > >> > * "Does it retain iw10 (the Linux default nowadays sadly)?" >> >= >> > Each TCP/IP connection if sent from a host that uses a obnoxiously >= > > large initial window, will have an obnoxiously large initial >> > windo= w=2E >> > >> > It is worth noting that since multiple Circuits originating= from >> > potentially numerous clients can and will reuse existing TCP/IP = >> > connections if able to (see 5=2E3=2E1 of the tor spec) that dropping >= packets >> > between tor relays is kind of bad, because all of the separate= >> > encapsulated flows sharing the singular TCP/IP link will suffer >(ECN= >> > would help here)=2E This situation is rather unfortunate as the good = >> > active queue management algorithms drop packets (when ECN is not >> > = available)=2E >> > >> > A better summary of tor's flow control/bufferbloat= woes is given >in: >> > >> > DefenestraTor: Throwing out Windows in Tor >= > > http://www=2Ecypherpunks=2Eca/~iang/pubs/defenestrator=2Epdf >> > >> >= The N23 algorithm suggested in the paper did not end up getting >> > imple= mented into Tor, but I do not remember the reason off the top >of >> > my h= ead=2E" >> > >> > >> > > >> > > >> > > >> > > On Dec 3, 2014, Guus Sliepe= n wrote: >> > >> >> > >> On Wed, Dec 03, 2014 at 12:0= 7:59AM -0800, Dave Taht wrote: >> > >> >> > >> [=2E=2E=2E] >> > >>> >> > >>= > https://github=2Ecom/dtaht/tinc >> > >>> >> > >>> I successfully converte= d tinc to use sendmsg and recvmsg, >acquire >> > (at >> > >>> least on linu= x) the TTL/Hoplimit and IP_TOS/IPv6_TCLASS packet >> > fields, >> > >> >> >= >> >> > >> Windows does not have sendmsg()/recvmsg(), but the BSDs support= >it=2E >> > >> >> > >>> as well as SO_TIMESTAMPNS, and use a higher resolu= tion internal >> > clock=2E >> > >>> Got passing through the dscp values to= work also, but: >> > >>> >> > >>> A) encapsulation of ecn capable marked p= ackets, and >availability in >> > >>> the outer header, without correct dec= apsulationm doesn't work >well=2E >> > >>> >> > >>> The outer packet gets m= arked, but by default the marking >doesn't >> > make >> > >>> it back into = the inner packet when decoded=2E >> > >> >> > >> >> > >> Is the kernel stri= pping the ECN bits provided by userspace? In >the code >> > >> in your git = branch you strip the ECN bits out yourself=2E >> > >> >> > >>> So communica= ting somehow that a path can take ecn (and/or >diffserv >> > >>> markings) = is needed between tinc daemons=2E I thought of perhaps >> > >>> crafting a = special icmp message marked with CE but am open to >ideas >> > >>> that wou= ld be backward compatible=2E >> > >> >> > >> >> > >> PMTU probes are used t= o discover whether UDP works and how big >the path >> > >> MTU is, maybe it= could be used to discover whether ECN works as >well? >> > >> Set one of t= he ECN bits on some of the PMTU probes, and if you >receive a >> > >> probe= with that ECN bit set, also set it on the probe reply=2E If >you >> > >> s= uccesfully receive a reply with ECN bits set, then you know ECN >works=2E >= > > >> Since the remote side just echoes the contents of the probe, you >co= uld >> > >> also put a copy of the ECN bits in the probe payload, and then = >you can >> > >> detect if the ECN bits got zeroed=2E You can also define a= n >OPTION_ECN in >> > >> src/connection=2Eh, so nodes can announce their su= pport for ECN, >but that >> > >> should not be necessary I think=2E >> > >>= >> > >>> B) I have long theorized that a lot of userspace vpns >bottleneck= on >> > >>> the read and encapsulate step, and being strict FIFOs, >> > >>= > gradually accumulate delay until finally they run out of read >socket >> = > >>> buffer space and start dropping packets=2E >> > >> >> > >> >> > >> We= ll, encryption and decryption takes a lot of CPU time, but >context >> > >>= switches are also bad=2E >> > >> >> > >> Tinc is treating UDP in a strictl= y FIFO way, but actually it >does use a >> > >> RED algorithm when tunnelin= g over TCP=2E That said, it only looks >at its >> > >> own buffers to deter= mine when to drop packets, and those only >come into >> > >> play once the = kernel's TCP buffers are filled=2E >> > >> >> > >>> so I had a couple thoug= hts towards using multiple rx queues in >the >> > >>> vtun interface, and/o= r trying to read more than one packet at a >time >> > >>> (via recvmmsg) an= d do some level of fair queueing and queue >> > management >> > >>> (codel)= inside tinc itself=2E I think that's >> > >>> pretty doable without modify= ing the protocol any, but I'm not >sure >> > of >> > >>> it's value until I= saturate some cpu more=2E >> > >> >> > >> >> > >> I'd welcome any work in = this area :) >> > >> >> > >>> (and if you thought recvmsg was complex, look= at recvmmsg) >> > >> >> > >> >> > >> It seems someone is already working o= n that, see >> > >> https://github=2Ecom/jasdeep-hundal/tinc=2E >> > >> >> = > >>> D) >> > >>> >> > >>> the bottleneck link above is actually not tinc b= ut the gateway, >and >> > as >> > >>> the gateway reverts to codel behavior= on a single encapsulated >flow >> > >>> encapsulating all the other flows,= we end up with about 40ms of >> > >>> induced delay on this test=2E While = I have a better codel (gets >below >> > >>> 20ms latency, not deployed), *f= q*_codel by identifying >individual >> > >>> flows gets the induced delay o= n those flows down below 5ms=2E >> > >> >> > >> >> > >> But that should imp= rove with ECN if fq_codel is configured to >use that, >> > >> right? >> > >= > >> > >>> At one level, tinc being so nicely meshy means that the "fq" >pa= rt of >> > >>> fq_codel on the gateway will have more chance to work agains= t >the >> > >>> multiple vpn flows it generates for all the potential vpn >= > > endpoints=2E=2E=2E >> > >>> >> > >>> but at another=2E=2E=2E lookie her= e! ipv6! 2^64 addresses or more to >use! >> > >>> and port space to burn! W= hat if I could make tinc open up 1024 >ports >> > >>> per connection, and h= ave it fq all it's flows over those? What >could >> > >>> go wrong? >> > >>= >> > >> >> > >> Right, hash the header of the original packets, and then s= elect >a port >> > >> or address based on the hash? What about putting that= hash in >the flow >> > >> label of outer packets? Any routers that would a= ctually treat >those as >> > >> separate flows? >> > > >> > > >> > > -- Sen= t from my Android device with K-@ Mail=2E Please excuse my >brevity=2E >> >= > >> > > _______________________________________________ >> > > Cerowrt-de= vel mailing list >> > > Cerowrt-devel@lists=2Ebufferbloat=2Enet >> > > http= s://lists=2Ebufferbloat=2Enet/listinfo/cerowrt-devel >> > > >> > >> > >> = > >> > -- >> > Dave T=C3=A4ht >> > >> > thttp://www=2Ebufferbloat=2Enet/p= rojects/bloat/wiki/Upcoming_Talks >> > >> _________________________________= ______________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists=2Ebuffe= rbloat=2Enet >> https://lists=2Ebufferbloat=2Enet/listinfo/cerowrt-devel -= - Sent from my Android device with K-@ Mail=2E Please excuse my brevity=2E ------S0BX1XE97Q2HGGOATQ7A1GQSLH62KW Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable I'd be more likely to agree if I thought that the = network level technologies could work=2E The problem is that I've been in t= he system security business long enough (starting in 1973 in a professional= role) that I know how useless the network level techniques are and how lit= tle is gained by tinkering with them to improve them=2E Scada systems canno= t be secured by addons, period=2E Even airgap didn't protect against Stuxne= t=2E

It's approximately the same as = security theater to think that home nets can be secured by a fancy vpn=2E T= hat only deals with a single threat model and the solution really does not = scale at all=2E It just lets designers of home systems off the hook so they= can promote inherently bad designs by saying they don't need to fix their = designs=2E

As soon as a hacker can c= ontrol most of the stuff in a rich person's home because of the IoT craze, = we will see for profit rings promoting ransom attacks=2E
=
Vpn's aren't likely to fix that at the network level= =2E  So my point is a little subtle=2E Put effort where it pays off=2E= At the end to end authentication interoperability level rather than fantas= y based solutions that just break the network=2E At the creation of systems= for attribution and policing and prosecution of the motivated conspirators= =2E

On Dec = 4, 2014, Sebastian Moeller <moeller0@gmx=2Ede> wrote:
Hi,

on the danger of going off on a tangent=
=2E=2E=2E

On Dec 4, 2014, at 01:45 , d=
preed@reed=2Ecom wrote:

Awesome start on the issue, in you= r note, Dave=2E Tor needs to change for several reasons - not that it isn'= t great, but with IPv6 and other things coming on line, plus the understand= ing of fq_codel's rationale, plus =2E=2E=2E - the world can do much better= =2E Same with VPNs=2E

I hope we can s= et our sights on a convergent target that doesn't get bogged down in the tr= adeoffs that were made when VPNs were originally proposed=2E The world is = no longer a bunch of disconnected networks protected by Cheswick firewalls= =2E Cheswick said they were only temporary, and they've outlived their use= fulness - they actually create security risks more than they fix them (cent= ralizing security creates points of failure and attack that exponentially d= ecrease the attackers' work factor)=2E To some extent that is also true fo= r Tor after these many years=2E

But trying = to keep all computers on the end/edge secure also does not work/scale well,= so both ends of the continuum have their issues; I would not be amazed if = realistically we need to keep doing both… securing the end devices a= s well as intermediary devices=2E


By putting = the intelligence about security in the network, you basically do all the ba= d things that the end-to-end argument encourages you to avoid=2E
I might misinterpret your point here, but given the= devices people connect to their own networks full e2e without added layers= of security seems not very practical=2E There is an ever growing class of = devices orphaned by their makers (either explicitly like old ipods, or impl= icitly by lack of timely security fixes like Siemens SCADA systems, plus ol= d but useful hardware requiring obsolete operating systems like windows XP,= the list goes on=2E=2E=2E) that still can be used to good effect in a secu= red network but can not be trusted to access the wider internet, let alone = be contacted by the wider internet=2E So unless we want to retire all those= devices of dubious “security” we need a layer in the network t= hat can preempt traffic to and from specific devices=2E In the old IPv4 day= s the for “end-users” ubiquitous NAT tool care of the “tr= affic to specific devices” to some degree=2E I would be happy if even= in the brave new IPv6 world we could keep such gatekeepers/bouncers around= , ideally also controlling which devices can send packets to the internet= =2E
I do not propose to put these things into the core o= f the network, but the boundary between a implicitly trusted home-network a= nd the internet seems like a decent compromise to me=2E (I would also like = such a device to default to "no implicit connectivity”, so that = each device needs to be manually declared fit for the internet, so that the= users are aware of this system)=2E Since the number of connections between= the home-net and the internet often is smaller than the number of connecte= d devices in such a network, the transfer points/routers seem like ideal ca= ndidates to implement the “access control”=2E =2E (This does no= t mean that keeping end systems not secured and patched is a good idea, but= at least should greatly diminish the risk imposed by sub-optimally secured= end points, I think/hope)=2E
Being a biologist I like t= o think about this as maintaining a special niche for hard/impossible to se= cure devices in my home, avoiding their extinction/pawning by keeping the p= redators away; as fitness is relative=2E Might not work perfectly, but &ldq= uo;good enough” would do ;)
To cite the russians: = Dowerjai, no prowerjai, "Trust, but verify”…


We could also put congestion control in the network by re= -creating admission control and requiring contractual agreements to carry t= raffic across every intermediary=2E But I think that basically destroys al= most all the value of an "inter" net=2E It makes it a balkanized= proprietary set of subnets that have dozens of reasons why you can't conne= ct with anyone else, and no way to be free to connect=2E
=




On Wednesday, December 3, 2014 2:44pm, "Dave Taht&= quot; <dave=2Etaht@gmail=2Ecom> said:

On Wed, Dec 3, 2014 = at 6:17 AM, David P=2E Reed <dpreed@reed=2Ecom> wrote:
Tor needs this stuff v= ery badly=2E

Tor has many, many problematic = behaviors relevant to congestion control
in general=2E Le= t me paste a bit of private discussion I'd had on it in a second,
but a very good paper that touched upon it all was:

DefenestraTor: Throwing out Windows in Tor
http://www=2Ecypherpunks=2Eca/~iang/pubs/defenestrat= or=2Epdf

Honestly tor needs to mov= e to udp, and hide in all the upcoming
webrtc traffic=2E= =2E=2E=2E

http://blog=2Emozilla=2Eorg/futurerelea= ses/2014/10/16/test-the-new-firefox-hello-webrtc-feature-in-firefox-beta/

webrtc needs some sort of non-centr= alized rendezvous mechanism, but I am REALLY
happy to see= calls and video stay entirely inside my network when they can be
negotiated as such=2E

https://plus=2Egoogle=2Ecom/u/0/107942175615993706558/pos= ts/M4xUtpCKJ4P

And of course, peop= le are busily reinventing torrent in webrtc without
payin= g attention to congestion control at all=2E

https://github=2Ecom/feross/webtorrent/issues/39

Giving access to udp to javascript programmers=2E=2E= =2E what could go wrong?
:/

I do wonder whe= ther we should focus on vpn's rather than end to end
encr= yption that does not leak secure information through from inside as the
plan seems to do=2E

"pla= n"?

I like e2e encryption=2E I al= so like overlay networks=2E And meshes=2E
And working dns= and service discovery=2E And low latency=2E

vpns are useful abstractions for sharing an address space you
may not want to share more widely=2E

and: I've taken a lot of flack about how fq doesn't help on conv= entional
vpns, and well, just came up with an unconventio= nal vpn idea,
that might have some legs here=2E=2E=2E (ce= rtainly in my case tinc
as constructed already, no patche= s, solves hooking together the
12 networks I have around = the globe, mostly)

As for "leakin= g information", packet size and frequency is generally
an obvious indicator of a given traffic type, some padding added or
no=2E There is one piece of plaintext
in ti= nc (the seqno), also=2E It also uses a fixed port number for both
sides of the connection (perhaps it shouldn't)
=
So I don't necessarily see a difference between sending = a whole lot of
varying data on one tuple

2001:db8::1 <-> 2001:db8:1::1 on port 655

vs

2001= :db8::1 <-> 2001:db8:1::1 port 655
2001:db8::2 <= -> 2001:db8:1::1 port 655
2001:db8::3 <-> 2001:d= b8:1::1 port 655
2001:db8::4 <-> 2001:db8:1::1 port= 655
=2E=2E=2E=2E

wh= ich solves the fq problem on a vpn like tinc neatly=2E A security featurecould be source specific routing where we send stuff over = different paths
from different ipv6 source addresses=2E= =2E=2E and mixing up the src/dest ports
more but that com= plexifies the fq portion of the algo=2E=2E=2E=2E my thought
for an initial implementation is to just hard code the ipv6 address rang= e=2E

I think however that adding tons = and tons of ipv6 addresses to a given
interface is probab= ly slow,
and might break things like nd and/or multicast= =2E=2E=2E

what would be cooler would b= e if you could allocate an entire /64 (or
/118) to the vp= n daemon

bindtoaddress(2001:db8::/118)= (give me all the data for 1024 ips)

b= ut I am not sure how to go about doing that=2E=2E

=2E=2E=2Emoving back to a formerly private discussion about tor= s woes=2E=2E=2E


&qu= ot;This conversation is a bit separate from #11197 (which is an
implementation issue in obfsproxy), so separate discussion somewhere=
would probably be required=2E

So, there appears to be a slight misconception on how tor traffi= c
travels across the Internet that I will attempt to clar= ify, and
hopefully not get too terribly wrong=2E

Each step of a given connection over tor invol= ves multiple TCP/IP
connections=2E To use a standard exam= ple of someone trying to watch Cat
Videos on the "re= al internet", it will look approximately like thus:
=
Client <-> Guard <-> Relay <-> Exit &l= t;-> Cat Videos

Each step is a sepa= rate TCP/IP connection, authenticated and encrypted
via T= LS (TLS is likewise terminated at each hop)=2E Using a pluggable
transport encapsulates the first hop's TLS session with a differe= nt
protocol be it obfs2, obfs3, or something else=2E

The cat videos are passed through this pat= h of many TCP/IP connections
across things called Circuit= s that are created/extended by the Client
one hop at a ti= me (So the example above, the kitty cats travel across
4 = TCP/IP connections, relaying data across a Circuit that spans from
the Client to the Exit=2E If my art skills were up to it, I would= draw a
diagram=2E)=2E

Circuits are currently required to provide reliable, in-order delivery= =2E

In addition to the standard conges= tion control provided by TCP/IP on a
per-hop basis, there= is Circuit level flow control *and* "end to end"
flow control in the form of RELAY_SENDME cells, but given that multiple<= br clear=3D"none">circuits can end up being multiplexed over a singlular TC= P/IP
connection, propagation of these RELAY_SENDME cells = can get delayed due
to HOL issues=2E
So, with that quick and dirty overview out of the way:

* "Ah so if ecn is enabled it can be= used?"

ECN will be used if it is= enabled, *but* the congestion information
will not get p= ropaged to the source/destination of a given stream=2E
* "Does it retain iw10 (the Linux default nowadays sa= dly)?"

Each TCP/IP connection if = sent from a host that uses a obnoxiously
large initial wi= ndow, will have an obnoxiously large initial
window=2E
It is worth noting that since multiple C= ircuits originating from
potentially numerous clients can= and will reuse existing TCP/IP
connections if able to (s= ee 5=2E3=2E1 of the tor spec) that dropping packets
betwe= en tor relays is kind of bad, because all of the separate
encapsulated flows sharing the singular TCP/IP link will suffer (ECN
would help here)=2E This situation is rather unfortunate as th= e good
active queue management algorithms drop packets (w= hen ECN is not
available)=2E

A better summary of tor's flow control/bufferbloat woes is given = in:

DefenestraTor: Throwing out Window= s in Tor
http://www=2Ecypherpunks=2Eca/~iang= /pubs/defenestrator=2Epdf

The N23 = algorithm suggested in the paper did not end up getting
i= mplemented into Tor, but I do not remember the reason off the top of
my head=2E"





On Dec 3, 2014, Guus Sliepen <g= uus@tinc-vpn=2Eorg> wrote:

On Wed, Dec 03, 2014 at 12:07:59AM= -0800, Dave Taht wrote:

[=2E=2E=2E]
https:= //github=2Ecom/dtaht/tinc

I succes= sfully converted tinc to use sendmsg and recvmsg, acquire
(at
least on linux)= the TTL/Hoplimit and IP_TOS/IPv6_TCLASS packet
fields,

=
Windows does not have sendmsg()/r= ecvmsg(), but the BSDs support it=2E

<= blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 1ex 0=2E8ex; bord= er-left: 1px solid #e9b96e; padding-left: 1ex;">as well as SO_TIMESTAMPNS, = and use a higher resolution internal
clock=2E
Got passing through the dscp val= ues to work also, but:

A) encapsulatio= n of ecn capable marked packets, and availability in
the = outer header, without correct decapsulationm doesn't work well=2E

The outer packet gets marked, but by default t= he marking doesn't
make
it back into the inner packet when decoded=2E

Is the kernel stripping the ECN b= its provided by userspace? In the code
in your git branch= you strip the ECN bits out yourself=2E

So communicating somehow= that a path can take ecn (and/or diffserv
markings) is n= eeded between tinc daemons=2E I thought of perhaps
crafti= ng a special icmp message marked with CE but am open to ideas
that would be backward compatible=2E

PMTU probes are used to discover whether UDP works and how= big the path
MTU is, maybe it could be used to discover = whether ECN works as well?
Set one of the ECN bits on som= e of the PMTU probes, and if you receive a
probe with tha= t ECN bit set, also set it on the probe reply=2E If you
s= uccesfully receive a reply with ECN bits set, then you know ECN works=2ESince the remote side just echoes the contents of the probe= , you could
also put a copy of the ECN bits in the probe = payload, and then you can
detect if the ECN bits got zero= ed=2E You can also define an OPTION_ECN in
src/connection= =2Eh, so nodes can announce their support for ECN, but that
should not be necessary I think=2E

=
B) I have long theorized t= hat a lot of userspace vpns bottleneck on
the read and en= capsulate step, and being strict FIFOs,
gradually accumul= ate delay until finally they run out of read socket
buffe= r space and start dropping packets=2E


Well, encryption and decryption takes a lot of CPU time, but c= ontext
switches are also bad=2E

Tinc is treating UDP in a strictly FIFO way, but actually it do= es use a
RED algorithm when tunneling over TCP=2E That sa= id, it only looks at its
own buffers to determine when to= drop packets, and those only come into
play once the ker= nel's TCP buffers are filled=2E

so I had a couple thoughts towar= ds using multiple rx queues in the
vtun interface, and/or= trying to read more than one packet at a time
(via recvm= msg) and do some level of fair queueing and queue
management
(codel) inside = tinc itself=2E I think that's
pretty doable without modif= ying the protocol any, but I'm not sure
of
it's value until I saturate some cp= u more=2E


I'd welcome any= work in this area :)

(and if you thought recvmsg was complex, l= ook at recvmmsg)


It seems= someone is already working on that, see
https://github=2Ecom/ja= sdeep-hundal/tinc=2E

D)

the bottleneck link above is actually not tinc but the gateway, and
as
the ga= teway reverts to codel behavior on a single encapsulated flow
encapsulating all the other flows, we end up with about 40ms of
induced delay on this test=2E While I have a better codel (gets= below
20ms latency, not deployed), *fq*_codel by identif= ying individual
flows gets the induced delay on those flo= ws down below 5ms=2E


But = that should improve with ECN if fq_codel is configured to use that,
right?

At one level, tinc being so nicely meshy means= that the "fq" part of
fq_codel on the gateway = will have more chance to work against the
multiple vpn fl= ows it generates for all the potential vpn
<= /blockquote>
endpoints=2E=2E=2E

but at another=2E=2E=2E lookie here! ipv6! 2^64 addresses or more to use= !
and port space to burn! What if I could make tinc open = up 1024 ports
per connection, and have it fq all it's flo= ws over those? What could
go wrong?


Right, hash the header of the original packets= , and then select a port
or address based on the hash? Wh= at about putting that hash in the flow
label of outer pac= kets? Any routers that would actually treat those as
sepa= rate flows?


-- Sent from = my Android device with K-@ Mail=2E Please excuse my brevity=2E


Cerowrt-devel mailing list<= br clear=3D"none">Cerowrt-devel@lists=2Ebufferbloat=2Enet
https://lists=2Ebufferbloat=2Enet/listinfo/cerowrt-devel



--
Dave Täht

thttp://www=2Ebufferbloat=2Enet/projects/bloat/= wiki/Upcoming_Talks


Cerowrt-devel mailing list
Cerowrt-devel@lists=2Ebuffe= rbloat=2Enet
https://lists=2Ebufferbloat=2Enet/l= istinfo/cerowrt-devel


-- Sent from my Android device with K-@ Mail=2E Please excuse my brevity=2E ------S0BX1XE97Q2HGGOATQ7A1GQSLH62KW--