<html><head></head><body>Tor needs this stuff very badly.<br clear="none">
<br clear="none">
I do wonder whether we should focus on vpn's rather than end to end encryption that does not leak secure information through from inside as the plan seems to do. <br clear="none">
<br clear="none">
<br clear="none"><br clear="none"><div class="gmail_quote">On Dec 3, 2014, Guus Sliepen <guus@tinc-vpn.org> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k10mail">On Wed, Dec 03, 2014 at 12:07:59AM -0800, Dave Taht wrote:<br clear="none"><br clear="none">[...]<br clear="none"></pre><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><a shape="rect" href="https://github.com/dtaht/tinc">https://github.com/dtaht/tinc</a><br clear="none"><br clear="none">I successfully converted tinc to use sendmsg and recvmsg, acquire (at<br clear="none">least on linux) the TTL/Hoplimit and IP_TOS/IPv6_TCLASS packet fields,</blockquote><br clear="none">Windows does not have sendmsg()/recvmsg(), but the BSDs support it.<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">as well as SO_TIMESTAMPNS, and use a higher resolution internal clock.<br clear="none">Got passing through the dscp values to work also, but:<br clear="none"><br clear="none">A) encapsulation of ecn capable marked packets, and availability in<br clear="none">the outer header, without correct decapsulationm doesn't work well.<br clear="none"><br clear="none">The outer packet gets marked, but by default the marking doesn't make<br clear="none">it back into the inner packet when decoded.</blockquote><br clear="none">Is the kernel stripping the ECN bits provided by userspace? In the code<br clear="none">in your git branch you strip the ECN bits out yourself.<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">So communicating somehow that a path can take ecn (and/or diffserv<br clear="none">markings) is needed between tinc daemons. I thought of perhaps<br clear="none">crafting a special icmp message marked with CE but am open to ideas<br clear="none">that would be backward compatible.</blockquote><br clear="none">PMTU probes are used to discover whether UDP works and how big the path<br clear="none">MTU is, maybe it could be used to discover whether ECN works as well?<br clear="none">Set one of the ECN bits on some of the PMTU probes, and if you receive a<br clear="none">probe with that ECN bit set, also set it on the probe reply. If you<br clear="none">succesfully receive a reply with ECN bits set, then you know ECN works.<br clear="none">Since the remote side just echoes the contents of the probe, you could<br clear="none">also put a copy of the ECN bits in the probe payload, and then you can<br clear="none">detect if the ECN bits got zeroed. You can also define an OPTION_ECN in<br clear="none">src/connection.h, so nodes can announce their support for ECN, but that<br clear="none">should not be necessary I think.<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">B) I have long theorized that a lot of userspace vpns bottleneck on<br clear="none">the read and encapsulate step, and being strict FIFOs,<br clear="none">gradually accumulate delay until finally they run out of read socket<br clear="none">buffer space and start dropping packets.</blockquote><br clear="none">Well, encryption and decryption takes a lot of CPU time, but context<br clear="none">switches are also bad.<br clear="none"><br clear="none">Tinc is treating UDP in a strictly FIFO way, but actually it does use a<br clear="none">RED algorithm when tunneling over TCP. That said, it only looks at its<br clear="none">own buffers to determine when to drop packets, and those only come into<br clear="none">play once the kernel's TCP buffers are filled.<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">so I had a couple thoughts towards using multiple rx queues in the<br clear="none">vtun interface, and/or trying to read more than one packet at a time<br clear="none">(via recvmmsg) and do some level of fair queueing and queue management<br clear="none">(codel) inside tinc itself. I think that's<br clear="none">pretty doable without modifying the protocol any, but I'm not sure of<br clear="none">it's value until I saturate some cpu more.</blockquote><br clear="none">I'd welcome any work in this area :)<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">(and if you thought recvmsg was complex, look at recvmmsg)</blockquote><br clear="none">It seems someone is already working on that, see<br clear="none"><a shape="rect" href="https://github.com/jasdeep-hundal/tinc">https://github.com/jasdeep-hundal/tinc</a>.<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">D)<br clear="none"><br clear="none">the bottleneck link above is actually not tinc but the gateway, and as<br clear="none">the gateway reverts to codel behavior on a single encapsulated flow<br clear="none">encapsulating all the other flows, we end up with about 40ms of<br clear="none">induced delay on this test. While I have a better codel (gets below<br clear="none">20ms latency, not deployed), *fq*_codel by identifying individual<br clear="none">flows gets the induced delay on those flows down below 5ms.</blockquote><br clear="none">But that should improve with ECN if fq_codel is configured to use that,<br clear="none">right?<br clear="none"><br clear="none"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">At one level, tinc being so nicely meshy means that the "fq" part of<br clear="none">fq_codel on the gateway will have more chance to work against the<br clear="none">multiple vpn flows it generates for all the potential vpn endpoints...<br clear="none"><br clear="none">but at another... lookie here! ipv6! 2^64 addresses or more to use!<br clear="none">and port space to burn! What if I could make tinc open up 1024 ports<br clear="none">per connection, and have it fq all it's flows over those? What could<br clear="none">go wrong?</blockquote><br clear="none">Right, hash the header of the original packets, and then select a port<br clear="none">or address based on the hash? What about putting that hash in the flow<br clear="none">label of outer packets? Any routers that would actually treat those as<br clear="none">separate flows?<br clear="none"></blockquote></div><br clear="none">-- Sent from my Android device with <b><a shape="rect" href="https://play.google.com/store/apps/details?id=com.onegravity.k10.pro2">K-@ Mail</a></b>. Please excuse my brevity.</body></html>