<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jan 13, 2018, at 12:49 PM, Toke Høiland-Jørgensen <<a href="mailto:toke@toke.dk" class="">toke@toke.dk</a>> wrote:</div><div class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Any prioritisation of ICMP may just be because ICMP replies are</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">generated by the kernel, whereas other traffic goes all the way to the</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">userspace application generating it. However, it is certainly possible</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">that ICMP is being prioritised - some people do that in an effort to</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">game benchmarks...</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote></div><br class=""><div class="">I somehow missed this comment from before. Here’s what makes me think this is more than just kernel vs userspace, and something Ubiquiti may be doing, at least in their M series firmware:</div><div class=""><br class=""></div><div class="">- To measure overhead, between a RasPI (client) and an EdgeRouter-X (server), I see only a 354µs difference between ping’s mean RTT and irtt's mean RTT when tested with 100 packets over direct cabled Ethernet (865µs - 511µs). I assume that this overhead will remain about the same for the same client and server devices, regardless of what else lies between them.</div><div class=""><br class=""></div><div class="">- Next, ping vs irtt (mean rtt of 100 packets) to the same EdgeRouter-X over an unloaded Internet connection where the CPE is a Ubiquiti PowerBeam M5-400:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>- ping: 17.03ms</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>- irtt: 20.7ms</div><div class=""><br class=""></div><div class="">- Next, ping vs irtt (mean rtt of 100 packets) to the same EdgeRouter-X, same Internet connection, but with a saturated upload and no AQM:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>- ping: 22.9ms</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>- irtt: 90.23ms</div><div class=""><br class=""></div><div class="">Saturation seems to have a much larger impact on UDP than ICMP.</div><div class=""><br class=""></div><div class="">This is only a rough test and I’ll need to reproduce this in my lab setup when it’s up again. That will remove all devices along the route as possible sources of this behavior. But since I know FreeNet isn’t doing this in their backhaul I’m fairly certain that’s what’s going on.</div><div class=""><br class=""></div><div class="">I’ll update with some results later when repro’d in the lab with NSM5s…</div><div class=""><br class=""></div></body></html>