[Cerowrt-devel] Ubiquiti Launches a Speed Test Network

[Cerowrt-devel] Ubiquiti Launches a Speed Test Network

[Matt Taggart]

https://blog.ui.com/2019/08/13/ubiquiti-launches-a-speed-test-network/

The web app (which requires a lot of js) at http://speed.ui.com/ mostly
just gives a graph of bandwidth over time and only mentions a single
ping number (which was 10ms for me, so maybe just a bucket?)

The android app gives a few single ping times to major service providers
(google, facebook, twitter) but no further latency results when running
the test.

The press release says the UniFi Network Controller can run automated
speed tests and in the updated version (5.11.39) I can see where to
enable it, but it also says it requires the UniFi Security Gateway
product to enable.

To setup your own test server at https://speed-admin.ui.com/ requires an
account (and also goes into an infinite redirect loop on my browser at
least)

So an interesting idea but they have some things they could improve.

-- 
Matt Taggart
matt@lackof.org

Re: [Cerowrt-devel] Ubiquiti Launches a Speed Test Network

[Mikael Abrahamsson]

On Wed, 4 Sep 2019, Matt Taggart wrote:

> So an interesting idea but they have some things they could improve.

I've been considering what one should run in parallel with the speed test 
to get an impression if the speedtest impacts performance of other flows / 
realtime flows, similar to what dslreports speedtest does.

I've considered running one or several simulated voip calls (50pps) and 
record RTT, PDV, packet loss etc for this session.

It would be interesting to hear any suggestions people have for a fairly 
simple codebase that does this that can be included in these kinds of test 
clients (both server and client end, and of course one that protects 
against reflection attacks etc).

iperf3 can be used for this, but from what I can see the iperf3 server 
code isn't very friendly to multiple parallel tests or even resilient 
against hung clients that doesn't close the test nicely.

I also considered using WebRTC or VoIP libraries, does anyone know what 
RTT/PDV/packet loss data can be extracted from some common ones?

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Toke Høiland-Jørgensen]

Mikael Abrahamsson <swmike@swm.pp.se> writes:

> On Wed, 4 Sep 2019, Matt Taggart wrote:
>
>> So an interesting idea but they have some things they could improve.
>
> I've been considering what one should run in parallel with the speed test 
> to get an impression if the speedtest impacts performance of other flows / 
> realtime flows, similar to what dslreports speedtest does.
>
> I've considered running one or several simulated voip calls (50pps) and 
> record RTT, PDV, packet loss etc for this session.
>
> It would be interesting to hear any suggestions people have for a fairly 
> simple codebase that does this that can be included in these kinds of test 
> clients (both server and client end, and of course one that protects 
> against reflection attacks etc).
>
> iperf3 can be used for this, but from what I can see the iperf3 server 
> code isn't very friendly to multiple parallel tests or even resilient 
> against hung clients that doesn't close the test nicely.
>
> I also considered using WebRTC or VoIP libraries, does anyone know what 
> RTT/PDV/packet loss data can be extracted from some common ones?

Pete coded up this wonderful tool for UDP-based latency testing; it's
even supported in Flent, and available on some (all?) the public-facing
servers:

https://github.com/heistp/irtt

-Toke

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Sebastian Moeller]

Hi Toke,

> On Sep 6, 2019, at 10:27, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Mikael Abrahamsson <swmike@swm.pp.se> writes:
> 
>> On Wed, 4 Sep 2019, Matt Taggart wrote:
>> 
>>> So an interesting idea but they have some things they could improve.
>> 
>> I've been considering what one should run in parallel with the speed test 
>> to get an impression if the speedtest impacts performance of other flows / 
>> realtime flows, similar to what dslreports speedtest does.
>> 
>> I've considered running one or several simulated voip calls (50pps) and 
>> record RTT, PDV, packet loss etc for this session.
>> 
>> It would be interesting to hear any suggestions people have for a fairly 
>> simple codebase that does this that can be included in these kinds of test 
>> clients (both server and client end, and of course one that protects 
>> against reflection attacks etc).
>> 
>> iperf3 can be used for this, but from what I can see the iperf3 server 
>> code isn't very friendly to multiple parallel tests or even resilient 
>> against hung clients that doesn't close the test nicely.
>> 
>> I also considered using WebRTC or VoIP libraries, does anyone know what 
>> RTT/PDV/packet loss data can be extracted from some common ones?
> 
> Pete coded up this wonderful tool for UDP-based latency testing; it's
> even supported in Flent, and available on some (all?) the public-facing
> servers:
> 
> https://github.com/heistp/irtt

This reminds of a tangentially related question, do we/could we actually write the requested DSCP into the packet payloads so we could see/display dscp bleaching/remapping packets experience during transit? For irtt, ping and even netperf TCP/UDP flows?


Best Regards
	Sebastian

> 
> -Toke
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Mikael Abrahamsson]

On Fri, 6 Sep 2019, Sebastian Moeller wrote:

> This reminds of a tangentially related question, do we/could we actually 
> write the requested DSCP into the packet payloads so we could 
> see/display dscp bleaching/remapping packets experience during transit? 
> For irtt, ping and even netperf TCP/UDP flows?

That's extremely relevant, and I'd definitely like to simulate several UDP 
50pps sessions with different DSCP values and see if there is any 
difference between them, and indeed if bleaching etc is going on.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Hi Toke,
>
>> On Sep 6, 2019, at 10:27, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Mikael Abrahamsson <swmike@swm.pp.se> writes:
>> 
>>> On Wed, 4 Sep 2019, Matt Taggart wrote:
>>> 
>>>> So an interesting idea but they have some things they could improve.
>>> 
>>> I've been considering what one should run in parallel with the speed test 
>>> to get an impression if the speedtest impacts performance of other flows / 
>>> realtime flows, similar to what dslreports speedtest does.
>>> 
>>> I've considered running one or several simulated voip calls (50pps) and 
>>> record RTT, PDV, packet loss etc for this session.
>>> 
>>> It would be interesting to hear any suggestions people have for a fairly 
>>> simple codebase that does this that can be included in these kinds of test 
>>> clients (both server and client end, and of course one that protects 
>>> against reflection attacks etc).
>>> 
>>> iperf3 can be used for this, but from what I can see the iperf3 server 
>>> code isn't very friendly to multiple parallel tests or even resilient 
>>> against hung clients that doesn't close the test nicely.
>>> 
>>> I also considered using WebRTC or VoIP libraries, does anyone know what 
>>> RTT/PDV/packet loss data can be extracted from some common ones?
>> 
>> Pete coded up this wonderful tool for UDP-based latency testing; it's
>> even supported in Flent, and available on some (all?) the public-facing
>> servers:
>> 
>> https://github.com/heistp/irtt
>
> This reminds of a tangentially related question, do we/could we
> actually write the requested DSCP into the packet payloads so we could
> see/display dscp bleaching/remapping packets experience during
> transit? For irtt, ping and even netperf TCP/UDP flows?

irtt could definitely do this; not sure if it does. Ping and Netperf,
probably not...

-Toke

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Sebastian Moeller]

Hi Toke,


> On Sep 6, 2019, at 19:59, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> Hi Toke,
>> 
>>> On Sep 6, 2019, at 10:27, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> 
>>> Mikael Abrahamsson <swmike@swm.pp.se> writes:
>>> 
>>>> On Wed, 4 Sep 2019, Matt Taggart wrote:
>>>> 
>>>>> So an interesting idea but they have some things they could improve.
>>>> 
>>>> I've been considering what one should run in parallel with the speed test 
>>>> to get an impression if the speedtest impacts performance of other flows / 
>>>> realtime flows, similar to what dslreports speedtest does.
>>>> 
>>>> I've considered running one or several simulated voip calls (50pps) and 
>>>> record RTT, PDV, packet loss etc for this session.
>>>> 
>>>> It would be interesting to hear any suggestions people have for a fairly 
>>>> simple codebase that does this that can be included in these kinds of test 
>>>> clients (both server and client end, and of course one that protects 
>>>> against reflection attacks etc).
>>>> 
>>>> iperf3 can be used for this, but from what I can see the iperf3 server 
>>>> code isn't very friendly to multiple parallel tests or even resilient 
>>>> against hung clients that doesn't close the test nicely.
>>>> 
>>>> I also considered using WebRTC or VoIP libraries, does anyone know what 
>>>> RTT/PDV/packet loss data can be extracted from some common ones?
>>> 
>>> Pete coded up this wonderful tool for UDP-based latency testing; it's
>>> even supported in Flent, and available on some (all?) the public-facing
>>> servers:
>>> 
>>> https://github.com/heistp/irtt
>> 
>> This reminds of a tangentially related question, do we/could we
>> actually write the requested DSCP into the packet payloads so we could
>> see/display dscp bleaching/remapping packets experience during
>> transit? For irtt, ping and even netperf TCP/UDP flows?
> 
> irtt could definitely do this; not sure if it does. Ping and Netperf,
> probably not...

From man ping (on linux):
-p pattern
              You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing data-depen‐
              dent problems in a network.  For example, -p ff will cause the sent packet to be filled with all ones.

From man ping (macosx 10.14):
-p pattern
             You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing
             data-dependent problems in a network.  For example, ``-p ff'' will cause the sent packet to be filled
             with all ones.

With fping I come up empty

From man netperf (not sure how this wirks for servers):
-F fill_file
              Pre-fill the send buffers with data from the named file. This is intended to provide a means for  avoid-
              ing buffers that are filled with data which is trivially easy to compress. A good choice for a file that
              should be present on any system is this manpage - netperf.man.  Other files may be provided as  part  of
              the distribution.:
(so this would require us to distribute/generate 63 files for each dscp?)



From irtt help client:
--fill=fill    fill payload with given data (default none)
               none: leave payload as all zeroes
               rand: use random bytes from Go's math.rand
               pattern:XX: use repeating pattern of hex (default 69727474)
--fill-one     fill only once and repeat for all packets
--sfill=fill   request server fill (default not specified)
               see options for --fill
               server must support and allow this fill with --allow-fills

This might actually work, and if it required a packetdump to compare DSCP and intended DSCP that would already be much better than what we have today, no?



> 
> -Toke

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Hi Toke,
>
>
>> On Sep 6, 2019, at 19:59, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Sebastian Moeller <moeller0@gmx.de> writes:
>> 
>>> Hi Toke,
>>> 
>>>> On Sep 6, 2019, at 10:27, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>>> 
>>>> Mikael Abrahamsson <swmike@swm.pp.se> writes:
>>>> 
>>>>> On Wed, 4 Sep 2019, Matt Taggart wrote:
>>>>> 
>>>>>> So an interesting idea but they have some things they could improve.
>>>>> 
>>>>> I've been considering what one should run in parallel with the speed test 
>>>>> to get an impression if the speedtest impacts performance of other flows / 
>>>>> realtime flows, similar to what dslreports speedtest does.
>>>>> 
>>>>> I've considered running one or several simulated voip calls (50pps) and 
>>>>> record RTT, PDV, packet loss etc for this session.
>>>>> 
>>>>> It would be interesting to hear any suggestions people have for a fairly 
>>>>> simple codebase that does this that can be included in these kinds of test 
>>>>> clients (both server and client end, and of course one that protects 
>>>>> against reflection attacks etc).
>>>>> 
>>>>> iperf3 can be used for this, but from what I can see the iperf3 server 
>>>>> code isn't very friendly to multiple parallel tests or even resilient 
>>>>> against hung clients that doesn't close the test nicely.
>>>>> 
>>>>> I also considered using WebRTC or VoIP libraries, does anyone know what 
>>>>> RTT/PDV/packet loss data can be extracted from some common ones?
>>>> 
>>>> Pete coded up this wonderful tool for UDP-based latency testing; it's
>>>> even supported in Flent, and available on some (all?) the public-facing
>>>> servers:
>>>> 
>>>> https://github.com/heistp/irtt
>>> 
>>> This reminds of a tangentially related question, do we/could we
>>> actually write the requested DSCP into the packet payloads so we could
>>> see/display dscp bleaching/remapping packets experience during
>>> transit? For irtt, ping and even netperf TCP/UDP flows?
>> 
>> irtt could definitely do this; not sure if it does. Ping and Netperf,
>> probably not...
>
> From man ping (on linux):
> -p pattern
>               You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing data-depen‐
>               dent problems in a network.  For example, -p ff will cause the sent packet to be filled with all ones.
>
> From man ping (macosx 10.14):
> -p pattern
>              You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing
>              data-dependent problems in a network.  For example, ``-p ff'' will cause the sent packet to be filled
>              with all ones.

Yeah, but you can't read back the output...

> With fping I come up empty
>
> From man netperf (not sure how this wirks for servers):
> -F fill_file
>               Pre-fill the send buffers with data from the named file. This is intended to provide a means for  avoid-
>               ing buffers that are filled with data which is trivially easy to compress. A good choice for a file that
>               should be present on any system is this manpage - netperf.man.  Other files may be provided as  part  of
>               the distribution.:
> (so this would require us to distribute/generate 63 files for each dscp?)

We're already using -F /dev/urandom to prevent the netperf data from
being compressible... And also, this cannot be read back.


> From irtt help client:
> --fill=fill    fill payload with given data (default none)
>                none: leave payload as all zeroes
>                rand: use random bytes from Go's math.rand
>                pattern:XX: use repeating pattern of hex (default 69727474)
> --fill-one     fill only once and repeat for all packets
> --sfill=fill   request server fill (default not specified)
>                see options for --fill
>                server must support and allow this fill with --allow-fills

As above, we're doing --fill=rand today.

> This might actually work, and if it required a packetdump to compare
> DSCP and intended DSCP that would already be much better than what we
> have today, no?

Well, I'm sorta skeptical that anyone would actually look at those
packet dumps ;)

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Sebastian Moeller]

Hi Toke,


> On Sep 7, 2019, at 00:50, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> Hi Toke,
>> 
>> 
>>> On Sep 6, 2019, at 19:59, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> 
>>> Sebastian Moeller <moeller0@gmx.de> writes:
>>> 
>>>> Hi Toke,
>>>> 
>>>>> On Sep 6, 2019, at 10:27, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>>>> 
>>>>> Mikael Abrahamsson <swmike@swm.pp.se> writes:
>>>>> 
>>>>>> On Wed, 4 Sep 2019, Matt Taggart wrote:
>>>>>> 
>>>>>>> So an interesting idea but they have some things they could improve.
>>>>>> 
>>>>>> I've been considering what one should run in parallel with the speed test 
>>>>>> to get an impression if the speedtest impacts performance of other flows / 
>>>>>> realtime flows, similar to what dslreports speedtest does.
>>>>>> 
>>>>>> I've considered running one or several simulated voip calls (50pps) and 
>>>>>> record RTT, PDV, packet loss etc for this session.
>>>>>> 
>>>>>> It would be interesting to hear any suggestions people have for a fairly 
>>>>>> simple codebase that does this that can be included in these kinds of test 
>>>>>> clients (both server and client end, and of course one that protects 
>>>>>> against reflection attacks etc).
>>>>>> 
>>>>>> iperf3 can be used for this, but from what I can see the iperf3 server 
>>>>>> code isn't very friendly to multiple parallel tests or even resilient 
>>>>>> against hung clients that doesn't close the test nicely.
>>>>>> 
>>>>>> I also considered using WebRTC or VoIP libraries, does anyone know what 
>>>>>> RTT/PDV/packet loss data can be extracted from some common ones?
>>>>> 
>>>>> Pete coded up this wonderful tool for UDP-based latency testing; it's
>>>>> even supported in Flent, and available on some (all?) the public-facing
>>>>> servers:
>>>>> 
>>>>> https://github.com/heistp/irtt
>>>> 
>>>> This reminds of a tangentially related question, do we/could we
>>>> actually write the requested DSCP into the packet payloads so we could
>>>> see/display dscp bleaching/remapping packets experience during
>>>> transit? For irtt, ping and even netperf TCP/UDP flows?
>>> 
>>> irtt could definitely do this; not sure if it does. Ping and Netperf,
>>> probably not...
>> 
>> From man ping (on linux):
>> -p pattern
>>              You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing data-depen‐
>>              dent problems in a network.  For example, -p ff will cause the sent packet to be filled with all ones.
>> 
>> From man ping (macosx 10.14):
>> -p pattern
>>             You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing
>>             data-dependent problems in a network.  For example, ``-p ff'' will cause the sent packet to be filled
>>             with all ones.
> 
> Yeah, but you can't read back the output...

	Yes, unfortunatley.

> 
>> With fping I come up empty
>> 
>> From man netperf (not sure how this wirks for servers):
>> -F fill_file
>>              Pre-fill the send buffers with data from the named file. This is intended to provide a means for  avoid-
>>              ing buffers that are filled with data which is trivially easy to compress. A good choice for a file that
>>              should be present on any system is this manpage - netperf.man.  Other files may be provided as  part  of
>>              the distribution.:
>> (so this would require us to distribute/generate 63 files for each dscp?)
> 
> We're already using -F /dev/urandom to prevent the netperf data from
> being compressible... And also, this cannot be read back

	Well, we could do 8 bytes DSCP in ASCII followed by ~1498Bytes randomness, but really which uploads actually use compression?


> 
>> From irtt help client:
>> --fill=fill    fill payload with given data (default none)
>>               none: leave payload as all zeroes
>>               rand: use random bytes from Go's math.rand
>>               pattern:XX: use repeating pattern of hex (default 69727474)
>> --fill-one     fill only once and repeat for all packets
>> --sfill=fill   request server fill (default not specified)
>>               see options for --fill
>>               server must support and allow this fill with --allow-fills
> 
> As above, we're doing --fill=rand today.

	Sama as above, but maybe Pete could be convinced to do the read back of the first X bytes automatically.

> 
>> This might actually work, and if it required a packetdump to compare
>> DSCP and intended DSCP that would already be much better than what we
>> have today, no?
> 
> Well, I'm sorta skeptical that anyone would actually look at those
> packet dumps ;)

Oh, I promise I will do this, occasionally ;)

Best Regards
	Sebastian

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Hi Toke,
>
>
>> On Sep 7, 2019, at 00:50, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Sebastian Moeller <moeller0@gmx.de> writes:
>> 
>>> Hi Toke,
>>> 
>>> 
>>>> On Sep 6, 2019, at 19:59, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>>> 
>>>> Sebastian Moeller <moeller0@gmx.de> writes:
>>>> 
>>>>> Hi Toke,
>>>>> 
>>>>>> On Sep 6, 2019, at 10:27, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>>>>> 
>>>>>> Mikael Abrahamsson <swmike@swm.pp.se> writes:
>>>>>> 
>>>>>>> On Wed, 4 Sep 2019, Matt Taggart wrote:
>>>>>>> 
>>>>>>>> So an interesting idea but they have some things they could improve.
>>>>>>> 
>>>>>>> I've been considering what one should run in parallel with the speed test 
>>>>>>> to get an impression if the speedtest impacts performance of other flows / 
>>>>>>> realtime flows, similar to what dslreports speedtest does.
>>>>>>> 
>>>>>>> I've considered running one or several simulated voip calls (50pps) and 
>>>>>>> record RTT, PDV, packet loss etc for this session.
>>>>>>> 
>>>>>>> It would be interesting to hear any suggestions people have for a fairly 
>>>>>>> simple codebase that does this that can be included in these kinds of test 
>>>>>>> clients (both server and client end, and of course one that protects 
>>>>>>> against reflection attacks etc).
>>>>>>> 
>>>>>>> iperf3 can be used for this, but from what I can see the iperf3 server 
>>>>>>> code isn't very friendly to multiple parallel tests or even resilient 
>>>>>>> against hung clients that doesn't close the test nicely.
>>>>>>> 
>>>>>>> I also considered using WebRTC or VoIP libraries, does anyone know what 
>>>>>>> RTT/PDV/packet loss data can be extracted from some common ones?
>>>>>> 
>>>>>> Pete coded up this wonderful tool for UDP-based latency testing; it's
>>>>>> even supported in Flent, and available on some (all?) the public-facing
>>>>>> servers:
>>>>>> 
>>>>>> https://github.com/heistp/irtt
>>>>> 
>>>>> This reminds of a tangentially related question, do we/could we
>>>>> actually write the requested DSCP into the packet payloads so we could
>>>>> see/display dscp bleaching/remapping packets experience during
>>>>> transit? For irtt, ping and even netperf TCP/UDP flows?
>>>> 
>>>> irtt could definitely do this; not sure if it does. Ping and Netperf,
>>>> probably not...
>>> 
>>> From man ping (on linux):
>>> -p pattern
>>>              You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing data-depen‐
>>>              dent problems in a network.  For example, -p ff will cause the sent packet to be filled with all ones.
>>> 
>>> From man ping (macosx 10.14):
>>> -p pattern
>>>             You may specify up to 16 ``pad'' bytes to fill out the packet you send.  This is useful for diagnosing
>>>             data-dependent problems in a network.  For example, ``-p ff'' will cause the sent packet to be filled
>>>             with all ones.
>> 
>> Yeah, but you can't read back the output...
>
> 	Yes, unfortunatley.
>
>> 
>>> With fping I come up empty
>>> 
>>> From man netperf (not sure how this wirks for servers):
>>> -F fill_file
>>>              Pre-fill the send buffers with data from the named file. This is intended to provide a means for  avoid-
>>>              ing buffers that are filled with data which is trivially easy to compress. A good choice for a file that
>>>              should be present on any system is this manpage - netperf.man.  Other files may be provided as  part  of
>>>              the distribution.:
>>> (so this would require us to distribute/generate 63 files for each dscp?)
>> 
>> We're already using -F /dev/urandom to prevent the netperf data from
>> being compressible... And also, this cannot be read back
>
> 	Well, we could do 8 bytes DSCP in ASCII followed by ~1498Bytes
> 	randomness,

That would be less straight-forward, though, because then we can't just
pass in /dev/urandom. Besides, for TCP you can already identify the
packets based on the 5-tuple (since you're supposedly doing this
manually anyway ;)).

> but really which uploads actually use compression?

Tunnels, mostly...

>>> From irtt help client:
>>> --fill=fill    fill payload with given data (default none)
>>>               none: leave payload as all zeroes
>>>               rand: use random bytes from Go's math.rand
>>>               pattern:XX: use repeating pattern of hex (default 69727474)
>>> --fill-one     fill only once and repeat for all packets
>>> --sfill=fill   request server fill (default not specified)
>>>               see options for --fill
>>>               server must support and allow this fill with --allow-fills
>> 
>> As above, we're doing --fill=rand today.
>
> 	Sama as above, but maybe Pete could be convinced to do the read back of the first X bytes automatically.

Certainly not opposed to adding this support to Flent if it materialises
in irtt :)

-Toke

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Pete Heist]

> On Sep 7, 2019, at 1:12 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
>>>> From irtt help client:
>>>> --fill=fill    fill payload with given data (default none)
>>>>              none: leave payload as all zeroes
>>>>              rand: use random bytes from Go's math.rand
>>>>              pattern:XX: use repeating pattern of hex (default 69727474)
>>>> --fill-one     fill only once and repeat for all packets
>>>> --sfill=fill   request server fill (default not specified)
>>>>              see options for --fill
>>>>              server must support and allow this fill with --allow-fills
>>> 
>>> As above, we're doing --fill=rand today.
>> 
>> 	Sama as above, but maybe Pete could be convinced to do the read back of the first X bytes automatically.
> 
> Certainly not opposed to adding this support to Flent if it materialises
> in irtt :)

Coming into this late so haven’t parsed the full request, but irtt sends the requested DSCP value passed in via --dscp to the server during the handshake. It would be possible, though not very intuitive, to pull this out of the initial request packet, which contains type/value pairs encoded with varint style encoding: https://developers.google.com/protocol-buffers/docs/encoding.

The format of the request is “documented” in code in the bytes() method in params.go. Visually, the DSCP value is often the value 0x08, close to the end of the initial packet, following by the DSCP value as a signed varint. (Clearly, it would have made more sense if I’d just sent that as an unsigned byte instead of using a varint, let alone a signed one, but I just leaned on the binary package’s varint support, such as it is, so one has to grok this: https://developers.google.com/protocol-buffers/docs/encoding#signed-integers. I’ve added it to irtt’s todo list to clean this up before 1.0, which will mean a protocol version bump).

One unfortunate thing is that if the goal is to verify that DSCP values have not been modified along the way (without a pcap), afaik the receiver has no way of obtaining the received DSCP value in user space without opening up a raw socket and parsing the IP packet in full (requiring root). But, figuring it out from packet dumps would be possible. If I ever get around to adding irtt support to scetrace (https://github.com/heistp/scetrace), I could detect and count changes to dscp values there, though that’s a ways off at the moment.

Knowing all this, are there any simple changes I can make to get you what you need?

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 3454 bytes --]

Hi Pete,

If the PayPal ad of an irtt packet would contain the requested DSCP as ascci string (maybe starting with a string like "DSCP: 46: 101110 (EF)" in the first few bytes of the payload would make confirming bleaching/remapping from packetdumps relatively convenient, say just by looking at a packet in Wireshark and comparing the IP headers DSCP value with the string in the payload. Sure that is not automated, but would be great in at least allowing to test for bleaching in the packets received from a irtt server....

But, as much as I would like that feature, I believe the total audience will be quite small....

Best Regards
        Sebastian

On September 7, 2019 1:33:48 PM GMT+02:00, Pete Heist <pete@heistp.net> wrote:
>
>> On Sep 7, 2019, at 1:12 AM, Toke Høiland-Jørgensen <toke@toke.dk>
>wrote:
>> 
>>>>> From irtt help client:
>>>>> --fill=fill    fill payload with given data (default none)
>>>>>              none: leave payload as all zeroes
>>>>>              rand: use random bytes from Go's math.rand
>>>>>              pattern:XX: use repeating pattern of hex (default
>69727474)
>>>>> --fill-one     fill only once and repeat for all packets
>>>>> --sfill=fill   request server fill (default not specified)
>>>>>              see options for --fill
>>>>>              server must support and allow this fill with
>--allow-fills
>>>> 
>>>> As above, we're doing --fill=rand today.
>>> 
>>> 	Sama as above, but maybe Pete could be convinced to do the read
>back of the first X bytes automatically.
>> 
>> Certainly not opposed to adding this support to Flent if it
>materialises
>> in irtt :)
>
>Coming into this late so haven’t parsed the full request, but irtt
>sends the requested DSCP value passed in via --dscp to the server
>during the handshake. It would be possible, though not very intuitive,
>to pull this out of the initial request packet, which contains
>type/value pairs encoded with varint style encoding:
>https://developers.google.com/protocol-buffers/docs/encoding.
>
>The format of the request is “documented” in code in the bytes() method
>in params.go. Visually, the DSCP value is often the value 0x08, close
>to the end of the initial packet, following by the DSCP value as a
>signed varint. (Clearly, it would have made more sense if I’d just sent
>that as an unsigned byte instead of using a varint, let alone a signed
>one, but I just leaned on the binary package’s varint support, such as
>it is, so one has to grok this:
>https://developers.google.com/protocol-buffers/docs/encoding#signed-integers.
>I’ve added it to irtt’s todo list to clean this up before 1.0, which
>will mean a protocol version bump).
>
>One unfortunate thing is that if the goal is to verify that DSCP values
>have not been modified along the way (without a pcap), afaik the
>receiver has no way of obtaining the received DSCP value in user space
>without opening up a raw socket and parsing the IP packet in full
>(requiring root). But, figuring it out from packet dumps would be
>possible. If I ever get around to adding irtt support to scetrace
>(https://github.com/heistp/scetrace), I could detect and count changes
>to dscp values there, though that’s a ways off at the moment.
>
>Knowing all this, are there any simple changes I can make to get you
>what you need?

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 4377 bytes --]

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Pete Heist]

> On Sep 7, 2019, at 2:02 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Hi Pete,
> 
> If the PayPal ad of an irtt packet would contain the requested DSCP as ascci string (maybe starting with a string like "DSCP: 46: 101110 (EF)" in the first few bytes of the payload would make confirming bleaching/remapping from packetdumps relatively convenient, say just by looking at a packet in Wireshark and comparing the IP headers DSCP value with the string in the payload. Sure that is not automated, but would be great in at least allowing to test for bleaching in the packets received from a irtt server....
> 
> But, as much as I would like that feature, I believe the total audience will be quite small....
> 
> Best Regards
> Sebastian

You probably noticed, but it’s possible to fill the whole payload with the dscp byte, although the server has to allow that to put it in replies (requests can always contain it), for example:

irtt server --allow-fills=“*"

irtt client --dscp=0x10 -l 256 --fill=pattern:10 --sfill=pattern:10 localhost

If you still want the payload to be mostly random, I could potentially add a new fill mode which fills first with some arbitrary bytes (or a string) then the rest is random. I think in the case of flent, the irtt runner fills with random, but this only applies when there’s actually a payload (i.e. for the voip tests).

Re: [Cerowrt-devel] [Bloat] Ubiquiti Launches a Speed Test Network

[Valdis Klētnieks]

[-- Attachment #1: Type: text/plain, Size: 310 bytes --]

On Sat, 07 Sep 2019 14:02:35 +0200, Sebastian Moeller said:

> If the PayPal ad of an irtt packet would contain the requested (....)

And here I though the attempts to monetize DNS via wildcarding a TLD were over the top.

Either that, or somebody's phone tried to auto-cucumber something and failed...

[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]