[Bloat] Can't Run Tests Against netperf.bufferbloat.net

[Rich Brown]

Hi all,

Thanks for the note. Yes, the netperf server at netperf.bufferbloat.net <http://netperf.bufferbloat.net/> is turned off. The VPS that runs it is consuming its bandwidth limit (4TB per month) at an ever-increasing rate. When that happens, my hosting service (Ramnode.com <http://ramnode.com/> - good guys, stable hosting, great tech support service) automatically turns off the VPS 'til the start of the next month.

In the distant past, the 4 TB sufficed for the entire month. More recently, I would occasionally get a 90% warning by the 25th or 26th of the month. Last month, I hit 90% on Jan 6th(!) so I shut off the netperf server so I could continue to work on it.

I briefly turned on netserver on the VPS today. At 08:15 today, the VPS control panel showed: 181.7 MB of 3.9 TB Used. At 09:16 today, it showed 46.3 GB. 46 GBytes in 1 hour => ~30 TB/month (!) 

I'm going to appeal to the group's collective wisdom to find a better solution.

Current mitigations:

- I use iptables to log all netperf connections. I see a pattern of certain IP addresses that seem to be firing off a test every five minutes, 24 x 7 for days at a time.

- A few times a month, I run a script (see findunfilteredips.sh in https://github.com/richb-hanover/netperfclean <https://github.com/richb-hanover/netperfclean>) that scans the log files to count netperf connections and to block devices (using iptables) that have made more than 5,000 connections in the last seven days. This helps, but only delays the inevitable.

Potential (additional) mitigations:

- We could change DNS to spread the load of netperf.bufferbloat.net <http://netperf.bufferbloat.net/> across our fleet of servers. (Researchers who need consistent results could still choose a specific server: netperf-east, netperf-west, etc.)

- I could automate the current script to look for heavy users every day or two.

- Maybe I'm doing iptables imperfectly - comments appreciated.

- I have toyed with the notion of tweaking the iptables rules to throttle heavy users (over a certain number of tests/connections per time-period). That way, the 24x7 people would receive, say 3kbps instead of the actual link speed. There are a couple difficulties:
	a) I don't want to inconvenience actual researchers/bufferbloat testers. When I test a connection, I typically make 3-10 tests in rapid succession before I go away. This looks an awful lot like the 24x7 folks, except that real testers stop after 15 minutes. Could iptables be tweaked to tell one from the other?
	b) When I looked into this, I realized I might need to move the VPS from OpenVZ (which has limited iptables capabilities - no 'ipset' for example) to KVM (which is full virtualization).

- I could just buy more bandwidth. Currently, I pay $194/year for this server with the 4TB limit. Additional bandwidth on this provider is $48/year per additional TB. But 30 TB/month would be pricey.

- I could move to a different hosting service where bandwidth is cheaper. (Any recommendations?)

- Other thoughts? 

Thanks.

Rich


> On Feb 5, 2020, at 3:15 AM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
> 
> Taran Lynn <taranlynn0 at gmail.com> writes:
> 
>> All of my attempts to run netperf (and flent) against
>> netperf.bufferbloat.net have failed for the past couple of weeks.
>> Netperf seems to work fine between my desktop and laptop, so I don't
>> think the issue is on my end. Can anyone verify if the server is up?
> 
> Rich, I think this one is yours? Did netserver die or something?
> 
> -Toke

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20200205/9af09906/attachment-0002.html>