From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id C7FB321F644 for ; Sat, 26 Jul 2014 13:54:51 -0700 (PDT) Received: from hms-beagle.home.lan ([217.231.210.84]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0LuPYt-1WT6pC2JuR-011nKO; Sat, 26 Jul 2014 22:54:41 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: Date: Sat, 26 Jul 2014 22:54:39 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: David Lang X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:AG0IYcXAd84OhwR5OB7rY3VNi4ILJMbWIS5PtYXrdW+t96bRceL OC3vm6WnyOBPA90X6NXIgkFLYN82EcNeaFYCKSeHjJXD5E5X+ZKymSQVDYMePqF5mKvX5vM ab1ChVrt+zannDv1n23FB9ZEIvdzBW7kRBA3sVY67a99HiOVUqL2OB35IH7yn4drzqWIwCP AA1jtRz4SSLLlgJ0apjZg== Cc: Wes Felter , cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] Ideas on how to simplify and popularize bufferbloat control for consideration. X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2014 20:54:52 -0000 Hi David, On Jul 26, 2014, at 22:21 , David Lang wrote: > On Sat, 26 Jul 2014, Sebastian Moeller wrote: >=20 >> Hi David, >>=20 >>=20 >> On Jul 25, 2014, at 22:57 , David Lang wrote: >>=20 >>> On Fri, 25 Jul 2014, Wes Felter wrote: >>>=20 >>>> The Netgear stock firmware measures bandwidth on every boot or link = up (not sure which) and I would suggest doing the same for CeroWRT. >>>>=20 >>>> Do you need to measure Internet bandwidth or last mile bandwidth? = For link bandwidth it seems like you can solve a lot of problems by = measuring to the first hop router. Does the packer pair technique work = on TDMA link layers like DOCSIS? >>>=20 >>> The trouble is that to measure bandwidth, you have to be able to = send and receive a lot of traffic. >>=20 >> Well that is what you typically do, but you can get away with = less measurement traffic: in an ideal quiescent network sending two = packets back to back should give you the bandwidth (packet size / = incoming time difference of both packets), or send two packets of = different size (needs synchronized clocks, then difference of packet = sizes / difference of transfer times). >=20 > Except that your ideal network doesn't exist in the real world. You = are never going to have the entire network quiescent, the router you are = going to be talking to is always going to have other things going on, = which can affect it's timing. Sure, the two packets a required per measurement, guess I would = calculate the average and confidence interval over several of these = (potentially by a moving window) to get a handle on the variability. I = have done some RTT measurements on a ADSL link and can say that = realistically one needs in the hundreds data points per packet size. = This sounds awe full, but at least it does not require to saturate the = link and hence works without dedicated receivers on the other end... >=20 >>> unless the router you are connecting to is running some sort of = service to support that, >>=20 >> But this still requires some service on the other side. You = could try to use ICMP packets, but these will only allow to measure RTT = not one-way delays (if you do this on ADSL you will find the RTT = dominated by the typically much slower uplink path). If network = equipment would be guaranteed to use NTP for decent clock = synchronization and would respond to timestamp ICMP messages with = timestamp reply measuring bandwidth might be =93cheap=94 enough to keep = running in the background, though. >> Since this looks too simple there must be a simple reason why = this would fail. (It would be nice if ping packets with timestamps would = have required the echo server top also store its incoming timestamp in = the echo, but I digress) >> I note that gargoyle uses a sparse stream of ping packets to a = close host and uses increases in RTT as proxy for congestion and signal = to throttle down stream link=85 >=20 > As you say, anything that requires symmetrical traffic (like ICMP = isn't going to work, and routers do not currently offer any service that = will. Well I think the gargoyle idea is feasible given that there is a = reference implementation out in the wild ;).=20 >=20 > you also can't count on time being synced properly. Quick testing today drove him that message (ICMP time requests = showing receive time before originating times, quite sobering). Naive me = had thought that NTP would guarantee <1ms deviation from reference time, = but I just figured it is rather low ms to 100ms, so basically useless = for one-way delay measurements for close hosts=85. > Top Tier companies have trouble doing that in their dedicated = datacenters, depending on it for this sort of testing is a non-starter Agreed. >=20 >>> you can't just test that link, you have to connect to something = beyond that. >>=20 >> So it would be sweet if we could use services that are running = on the machines anyway, like ping. That way the =93load=94 of all the = leaf nodes of the internet continuously measuring their bandwidth could = be handled in a distributed fashion avoiding melt-downs by synchronized = measurement streams=85 >=20 > Well, let's talk about what we would like to have on the router >=20 > As I see it, we want to have two services >=20 > 1. a service you send a small amount of data to and it responds by = sending you a large amount of data (preferrably with the most accurate = timestamps it has and the TTL of the packets it received) >=20 > 2. a service you send a large amount of data to and it responds by = sending you small responses, telling you how much data it has received = (with a timestamp and what the TTL of the packets it received were) >=20 > questions: >=20 > A. Protocol: should these be UDP/TCP/SCTP/raw IP packets/??? >=20 > TCP has the problem of slow start so it would need substantially more = traffic to flow to reach steady-state. >=20 > anything else has the possibility of taking a different path through = the router/switch software and so the performance may not be the same. You thing UDP would not work out? >=20 > B. How much data is needed to be statistically accurate? >=20 > Too many things can happen for 1-2 packets to tell you the answer. The = systems on both ends are multi-tasking, and at high speeds, scheduling = jitter will throw off your calculations with too few packets. Yeah, but you can (to steal an I idea from Rick Jones netperf) = just keep measuring until the confidence interval around the mean of the = data falls below a set magnitude. But for the purpose of traffic shaping = you do not need the exact link bandwidth anyway just a close enough = proxy to start the search for a decent set point from a reasonable = position. I think that the actual shaping rates need to be iteratively = optimized. >=20 > C. How can this be prevented from being used for DoS attacks, either = against the thing running the service or against someone else via a = reflected attack if it's a forgable protocol (i.e. UDP) Well, if it only requires a sparse packet stream it is not going = to be to useful for DOS attacks,=20 >=20 > One thought I have is to require a high TTL on the packets for the = services to respond to them. That way any abuse of the service would = have to take place from very close on the network. >=20 > Ideally these services would only respond to senders that are directly = connected, but until these services are deployed and enabled by default, = there is going to be a need to be the ability to 'jump over' old = equipment. This need will probably never go away completely. But if we need to modify DSLAMs and CMTSs it would be much nicer = if we could just ask nicely what the current negotiated bandwidths are = ;) >=20 >=20 > Other requirements or restrictions? I think the measurement should be fast and continuous=85 Best Regards Sebastian >=20 > David Lang