[Cerowrt-devel] Fwd: You're Invited: DWeb Virtual =?utf-8?Q?Meetup=2014next?= Wednesday--Decentralized Storage =?utf-8?Q?=D83D=DCBE?= =?utf-8?Q?_=D83D=DCBB?=

[Valdis Kl=?utf-8?Q?=c4=93?=tnieks]

On Fri, 08 May 2020 16:40:20 -0700, Dave Taht said:

> while I'm referring to stuff that's actually fun and off-topic... I've long
> off been watching progress in this area, knowing that
> "fixing bufferbloat" is a key requirement for technologies like this to
> succeed.

Amen to that.  Probably around a decade ago, $DAYJOB at the time was approached
by a vendor who had a very compelling and cost-effective erasure-coded storage
product that would have allowed us to have the data at 3 sites, and total loss
of any one would be survivable.  Two sites were about 3 cable miles apart, and
the third was 95 cable miles and could only run at 10Gig speed back then.

And what sank the deal was uncertainty if we could meet throughput requirements
under heavy load with one site that distant.  (You'd be *amazed* what happens
in the first 3 minutes after semester add/drop goes live at midnight, when you
have 35,000 students. :)

The other great bugaboo - How do you back up a half billion files that total
12+ petabytes? (It doesn't help the solution space when the majority of it is
one research group that (a) will probably eventually data-mine their data for
$100M+ in research grants and also (b) the terms of some of the grants
specified a 30 year retention  on the data *and* some wicked nasty PII issues,
because the data includes identifiable video of people who had not consented to
be part of the research study...

Storage - it turns out you can't just throw 10 terabyte drives at the problem. :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cerowrt-devel/attachments/20200509/62cf8f63/attachment.sig>