Yottabytes--------YOTTABYTES?

[Are_grits_groceries]

The NSA is constructing a datacenter in the Utah desert that they project will be storing yottabytes of surveillance data. And what is a yottabyte? I’m glad you asked.

There are a thousand gigabytes in a terabyte, a thousand terabytes in a petabyte, a thousand petabytes in an exabyte, a thousand exabytes in a zettabyte, and a thousand zettabytes in a yottabyte. In other words, a yottabyte is 1,000,000,000,000,000GB. Are you paranoid yet?

The more salient question is, of course, what are they storing that, by some estimates, is going take up thousands of times more space than all the world’s known computers combined? Don’t think they’re going to say; they didn’t grow to their current level of shadowy omniscience by disclosing things like that to the public. However, speculation isn’t too hard on this topic. Now more than ever, surveillance is a data game. What with millions of phones being tapped and all data duplicated, constant recording of all radio traffic, 24-hour high definition video surveillance by satellite, there’s terabytes at least of data coming in every day. And who knows when you’ll have to sift through August 2007’s overhead footage of Baghdad for heat signatures in order to confirm some other intelligence?

http://www.impactlab.com/2009/11/02/nsa-to-store-yottabytes-of-surveillance-data/

And people wonder why the dots can't be connected???????? There are too many effing dots.
This is just the NSA. Who knows how much data is stored by the other agencies.

[HiFructosePronSyrup]

1. Reminds me of the scene in the Simpsons movie.

Where hundreds of NSA agents are sitting around in a big room listening to innane phone calls.

[NYC_SKP]

2. "Yotabytes", Plural??? Like more than one?, Like one yotabyte wouldn't do it?

Damn, who would ever have time to read, review, etc., all that data?

[Systematic Chaos]

3. What comes after that? Zenyatta Mondattabytes?

:scared:

[Stinky The Clown]

4. I was thnking "Yoda bites"

[roguevalley]

5. I would be like stacking all the world's insert mailers for grocery

stores and JC Penneys. You would have them but 99% is just crap.

[QC]

6. Yatta!

http://www.youtube.com/watch?v=rW6M8D41ZWU

[napoleon_in_rags]

7. +1 LOL! nt

[Webster Green]

8. Yikes!

That was really scary! :wtf:

[Kitty Herder]

11. LOL WTF?

[Occulus]

13. What the... what in.... but......

:wow:

My brain just broke. Did you see how wigged-out happy the audience was?

:wtf:

[dmr]

17. Sure did!

Didn't know which to be more puzzled over - the skinny, fig-leafed-underpant naked guys or the overly thrilled audience.

:crazy:

[givemebackmycountry]

9. I want a IPOD that holds yottabytes of data -

That would be cool.

Maybe.

[lapfog_1]

10. Well, without some major breakthroughs in storage

I highly doubt that they expect to store a "yottabyte" anytime soon.

Let's say that they can buy a 10 Terabyte storage unit (disk, tape, holographic block, whatever) for $100. That's not out of the question at this point, but also not exactly commodity pricing yet.

$100 x 100 (per petabyte) x 1000 (per exabyte) x 1000 (per zettabyte) x 1000 (per yottabyte) = $10,000,000,000,000.

That's 10 Trillion dollars.

Now, the NSA does have a large budget, but I suspect that $10T is out of their reach. especially since all you've done is buy storage, you haven't hooked it up to anything nor have you designed software that can use it (just how do you find anything in a yottabyte).

Also, that's 172 Terabytes for every person alive on this planet. With a modest amount of compression, that would be enough to have hi-def video of every minute of every day of your life stored for review (assuming you live to be 80 years old).

not to mention that all the disk and tape ever made, recorded at the highest density now within our technological grasp, doesn't even equal ONE zettabyte... and they are going to have the ability to store 1000 times that amount?

Someone is dreaming (or pulling your leg).

Take it from someone that built the government's largest computer storage system in 1995. Bigger than anything the DOE or the NSA dreamed of having. It will be more than a decade (my guess is 3 decades) before they can pull this off.

Not to mention that the article points out that there isn't enough electricity to have all that storage online, even if you could buy the disk drives. Of course, the simple answer to that is, you DON'T power it all up at once. I consulted briefly for Fermi national labs and they had this data collection problem (the Tevatron produces an enormous amount of data), and their solution was 1000s of external drives, each individually powered up by software... a little lag time to access your file, but hey, you only have to power maybe 100 drives at a time.

[cliffordu]

16. Outstanding illumination!!

very nice.

[Occulus]

12. I think that's enough to store a recording of every call made every day.

I could be wrong, but that is just a gratuitously ginormous data depot by any reasonable person's standards. The only reason I can think of to need that much storage space is it to record something frequently on an automated basis, and storing all telephone calls made daily in a compressed form is one of them.

Remember, there are no theoretical limits on how much storage space one can access at a given time. All you have to do to add more space is add more hardware. Also keep in mind that we're not dealing with high-definition audio streams here. Finally, note the pluralization of the word "yottabytes" in the OP. As in, more than one. They could be about to store five, or ten, or fifty, or one hundred YB, for all we know.

I would apply this even more readily to emails; text compresses quite readily. Possibly, they will be storing both.

"thousands of times more space than all the world’s known computers combined"

Oh, and I wouldn't imagine that they'll be using conventional hard drives, either. I bet they're solid state, to cut down on maintenance issues. Trust me, they have all their bases covered. Now check this out, from the link in the OP:

As for the medium on which the data might be stored on, that’s anybody’s guess. Whoever’s making the estimates is probably playing a bit fast and loose with exponential curves, but if any of the alternative storage technologies we cover here on CG are any indication, yottabytes won’t seem so big a few years from now.

They mean this in exactly the same sense that terabytes used to be very impressively huge. Today, I can go pick up a terabyte for less than $120. Who knows how huge our commonly available storage space will be after the next leap forward in data storage? Or is the NSA already using that next leap forward?

Utterly plausible, completely possible, apparently doable thin aluminum millinery. You gotta love it...

[lapfog_1]

19. Read my response upstream...

Out of the 7 billion people on this planet, say about 2 billion have phones or texting devices, and let's say that each of those people does 10 phone calls a day of a duration of, say, 5 minutes each (that's almost a hour on the phone for 2 billion people on the planet).

32kbits is a good enough audio pickup to record that phone call.

2 billion X 50 minutes (3000 seconds) x 365 (days a year) x 10 years x 32kbits = 79,671 exabytes or about 80 zettabytes.

Note that I didn't add text messages to the number (for the simple reason that even if you sent 100s of texts a day, it's "noise" in the calculation - your daily phone calls - using my ridiculously large assumptions - is 12,288,000 bytes or characters of data)

Using current technology an 80 zettabyte system would cost something on the order of $20 Trillion dollars.

$15,000 will buy a nice 60 TB storage system (or at least, if you hand me $15,000, I can build it for you). You need 17 such systems to get 1 Petabyte (well, very close to a PB). 17,408 to get to one exabyte, and 17,825,792 such systems to get to 1 zettabyte. The one zettabyte system would occupy (using standard 19" rackmount systems - each 60 TB server is 5U in height so 8 servers per rack, each rack takes up 12 sq ft -allowing for cabinet clearances on front and back of each rack) 26 million square feet of raised floor environmentally controlled computer room space. For comparison, the recently completed Burj Khalifa, the tallest building in the world, has 3.595 million sq feet. So you need the equivalent of 7.5 Burj Khalifas (obviously you don't need to build "up" so much if you locate somewhere where land is cheap). That's a lot of finished computer room floor space.

Then you need to replicate that 80 times.

So now take my assumptions and decide that they are wrong, that your average person on this planet doesn't spend 50 minutes in phone conversation, but rather 5 minutes, and that we don't need the last 10 years of phone calls, only the last 1 year. A two order of magnitude reduction in requirements, and you arrive at nearly the one zettabyte and $200 Billion cost. Doable, and by spreading the cost over a number of budget cycles (10 for example) you could hide such a project in the federal budget.

Of course, you would be much better off spending a fraction of that money on human intelligence and finding 95% of the "bad guys" and just recording what they do. In fact, there are probably less than, what, 50,000 radical jihadists in the world (even ones like the "Fruit of the Boom" bomber - who would have surfaced only after his father reported him). Hire 50,000 to be individually assigned to track their "potential terrorist" target. That's only about $4 Billion a year ($80K per tracker). They shadow each of the targets and report and evaluate on what they do every day, who they meet with, and only use the collection assets to track the phone calls and text messages of just these people. Much cheaper and probably much more effective than collecting the phone calls of the world.

And what about the 5% that we didn't track? Well, every so often, they will blow up a building or shoot up an army base or something. That's life. If we disengage with the middle east financially (quit buying their oil) but engage with them educationally (and not with Mormon missionaries)... maybe they would be motivated to be "radical" as much.

[Matariki]

14. The Akashic Records?

[ThoughtCriminal]

15. Maybe they're working on The 9 Yotta Names of God

And when they're done, without any fuss, the stars will go out. (apologies to A.C. Clarke)

[HysteryDiagnosis]

18. I'm sure in a couple of months it will all be able to fit on a

USB stick...