Data-driven Analysis Debunks Claims NSA Out Of Control (Maybe)

[KittyWampus]

Note- This article explains the SCALE of data referenced in NSA conversations. Don't think many will agree w/authors ultimate conclusion but one can learn a lot reading this whole thing.

By David Gewirtz for ZDNet Government | August 19, 2013 -- 11:12 GMT (04:12 PDT)

snip
A single byte- roughly one character (like "B&quot could be compared to a penny.

A kilobyte is roughly a ten bucks

A megabyte is worth about a million pennies, or about $10,000 dollars

A gigabyte (which in video form will hold just about one episode of a TV show) would be a billion pennies, or about $10 million dollars

A terabyte is worth a trillion pennies. In dollars, that puts you in billionaire territory

A petabyte is one quadrillion pennies, or about $10 trillion dollars. $10 trillion is the entire Gross Domestic product of China and Japan...combined.

snip

the NSA "touches" about 30 petabytes …. it only selects for review about 7.3 terabytes ….

snip
Before we start playing with this number, let's add one more fact. This number is over the course of a year, while the other data we're looking at is over the course of a day.

snip

So now, we can start putting the heinousness in perspective. 32K times 2,776 errors is a little under 90 megabytes ….

To fit this into the daily numbers we've been working with, let's divide that 90 megabytes by 365. That gives us about 252K. In penny-per-byte terms, that's about $2,500 (or about the cost of one nicely equipped iMac).

In terms of dollars, which is the analogy we've been using throughout this article, the NSA mistakenly grabs the penny-per-byte data equivalent of an iMac as compared to the penny-per-byte equivalent of the overall net worth of Bill Gates plus Jeff Bezos.

The bottom line is this: the NSA runs about 30 quadrillion bytes through its systems each day. It records about 7 trillion of those bytes. It mistakenly records less than a megabyte a day — less than one MP3 worth of data per day.

Let's put it another way. When we talk about our goals for measuring excellent data center high-availability performance, we look for "five nines" of service availability, meaning that uptime is 99.999 percent. In terms of operating time, five nines means the network will be down all of 5 minutes and 26 seconds for the entire year.

If we picture the NSA's accuracy by comparing it to the commonly accepted IT goal of five-nines of high availability (or about five and a half minutes per year), the NSA's error rate (described in terms of time) would be 0.2649 milliseconds per year. That's not the Holy Grail of five nines of accuracy. That's more like twelve nines.

These numbers don't look to me like a heinous disregard for privacy on the part of the NSA's coders and systems engineers. Instead, it looks to me more like a triumph of IT and database engineering.

Of course, information like that doesn't cause outrage, it doesn't sell newspapers, and it doesn't generate page views. It's just accurate. Looking at actual data rather than breathless hyperbole paints a far clearer picture of the activities of America's most advanced technical intelligence gathering operation.

http://www.zdnet.com/data-driven-analysis-debunks-claims-that-nsa-is-out-of-control-special-report-7000019522/

[Vinnie From Indy]

1. Or, they could be lying again!

It could also be that the NSA simply farms out the data collection to the Brits and then they share information. In short, between the lies and the weasel words, who really knows how deeply they are breaking the law!

Cheers!

[KittyWampus]

3. The information came from an internal document. There's no way you read all that info.

[Vinnie From Indy]

4. So they produced a document that makes them look less sinister?

Sorry, but I will need a bit more independent scrutiny to be swayed.

[KittyWampus]

6. Um, no. Gerwitz is someone who understands IT and explains the scale of what happens.

And the document in question was not for public consumption.

[99th_Monkey]

29. In other words, only people who "understand IT" have a say in this.

[uponit7771]

2. Naw...naw, what people need to do is keep chasing NSA little green men cause if Greenwald

...said it and then put it in his book then that means it's true.

The HOF crew isn't going to believe any oversight from the NSA or the "gov = bad" folk in Washington.

If half their energy were put into making sure the 14 and 15th amendment were keep safe then we'd have a half a chance in 14

[MineralMan]

5. Stop Making Sense!

To steal a movie title. Most people have no understanding of the vast amount of data the NSA deals with, so they don't understand what a incredibly small amount is involved with this whole "kerfunkle." (credit where due for that word.)

A little understanding is not much, and it doesn't serve well.

[U4ikLefty]

7. Yeah, "most people" are too stupid

to know the NSA are good guys & NOT (I repeat NOT) spying on Americans.

[MineralMan]

8. OK. The NSA is the NSA.

It does good stuff and stuff that's questionable. Just like every government agency. Have a nice day.

[uponit7771]

20. Come on MM, you know the NSA was made to Supermans replacement... all perfect and shit

[uponit7771]

19. +1, this whole thing has been driven by general publics lack of knowledge of what computers can

...or can't do.

It's like a theme out of the X-Men or something

[railsback]

9. I still like how some believe the NSA writes all these internal secret reviews

to lie to themselves.. in secret. Makes about as much sense as the accusation being interpreted as 'fact' that the NSA is spying on every single person in the U.S.

[ljm2002]

10. 'the NSA "touches" about 30 petabytes ….' ...

...'it only selects for review about 7.3 terabytes …. '

Wow, I feel better already.

[KittyWampus]

11. When you know what terms mean, it becomes much less scary than some want to imply.

And "review" from what I've gleaned does NOT mean visually opened and read by NSA analysts.

[ljm2002]

12. Well KW...

...I won't be taking lessons from you on "what terms mean", thank you very much.

Got any maps for us today?

[KittyWampus]

23. I'm passing on relevant info from someone else. That you don't like relevant info says volumes about

you as does your obsession with maps…. a total non sequitor.

[ljm2002]

38. Who said that I "don't like relevant info"?

Again, your comprehension skills are shown to be lacking. On the contrary, I am glad to have the information. My comment noted two specific facts from your post, namely: 1 - how much data the NSA "touches" (i.e. has access to, can look at or hoover up anytime they choose), and 2 - how much data they currently save for review (i.e. actually scan). Both numbers are distressingly large.

If you do not want people to comment on information that you provide, then why provide it? What do you expect, a bunch of posts basically nodding in agreement with whatever point you think you were making? That seems a bit unrealistic.

Got any maps for us today?

[snooper2]

34. you want to come to a telecom 101 class?

[ljm2002]

39. Wow...

...like, man, I'm really impressed with your display of technological erudition there.

[GeorgeGist]

15. Ten Terabytes could hold the printed collection of the Library of Congress.

That's a lot of data.


http://whatsabyte.com/

[uponit7771]

22. Touches vs analysis?! See, sophistic language already!! I "touch" about a q123401234 of particles

...day and shit

[KittyWampus]

27. And you pretend to even know what is happening. But you enjoy creating a scary SCARY impression.

it's really too bad.

[ljm2002]

28. You think you are being clever...

...I am not impressed.

It was a simple factual statement. It is alarming how much data the NSA "touches" since it means they have access to that data and can decide to scoop it up any time they want to. Furthermore, my comment also applied to the amount of data they actually "review" which is still very large, and I don't believe for one single solitary minute that they need to review that much data in order to catch terrorists.

But you go ahead and keep whistling. If you don't think our privacy rights are of any consequence, that is your right.

[MFrohike]

13. What does mistakenly mean?

I really should ask what it implies. The statement that the NSA overstepped its boundaries tells me nothing of the location of those boundaries. All I know from this statement is the boundaries, whatever they were, were crossed. In effect, I know nothing about the actual operation of the NSA from this story. The author has underlain the story with his own assumptions of the basic goodness and lawfulness of the NSA in reaching his conclusion that it's not a big deal. These assumptions may be warranted, but the history of that particular agency is checkered at best.

In summary, this article is worthless. It's an exercise in bad analogies and basic math which don't deal with the underlying issue of exactly where the agency's boundaries lie. The author could have written about any random celebrity's dog and produced a more useful article.

[Raine1967]

33. Or you could read the WaPo article the author of the article was dressing:

http://www.washingtonpost.com/world/national-security/nsa-broke-privacy-rules-thousands-of-times-per-year-audit-finds/2013/08/15/3310e554-05ca-11e3-a07f-49ddc7417125_story.html

(it is linked in the article you claim as useless.)

If you know nothing about what is being discussed, I would be happy to help you understand with links. You are claiming ignorance and making judgements after saying you don't know what is going on. The article is NOT worthless -- you saying it is does not make it true.

Do some research. You might understand why this is being debated.

[MFrohike]

37. Internet condescension at its best

I was speaking only of the article linked. That being said, I'm familiar with the Post's article and it does not answer the questions I asked. If it had, I would not need to ask.

The NSA broke the rules a tiny fraction of the time? Great. What exactly are the rules? That's the question that does not get answered in any meaningful way. I see legal terms of art like "reasonable, articulable suspicion" tossed around, but that phrase by itself means nothing. Without context to actually define it, it's a meaningless phrase. The context of the compliance issues requires actually being able to know and understand the rules. That is why I called this article worthless and why the Post article is lacking.

By the way, I lifted the word mistakenly from the zdnet article to illustrate how it means nothing without the necessary context.

[Ichingcarpenter]

14. David Gewirtz Is A member of the FBI's InfraGard BTW

He is also director of the U.S. Strategic Perspective Institute

David is a member of FBI InfraGard, the Cyberwarfare Advisor for the International Association for Counterterrorism & Security Professionals



http://www.zdnet.com/meet-the-team/us/david-gewirtz/

http://en.wikipedia.org/wiki/InfraGard



He might have a slight bias. on where his bread is buttered.......you think?

Maybe not....lol but maybe

[GeorgeGist]

16. But he's cool ...

[Rex]

17. LOL!

Maybe?

[KittyWampus]

21. Referencing the SCALE of numbers has no bias. But aren't you clever in googling his name.

[Raine1967]

18. Really good article.

Disappointed at a few of the responses thus far.

I'm starting to believe anti-authoritarian means crapping on anything that might shed a little more education about things. You clearly are authoritarian as is ZDNET now.

Wow. People are really dismissing data and analysis.

[Ichingcarpenter]

24. He gets slammed in his comment section

for poor math and inaccuracies.

I read the article yesterday
Always check the comments and who the author is.

[KittyWampus]

26. Oh, the commentators have no bias. LOL! Most go after him for mentioning Facebook

which was a big mistake on his part.

[Ichingcarpenter]

36. i didn't read those

I read the rebuttals on the science, math and legality. Not the petty attacks.

I can megafilter too.


I covered some of these points a few days ago in this threads


http://www.democraticunderground.com/10023481445

[Raine1967]

30. I will.

However I am often disappointed in the comments section of most articles. The past few years comment sections have just become very ugly in general.

I also don't choose to have my opinion of an article swayed by the commentary of it. I read up-thread about the author. Right now I'm still inclined to believe his data. It's Data.

DATA. Dismissing data right out of the gate is rathr strange. Discussing the hypothesis of Data is another thing entirely.

There is too much purity crap going on here at DU with regard to the NSA Discussions. The guy writes for ZDnet -- am I now supposed to believe that they (and their employees) are not a good source? I'm not trying to be combative. There are layers to this story. Questioning everyone who actually tries to present another layer (that might question the NSA being all evil or all good or somewhere in between) doesn't seem to be helpful.

[kelliekat44]

25. Hell, I saw my car parked in our parking lot and my neighbor walking her dog on Google Earth.

So a satellite can snap me parking my car or take shots of my home and people entering or exiting and I should worry about NSA tapping my phone? Now folks can hack your own computer and watch you on your computer cam without you knowing it...and I just can't get excited about NSA surveillance. Sorry.

[frylock]

40. does google have the authority to detain you indefinitely?

[FreakinDJ]

31. 1 Byte of spying on innocent Americans is UnConstitutional

Or would you prefer to Shred the Constitution like a Drunken Freeper at a Bushco Party

[matt819]

32. Another apologist for wholesale surveillance of American citizens

This analysis seems to be basing the rate on the 30 quadrillion bytes the NSA "touches" per day, rather than the 7 trillion it collects. Minor point, though. The error rate is minuscule.

If the information released is true. Does anyone really believe it is?

And let's back up a bit. The larger argument is whether the NSA should be touching 30 quadrillion bytes per day and collection 7 trillion bytes per day of communications of one sort or another of American citizens.

The answer to that one is really quite simple. No, they shouldn't.

So, thanks for the arithmetic, but it still doesn't address the larger issue, and that is whether a democracy should have this sort of access to its citizens' communications.

[Warren DeMontague]

35. And how much of that data was fed to the DEA, who lied about it in court to prosecute drug users?

Got any numbers on that?