Snowden Used Cheap, Readily Available 'Web Crawler' Software to Collect NSA Files

[KoKo]

Snowden Used Cheap, Widely Available 'Web Crawler' Software to Collect NSA Files

Published on Sunday, February 9, 2014 by Common Dreams
Snowden Used Cheap, 'Web Crawler' Software to Collect NSA Files

Thousands of websites on Tuesday will take a stand against government surveillance
- Common Dreams staff
http://www.commondreams.org/headline/2014/02/09

Whistleblower Edward Snowden who alerted the world to America’s out of control National Security Agency reportedly used cheap and widely available software to scoop up thousands of files on the NSA’s online activities.

The New York Times Sunday quoted a senior US intelligence officials as saying: “We do not believe this was an individual sitting at a machine and downloading this much material in sequence,”adding that the process was “quite automated.”

The spider can be programmed to jump from website to website following embedded links and copying everything in its way. Snowden reportedly set the right algorithm for the software that indicated subjects and how far it was to follow the links. The whistleblower was reportedly able to gain access to 1.7 million files, including NSA’s internal “wiki" materials that are used to share data across the world.

Through his lawyer at the American Civil Liberties Union, Mr. Snowden did not specifically address the government’s theory of how he obtained the files, saying in a statement:

“It’s ironic that officials are giving classified information to journalists in an effort to discredit me for giving classified information to journalists. The difference is that I did so to inform the public about the government’s actions, and they’re doing so to misinform the public about mine.”

------------------

This coming Tuesday, February 11, thousands of websites will take a stand against government surveillance by plastering protests across their home pages.

------------------------

'The Day We Fight Back,' will give a big boost of support for the USA Freedom Act, which would end or curtail many of the most controversial surveillance programs at the National Security Agency (NSA) and elsewhere.



“The idea is to really harness the outrage of the Internet community in speaking out in one big voice on Feb. 11,” said Rainey Reitman, the director of activism at the Electronic Frontier Foundation.

According to Josh Levy of Free Press: "Since the first revelations last summer, hundreds of thousands of Internet users have come together online and offline to protest the NSA’s unconstitutional surveillance programs. These programs attack our basic rights to connect and communicate in private, and strike at the foundations of democracy itself. Only a broad movement of activists, organizations and companies can convince Washington to restore these rights.”



This work is licensed under a Creative Commons Attribution-Share Alike 3.0 License.

[xiamiam]

1. k and r..thanks.

[The The]

2. Linkbait

If a news website wants people in message boards to quickly re-post their articles, they must leave the words "reportedly" or "allegedly" off the headlines. The least accurate assertion portraying the allegation as fact is more exciting.

[uppityperson]

5. What do you mean, "linkbait"? Welcome to Du.

[reddread]

4. easy, and been done many times

kids have broken into government systems, received visits and gained notoriety for their antics.
When people wonder what could happen to influence and hobble action by their elected officials,
they dont need to go much further than what is being alleged here, and they can count on this being
the future of government oversight by people who were not elected, and most on this site would probably
prefer that these particular individuals didnt hold the reins. But they will, if they dont already.

[struggle4progress]

6. It certainly squelches the "whistle-blower" defense, if Snowden was automatically

finding and downloading 550K+ documents/month during his short stint in Hawaii: it works out to an average of about 12 documents per minute, 24 hours a day, 7 days a week, week after week after week

And how many of those documents could he have actually read?

There are 168 hours in a week, and he's gonna spend about 8 hrs/night asleep, leaving 112 hours. He also had a fulltime job, which leaves him about 70 hours, if we count (say) a fifteen minute one-way commute. Basic needs (dressing, showering and shaving and eating, &c) must chop the total down to perhaps 60 hours. He has to have clean clothes, grocery shop and cook and clean-up or grab take-out, gas the car, &c): say (generously) that leaves him about 55 hours. And there's the relationship with the girlfriend, who claims she didn't notice anything out of the ordinary, so beyond snoring beside her and maybe chatting while he gets ready for work or over meals, he's got to spend some quality time with her: watching TV or going to the movies with her, attending her performances, honey-do and other relationship maintenance: say that leaves him about 45 hours/week -- or 6 hours a day

In all he's got about 550 hours to review 1.7 million documents -- which works out to reviewing an impossible 50+ documents a minute. Of course, he didn't do that: he probably did mechanized searches of his downloaded collection: but let's ignore the time spent thinking up search criteria and programming the searches. At best, he'll skim about a document a minute visually -- and nobody can keep that up for 6 hours without becoming mentally exhausted: I'd guess the long-term average can't really be much better than 30 documents an hour, which puts him below 17K documents receiving even cursory examination

The conclusion is that Snowden glanced at no more than 1% of his download

[The The]

7. Are you saying that before this article, you considered him a whistleblower?

Or are web crawlers not necessary for you to consider him a traitor?

[struggle4progress]

8. I suppose that I should be flattered that you, barely two hours after registering here,

are interested in my prior views of Snowden

But my post #6 makes no reference to anything I might have said previously regarding Snowden, does not depend upon anything I might have said previously regarding Snowden, and so can be understood without any knowledge of anything I might have said previously regarding Snowden: it provides a comparison of the result, of a simple calculation, to numbers provided in the article linked by the OP

To state just one point more clearly for your benefit, the mass downloading of unread documents, and the transfer of those unread documents to third-parties, does not meet any standard definition of "whistle-blowing"

I should think it more courteous of you, were you to refrain from putting words in my mouth, as I am quite able to express my actual views clearly by myself, without the distortions introduced by your misreadings (or perhaps ideologically-motivated misrepresentations) of my text. If you read my #6 carefully, you will find that words such as "traitor" do not appear therein: the crime "treason" is, in fact, defined in Article III Section 3 Paragraph 1 of the Constitution, which also limits the conditions under which conviction for that crime may occur; the crime is codified at 18 USC 2381, incorporating the Constitutional language; and to my knowledge the criminal complaint against Snowden involves violation of 18 USC 641, 18 USC 793(d), and 18 USC 798(a)(3) but does not include violation of 18 USC 2381

[Madam45for2923]

9. interesting! It certainly squelches the "whistle-blower" defense, if Snowden was automatically

[Madam45for2923]

10. Snowden Used Cheap, Readily Available 'Web Crawler' Software to Collect NSA Files