Cheap software helped Snowden “scrape” NSA’s networks (NYT)

[jsr]

http://www.nytimes.com/2014/02/09/us/snowden-used-low-cost-tool-to-best-nsa.html

Snowden Used Low-Cost Tool to Best N.S.A.
By DAVID E. SANGER and ERIC SCHMITT

WASHINGTON — Intelligence officials investigating how Edward J. Snowden gained access to a huge trove of the country’s most highly classified documents say they have determined that he used inexpensive and widely available software to “scrape” the National Security Agency’s networks, and kept at it even after he was briefly challenged by agency officials.

Using “web crawler” software designed to search, index and back up a website, Mr. Snowden “scraped data out of our systems” while he went about his day job, according to a senior intelligence official. “We do not believe this was an individual sitting at a machine and downloading this much material in sequence,” the official said. The process, he added, was “quite automated.”

The findings are striking because the N.S.A.’s mission includes protecting the nation’s most sensitive military and intelligence computer systems from cyberattacks, especially the sophisticated attacks that emanate from Russia and China. Mr. Snowden’s “insider attack,” by contrast, was hardly sophisticated and should have been easily detected, investigators found.

Moreover, Mr. Snowden succeeded nearly three years after the WikiLeaks disclosures, in which military and State Department files, of far less sensitivity, were taken using similar techniques.

[RC]

1. The NSA was so busy running rough shod over other people's security, they forgot their own.

This has been known for quite some time. At least since the time that the information Snowden had, came to light.

[X_Digger]

2. Done well, a scrape / spidering just looks like someone browsing.

Mimic the user-agent string of a standard browser, keep requests to a reasonable speed (don't get flagged as suspicious for overloading the network equipment), and it's damned hard to detect.

[WillyT]

3. K & R !!!

[KoKo]

4. Massive Arrogance Fail! By NSA?

I'm pretty clueless about tech stuff...but isn't "Web Crawler" something from way back? The name rings a bell from like 2000 or something?

[Downwinder]

5. Embarrassing when someone uses your own tools against you.