The Surveillance Engine: How the NSA Built Its Own Secret Google(firstlook.org)
firstlook.org
The Surveillance Engine: How the NSA Built Its Own Secret Google
https://firstlook.org/theintercept/article/2014/08/25/icreach-nsa-cia-secret-google-crisscross-proton
19 comments
People always assumed they did this but now we have proof.
I thought it was amazing that the government spent so much time discussing the call records being logged.. when they are doing so much worse. Maybe that's how they keep people focusing on what the government wants to talk about? (aka look over here, nevermind that thing over there...)
This find is way worse than call detail records..
I thought it was amazing that the government spent so much time discussing the call records being logged.. when they are doing so much worse. Maybe that's how they keep people focusing on what the government wants to talk about? (aka look over here, nevermind that thing over there...)
This find is way worse than call detail records..
I think it's a combination of things. On the one hand, the phone records issue is the most easily understood by the general public, and so it already had traction there. It sort of fell into the government's lap that they could focus all of their press' attention on that issue in particular.
I agree doubly so however - the phone records are near the very bottom of the list when it comes to the severity of what Snowden's(and possibly other's) revelations have brought to light.
I agree doubly so however - the phone records are near the very bottom of the list when it comes to the severity of what Snowden's(and possibly other's) revelations have brought to light.
It was pretty obvious that's why they were doing it. The FISA Amendments Act is also much worse than the 215 section of the Patriot Act - or the Executive Order 12333, which is even less talked about.
That doesn't mean the call records issue wasn't important. After all, they pick their torture and drone assassination targets based on that alone, in many cases. But considering the Internet is such a big part of everyone's lives these days, I think NSA getting a firehose from the big carriers and cable providers is a bigger deal, yet I've heard very little talk about denying NSA access to the undersea cables.
That doesn't mean the call records issue wasn't important. After all, they pick their torture and drone assassination targets based on that alone, in many cases. But considering the Internet is such a big part of everyone's lives these days, I think NSA getting a firehose from the big carriers and cable providers is a bigger deal, yet I've heard very little talk about denying NSA access to the undersea cables.
[deleted]
This has been released and is use by almost two dozen federal agencies and now is the first time we're hearing about it?
When folks tell us crazy things, like the government is tracking every place you go and your opinions through your cell phone and social networks, we're supposed to say something like "That's extraordinary. With extraordinary claims, we require extraordinary proof" Then, if they persist, we're supposed to say something like "Such a program would require far too many people to keep a secret. We couldn't even keep the atom bomb a secret. The government is terrible at keeping secrets. Such a claim is just too far-fetched."
These are the traditional things taught to people who are supposed to be clear-headed and rational. It's the way we engage crackpots without taking them too seriously.
These responses seem to have failed us miserably in the current circumstances. As it turns out, yes, that's what they were doing, and yes, it was extraordinary and required lots of people to keep incredible secrets. But it still happened.
These things keep happening in the realm of automated surveillance, both by the government and corporations (and worse, when corps do it and the govt scoops it all up later) that would have been considered completely whacked just ten years ago. The stuff of paranoid fantasies.
Our tools of rational inquiry have failed us.
When folks tell us crazy things, like the government is tracking every place you go and your opinions through your cell phone and social networks, we're supposed to say something like "That's extraordinary. With extraordinary claims, we require extraordinary proof" Then, if they persist, we're supposed to say something like "Such a program would require far too many people to keep a secret. We couldn't even keep the atom bomb a secret. The government is terrible at keeping secrets. Such a claim is just too far-fetched."
These are the traditional things taught to people who are supposed to be clear-headed and rational. It's the way we engage crackpots without taking them too seriously.
These responses seem to have failed us miserably in the current circumstances. As it turns out, yes, that's what they were doing, and yes, it was extraordinary and required lots of people to keep incredible secrets. But it still happened.
These things keep happening in the realm of automated surveillance, both by the government and corporations (and worse, when corps do it and the govt scoops it all up later) that would have been considered completely whacked just ten years ago. The stuff of paranoid fantasies.
Our tools of rational inquiry have failed us.
For people paying attention to the evidence, the writing has been on the wall for many years-- enough to build credible, defensible, strong suspicions, at least. Of course, these people have been slandered in perpetuity by the government, who stands to gain stability from the "rationalist" suspicions seeming ridiculous. The confirmatory value of these reveals cannot be understated: a subset of the people who have been much maligned as "conspiracy theorists" were in fact reasonable in their assumptions and correct in their hunches.
If these evidence-attenders and conspiracy theorists were maligned as being on the lunatic fringe up until recently and were fully exonerated, what else could they have been right about? This question has the government running scared, scrambling to reassure the public that the now-redeemed former crackpots are still off the rocker, and have nothing else that is relevant to say about things which the public doesn't yet believe.
The public failed these conspiracy theorists because they accepted the establishment friendly line that they were kooky-- the public accepted the false associations and intentional counfoundment made between people with legitimate but not mainstream ideas backed with evidence, and various cultish, indefensible fantasy theories like aliens controlling the government, and the moon landing being faked. Thankfully, the public received a gut shot when Snowden went public with the truth. Maybe next time people will listen a bit closer to voices that the government would love to suppress.
If these evidence-attenders and conspiracy theorists were maligned as being on the lunatic fringe up until recently and were fully exonerated, what else could they have been right about? This question has the government running scared, scrambling to reassure the public that the now-redeemed former crackpots are still off the rocker, and have nothing else that is relevant to say about things which the public doesn't yet believe.
The public failed these conspiracy theorists because they accepted the establishment friendly line that they were kooky-- the public accepted the false associations and intentional counfoundment made between people with legitimate but not mainstream ideas backed with evidence, and various cultish, indefensible fantasy theories like aliens controlling the government, and the moon landing being faked. Thankfully, the public received a gut shot when Snowden went public with the truth. Maybe next time people will listen a bit closer to voices that the government would love to suppress.
Tracking of information has been known about since the initial NSA whistleblowers after 9/11
I have not heard a single conspiracy theorist suggest any of the things disclosed by Snowden, and I have yet to see evidence of RFID chips and the government monitoring us through our microwaves, or that Person of Interest is based on a true story.
Extraordinary claims still require extraordinary evidence, and that evidence usually comes from someone involved, not from a random blog/site on the internet, or a youtube channel.
The one thing everyone misses with this, which is my main concern, is less that the entire NSA/Insert Alphabet Soup here can track me, but that any one individual within these organisations who decides they don't like you for any reason can majorly screw you over. It is not the whole that concerns me, it is the parts.
I have not heard a single conspiracy theorist suggest any of the things disclosed by Snowden, and I have yet to see evidence of RFID chips and the government monitoring us through our microwaves, or that Person of Interest is based on a true story.
Extraordinary claims still require extraordinary evidence, and that evidence usually comes from someone involved, not from a random blog/site on the internet, or a youtube channel.
The one thing everyone misses with this, which is my main concern, is less that the entire NSA/Insert Alphabet Soup here can track me, but that any one individual within these organisations who decides they don't like you for any reason can majorly screw you over. It is not the whole that concerns me, it is the parts.
You're up against a growing population of people who inherently think that "the government is tracking every place you go and your opinions through your cell phone and social networks" is NOT crazy.
Just curious, where is the bit about keeping track of your opinions? Surely it's been proposed(so has everything and the kitchen sink), but that doesn't mean it's effective or useful in any way.
Somewhat OT but its amazing out of 3 "FTEs" 1 was for design ("GUI") and this is what they came up with. [1] pg 22
https://firstlook.org/theintercept/document/2014/08/25/shari...
https://firstlook.org/theintercept/document/2014/08/25/shari...
Also notice the entire screen shot. Netscape on Windows XP in, best we can tell, sometime after March 2006.
Netscape. With an AIM launcher in the status bar. Accessing TOP SECRET//COMINT//NOFORN resources.
Netscape. With an AIM launcher in the status bar. Accessing TOP SECRET//COMINT//NOFORN resources.
This looks like Netscape 7 (http://en.wikipedia.org/wiki/Netscape_(version_7)#mediaviewe...), which had its final release in August 2004, not the original Netscape.
Would it have been more secure if they used IE 6?
#mediaviewer/File:Netscape_7_En.PNG){kind=link}
Would it have been more secure if they used IE 6?
Maybe all software released after this is infested with spyware and exploits, and we are just led to believe it's the other way around
[deleted]
It's kind of remarkable that they kept the lid on this on long as they did. With so many local/loosely-related agencies involved, I'm surprised somebody didn't leak this much sooner.
It clearly says only a few intelligence officers from the different agencies had access. So it's not like any ol' police officer of FBI agent had access to this at all times.
Or maybe they did, and the search page had been rebranded so the officers didn't know it was operated by the NSA. Perhaps "Fusion Center Search 9000" or similar.
Yes, I find it VERY hard to believe that this went on and nobody mentioned it at all. Not one disgruntled employee, not anyone mentioning it to a friend at a bar? No way.
Uhm. Just because you didn't believe it doesn't mean that it wasn't known. The CCC was talking about this stuff years ago, some of it conjecture, some of it information from old sources/anonymous sources and the like.
Now we have proof. So that's that.
Also, given the strong "disincentivization" for whistleblowing that is going on at intelligence agencies, I am not at all surprised that it took a few years before somebody went public with it.
Hell, it used to take decades before this kind of stuff became public. Just this year we were made aware that the West German spy agencies opened _all_ cross-border inter-German mail. You know, the kind of thing that we in West Germany were told only the East Germans would do. That was kept secret for more than 20 years after it stopped being done, even though thousands of people were involved.
Now we have proof. So that's that.
Also, given the strong "disincentivization" for whistleblowing that is going on at intelligence agencies, I am not at all surprised that it took a few years before somebody went public with it.
Hell, it used to take decades before this kind of stuff became public. Just this year we were made aware that the West German spy agencies opened _all_ cross-border inter-German mail. You know, the kind of thing that we in West Germany were told only the East Germans would do. That was kept secret for more than 20 years after it stopped being done, even though thousands of people were involved.
Well, it is easy to dismiss people saying kind of stuff as 'crackpots' or 'tin-foil-hatters'.
I personally keep to a simple rule. If something is technologically feasible[0], economically viable and someone has an incentive for it, it will happen, period. This rule of thumb successfully filters out weather control crackpottery while correctly identifying surveillance capabilities (and no, it's not hindsight bias; the only thing I was surprised about Snowden revelations was that it was underwhelming).
[0] - and keep in mind that pretty much anything is.
I personally keep to a simple rule. If something is technologically feasible[0], economically viable and someone has an incentive for it, it will happen, period. This rule of thumb successfully filters out weather control crackpottery while correctly identifying surveillance capabilities (and no, it's not hindsight bias; the only thing I was surprised about Snowden revelations was that it was underwhelming).
[0] - and keep in mind that pretty much anything is.
In what way were the revelations underwhelming? For me overwhelming is their ability to coerce companies and their willingness to do so, in spite of the potential for a loss of trust in US companies. Sometimes such maneuvers can have a big cost long term - I can imagine at least governments switching software, hardware or services to more local providers. I mean, think about backdoors - you may trust the NSA, but if backdoors exist, it is only a matter of time before they get discovered by other parties. Countries now also have the ultimate argument for the balkanization of the Internet. This can't be good and all of this distinction between US citizens and foreigners doesn't mean anything for foreigners.
> In what way were the revelations underwhelming?
The technological ones. I was expecting (and I still believe) that they're tapping much better into the Internet that it is revealed, and that there are many more hardware backdoors out there.
> Countries now also have the ultimate argument for the balkanization of the Internet.
I agree. This is very bad. I didn't say I was underwhelmed with outcome (though I am with the reaction of public and of Internet companies), just with the revealed capabilities of the NSA.
The technological ones. I was expecting (and I still believe) that they're tapping much better into the Internet that it is revealed, and that there are many more hardware backdoors out there.
> Countries now also have the ultimate argument for the balkanization of the Internet.
I agree. This is very bad. I didn't say I was underwhelmed with outcome (though I am with the reaction of public and of Internet companies), just with the revealed capabilities of the NSA.
I wouldn't be surprised if they talked about it but if it's rebranded it could be seen as "Yet Another Tool That IT Provides". People talk about their favourite consumer apps, but only ever complain about the shit they're forced to use at work. In this case, the tool is working fine so there's no need to sing its praises. If it was a shitty search engine, they'd be telling everyone "we collect all this data and for the life of me, we can't even search it! what's the point of violating the constitution at all?!"
When your livelihood is based entirely on your security clearance and your ability to keep working in the security industry you make it a very big point to not talk about this stuff outside of work. I had a friend who went to work for a 3-letter agency and started consistently avoiding bars/drinking because he was afraid he'd say something. This is exactly why whistleblowing is a problem in the intelligence community - they'll sue you to oblivion and/or imprison you for saying anything at all.
My guess is a lack of understanding.
Wonder how and where they recruit their top-talent. Pretty much every top scorer in my school went to Facebook/Google/Msft/Amzn. The mediocre ones went to Northrop Grumman/Raytheon/Rockwell and the like.
The Company doesn't exactly encourage recruits to advertise their new position. You mostly see people just vanish for a while. Young recruitment is largely done through direct faculty recommendations at schools, so it's more of a "don't find us, we'll find you" proposition.
Yup. I had a friend who spent some time there. Easily the smartest kid in my major. Hired due to a direct referral from faculty. It gets scary, once you see how many math professors have security clearances.
> It gets scary, once you see how many math professors have security clearances.
Makes sense, though. The people who are the best at math are going to be the people who are the best at grokking and expanding upon the existing understanding of the math involved in cryptography.
Makes sense, though. The people who are the best at math are going to be the people who are the best at grokking and expanding upon the existing understanding of the math involved in cryptography.
Yeah, my mom majored in math and minored in Russian. She got the NSA offer in her last semester. But she wanted to teach, and thought the whole thing was more than a little spooky. And indeed, it was a faculty referral that prompted the interview offer.
Exactly, a good friend of mine in college was recommended for a job at the NSA by one of my professors in early 2001. He went to having a heavy online presence and being a constant in our social circles to completely disappearing. His personal site went dormant, he stopped showing up on irc, abruptly dumped his long time girlfriend. My roommate bumped into the guy at a grocery store in Virginia but other than that he doesn't exist to anyone who knew him for years or online.
I think the bridge between math and programming is relatively small, so if you have the most mathematicians in the world, you probably have some pretty good programmers too.[1]
But for contractor, there are job boards for people with clearances.[2] Although you get a pay boost of at least 10-15k, I'm sure the bureaucracy is awful.
I also wouldn't assume that all the top people go to biz. Some very talented people work in gov and academia. Not everyone is guided primarily by money & scores.
[1] http://www.math.uh.edu/~tomforde/Web/Jobs.html [2] http://www.clearedconnections.com/
But for contractor, there are job boards for people with clearances.[2] Although you get a pay boost of at least 10-15k, I'm sure the bureaucracy is awful.
I also wouldn't assume that all the top people go to biz. Some very talented people work in gov and academia. Not everyone is guided primarily by money & scores.
[1] http://www.math.uh.edu/~tomforde/Web/Jobs.html [2] http://www.clearedconnections.com/
For the same reason some MDs go to work for Médecins Sans Frontières instead of a top-paying job at a private hospital or partnership. A sense of duty. I grew up in a small town dominated by military/industrial companies. A lot of top talent moves there because they believe working for the government is what they should do.
The best programmer I know works for Northrop, because you can make >$100K and live in a rural or semirural area (near an AFB), you can have $1000/mo rent, vs living in SF making $150K and having massively higher expenses and have to be far away from "home".
Sure but isn't life a great deal more boring out there? It depends on a person's priorities.
My experience in school (and grad school) was similar, so I've been curious about the same thing. Perhaps, they only recruit folks from the industry after they've worked for a while and have specialized?
Maybe they recruit people while they are employed at FB/GG/Msft/Amz, you know, to help with national security while also receiving a second wage.
It would be amazingly naive to think that the various alphabet agencies have not made it their business to penetrate a wide variety of business, government, and NGO organizations. If it makes you feel any better, the Russians have plants in FB/Google/Amazon too.
I imagine that you'll get downvoted to hell for saying that, but if I was the NSA I'd do exactly this. I'd also threaten you if you refused to comply.
I would already know exactly which employee to target based on your dubious interests gleaned from your internet searches.
I would already know exactly which employee to target based on your dubious interests gleaned from your internet searches.
And not a single one of them has spoken up about being coerced into compromising their employer (a big one, with lots of lawyers) by the government?
If someone has been approached by a powerful government agency that threatens them with embarrassing information concerning their private life, would they volunteer to share that coercion in public (and be believed)?
You'd have to have pretty big balls to fuck with a three letter agency like that.
You'd have to have pretty big balls to fuck with a three letter agency like that.
These recruiters find you, you dont find them. I doubt that any of your classmates would be so dumb as to tell everyone they got hired by a 3 letter agency. They probably are looking for more seasoned devs than the typical college grad.
Also you have to be an American citizen with parents who live in America. At least I know this is the case for civilians who want high security clearances.
Also you have to be an American citizen with parents who live in America. At least I know this is the case for civilians who want high security clearances.
I am guessing they get them young.Who sponsors MATHCOUNTS,AMC and other contests for highly gifted kids?Raytheon,Northrop Grumman.
Is it just me, or is it tragicomical that I can't access firstlook from a government network?
According to the Intercept, the U.S Military has banned employees from visiting First Look because there could be classified information on the site. [1] [2] (extra source since you can't see the real one..)
Maybe that is what you are experiencing?
[1] https://firstlook.org/theintercept/2014/08/20/u-s-military-b...
[2] https://thebulletwire.com/u-s-military-bans-glenn-greenwalds...
Maybe that is what you are experiencing?
[1] https://firstlook.org/theintercept/2014/08/20/u-s-military-b...
[2] https://thebulletwire.com/u-s-military-bans-glenn-greenwalds...
The U.S. DoD is banned from accessing firstlook.org (among others) from government-owned computers. In fact a memo was sent explicitly permitting DoD members from accessing these websites from personally-owned systems.
There is a memo from a few years ago that prohibited accessing Wikileaks from personally-owned computers, but the order was rescinded.
There is a memo from a few years ago that prohibited accessing Wikileaks from personally-owned computers, but the order was rescinded.
That's similar to the wikileaks ban. I don't understand this policy: If something is accessible virtually from everywhere, what do you gain by not allowing your employees to access the info?!
The only thing I can think of, is internal issues at political level (who said what lies to whom to achieve more funding about his work, etc.) which even that won't take long before the interested party eventually finds out.
The only thing I can think of, is internal issues at political level (who said what lies to whom to achieve more funding about his work, etc.) which even that won't take long before the interested party eventually finds out.
It perpetuates the mythology of classification and information control. If you have clearance, you can lose it by accessing classified information improperly. That would probably mean losing your job. If you don't have clearance, you should be considering all the information you can get your hands on. Not doing so, would be poorly doing your job.
nicely explained, thanks :-)
Of all the people I would expect to be following the rules (even the "stupid" ones), it would be actual employees of the government and military itself.
Once the military decides that they get to ignore laws which are silly, they will decide they can ignore laws which are unenforceable in practice, and then the Republic will fall.
You either have a culture of civilian control of the military, and a nation of laws, or you don't. The Church Committee is what resulted from the last time the Intelligence Community decided that laws are silly; is that what you want to go back to? Before you answer, keep in mind that the NSA was actually following the laws this time, so the situation as you see it post-Snowden could easily have been far worse.
Once the military decides that they get to ignore laws which are silly, they will decide they can ignore laws which are unenforceable in practice, and then the Republic will fall.
You either have a culture of civilian control of the military, and a nation of laws, or you don't. The Church Committee is what resulted from the last time the Intelligence Community decided that laws are silly; is that what you want to go back to? Before you answer, keep in mind that the NSA was actually following the laws this time, so the situation as you see it post-Snowden could easily have been far worse.
I guess it's always cool to learn about top secret stuff that spies do, but I don't get why the specifics of how their search engine works is significant. Am I just missing something?
This, as with most of Greenwald's work on this subject, has been a slow effort to educate people about a very technical (for the average person) topic. Building on previous reports that showed the type of activity and the threat it can cause, this installment is about establishing the pattern of illegality.
If the NSA could justify the time and expense to make a search tool, there must be a lot of searching needs... which aren't legal for the NSA to send to the domestic-oriented agencies.
Also, more explicitly-stated details about how powerful "metadata" can be. Obvious to anybody who understands INNER JOIN, but this is still new for most people.
//Of course, I'm sure all this ignoring of warrants and the 4th Amendment, to quote a DOJ lawyer in the recent EFF mess regarding Jewel v NSA, is all for the "national security of this company". (emphasis mine)
If the NSA could justify the time and expense to make a search tool, there must be a lot of searching needs... which aren't legal for the NSA to send to the domestic-oriented agencies.
Also, more explicitly-stated details about how powerful "metadata" can be. Obvious to anybody who understands INNER JOIN, but this is still new for most people.
//Of course, I'm sure all this ignoring of warrants and the 4th Amendment, to quote a DOJ lawyer in the recent EFF mess regarding Jewel v NSA, is all for the "national security of this company". (emphasis mine)
> Also, more explicitly-stated details about how powerful "metadata" can be. Obvious to anybody who understands INNER JOIN, but this is still new for most people.
The power of metadata is implicit. No one at NSA is arguing that metadata isn't powerful, that's the whole reason they're collecting it.
What the NSA is arguing is against the meme that they're trodding roughshod over the Constitution and illegally scooping up data (as they have done in the past). On the contrary, this time they've been very careful to try to remain within the law, and one of those arguments is by pointing out that, according to the law, some types of information are less protected than others.
Greenwald can educate all he wants, but the problem isn't that Americans are stupid or whatever you all seem to think. Americans on the large have been assuming the NSA is doing far worse than what Snowden accused them of, because Americans (on the large) expect that the NSA (or some other branch of the government) is actually able to intercept and listen in on "those jihadis' phone calls and emails".
You're talking about a generation of Americans who grew up listening to PBS stories hounding the Federal government agencies of the Bush and Clinton Administrations for "not connecting the dots", and for having a "lack of imagination", and for "treating terrorism like a law enforcement" problem and for being hamstrung by a bunch of civil liberties protections.
This is by and large the same generation that decided to vote against privacy when they bought into to having their digital lives collectively managed by Google, Apple, Microsoft and Facebook. Explain the low-level bits and bytes to them all you want, but they've already voted on the big picture they want to see.
The power of metadata is implicit. No one at NSA is arguing that metadata isn't powerful, that's the whole reason they're collecting it.
What the NSA is arguing is against the meme that they're trodding roughshod over the Constitution and illegally scooping up data (as they have done in the past). On the contrary, this time they've been very careful to try to remain within the law, and one of those arguments is by pointing out that, according to the law, some types of information are less protected than others.
Greenwald can educate all he wants, but the problem isn't that Americans are stupid or whatever you all seem to think. Americans on the large have been assuming the NSA is doing far worse than what Snowden accused them of, because Americans (on the large) expect that the NSA (or some other branch of the government) is actually able to intercept and listen in on "those jihadis' phone calls and emails".
You're talking about a generation of Americans who grew up listening to PBS stories hounding the Federal government agencies of the Bush and Clinton Administrations for "not connecting the dots", and for having a "lack of imagination", and for "treating terrorism like a law enforcement" problem and for being hamstrung by a bunch of civil liberties protections.
This is by and large the same generation that decided to vote against privacy when they bought into to having their digital lives collectively managed by Google, Apple, Microsoft and Facebook. Explain the low-level bits and bytes to them all you want, but they've already voted on the big picture they want to see.
I think that's the key insight. It may not be legal for NSA to look at records of US citizens, but build a search engine and there are other departments for whom it is legal to do that. NSA just becomes the facilitator or router of the data.
Is that even what's being described? I don't see any reason why it would be bad to let the DEA search an NSA database of intelligence about foreign suspects.
The problem is (as it's always been) that they aren't just criminal suspects and they aren't all foreign. But that was the problem whether or not the NSA built a "Google."
The problem is (as it's always been) that they aren't just criminal suspects and they aren't all foreign. But that was the problem whether or not the NSA built a "Google."
Also, this article goes over "parallel construction" quite a lot. Although we've heard about it a lot in the past year, this leak shows how and what kind of data NSA is sharing with DEA, FBI, etc.
It's even more important for us to know what kinds of data about us domestic law enforcement has access to, than it is to know what foreign intelligence has access to.
It's even more important for us to know what kinds of data about us domestic law enforcement has access to, than it is to know what foreign intelligence has access to.
Glenn Greenwald got a treasure trove and it's time to cash in.
Glenn Greenwald has a lot of Powerpoint slides. Am I the only one who wants to see the source code for ICREACH, and all the other ALLCAPS projects?
I wonder if that's a line even Snowden wouldn't cross. It risks those tools falling into the "wrong hands", and is the most extreme way he could be said to be aiding terrorists by alerting them of our intelligence capabilities (which he is already accused of.)
Its not new news that Google was contracted to build the search engine[1].
"Google was also contracted to provide the search technology and servers which makes up “Intellipedia”, a massive, incredibly secure database through which over 100,000 United States spies and intelligence professionals from 16 government agencies share information. And the NSA has purchased numerous servers from Google, utilizing their search technology to analyze and organize massive amounts of secure data all over the world."
At this point I think it safe to assume its the exact same code Google used in 2010, just a bit dumber, since Googles search engine only learns by being used, and since there are far fewer using it, it learns slower.
[1]http://questioneverything2012.wordpress.com/2012/03/14/googl...
"Google was also contracted to provide the search technology and servers which makes up “Intellipedia”, a massive, incredibly secure database through which over 100,000 United States spies and intelligence professionals from 16 government agencies share information. And the NSA has purchased numerous servers from Google, utilizing their search technology to analyze and organize massive amounts of secure data all over the world."
At this point I think it safe to assume its the exact same code Google used in 2010, just a bit dumber, since Googles search engine only learns by being used, and since there are far fewer using it, it learns slower.
[1]http://questioneverything2012.wordpress.com/2012/03/14/googl...
The claim that wordpress.com-hosted blog makes isn't sourced. To further stretch its credibility, it also goes on to describe how Google was actually funded by secret CIA seed money and implies a Bilderberg Group controlling interest of Google.
So it is news, to those of us who haven't read this blog that makes an unsourced claim.
Besides, from the Greenwald docs, we've seen that Intelipedia is simply a giant Wikimedia instance, nothing fancy.
So it is news, to those of us who haven't read this blog that makes an unsourced claim.
Besides, from the Greenwald docs, we've seen that Intelipedia is simply a giant Wikimedia instance, nothing fancy.
Why wouldn't Snowden cross that line, given all the other ones he crossed? I mean, blowing PRISM by itself was far more than "alerting them of our intelligence capabilities", so he's already crossed the Rubicon on that topic.
How was exposing PRISM worse? I'll admit to being biased in his favor, and somewhat ignorant of what exactly was disclosed and how it could help the enemy, so I welcome some input on that. My starting position is that I don't see how knowing that, say, Yahoo is tapped, has changed the bad guys' day-to-day operations to the degree that you could reasonably say Snowden significantly aided them. Or even if it did, it falls under the category of 'merely' "alerting them of our intelligence capabilities" which we both agree was a "line Snowden crossed."
Anyway, however damaging it was, my point is that some slides with a bunch of code names, "data-sharing partners", hints at legal justifications and the bureaucracy set up to carry this out, and maybe some high-level descriptions of how the systems work, is a lot different than releasing the actual code that does it, private keys to the servers involved, etc.
It's not like graph theory and data mining are elite subjects only NSA knows, so some high-level descriptions aren't going to help the bad guys much. Handing over complete source code would.
Anyway, however damaging it was, my point is that some slides with a bunch of code names, "data-sharing partners", hints at legal justifications and the bureaucracy set up to carry this out, and maybe some high-level descriptions of how the systems work, is a lot different than releasing the actual code that does it, private keys to the servers involved, etc.
It's not like graph theory and data mining are elite subjects only NSA knows, so some high-level descriptions aren't going to help the bad guys much. Handing over complete source code would.
Revealing the existence isn't anywhere the same as revealing the guts of its operations, which for all we know could allow compromise or direct access.
PRISM is bad enough in US hands, but can you imagine if Oceania/Arstotzka got their hands on the data? (irrelevant countries as to not have to argue that)
PRISM is bad enough in US hands, but can you imagine if Oceania/Arstotzka got their hands on the data? (irrelevant countries as to not have to argue that)
I'd be more interested in seeing all the open source software it uses, as I'm sure it does and whether it has funded open source software development which I am not sure of but I can imagine happening indirectly and whether the programmers were aware or not which I imagine not happening.
Imagine if hackers were to somehow to find out where this tool is being hosted and then brute force their way in? Presumably such a tool is heavily secured and off the grid, but if it is somehow accessible from the Internet and an attacker were to find out where, it's only a matter of time now that the cat is out of the bag. Could you imagine hackers having access to troves of metadata and information like that? It's a scary thought.
I'm surprised such a thing took so long to be revealed. If you've got as much data as the NSA has, wouldn't you want a Google like search engine to be able to search through it? It makes so much sense which is why I am surprised some people are surprised about this.
I'm surprised such a thing took so long to be revealed. If you've got as much data as the NSA has, wouldn't you want a Google like search engine to be able to search through it? It makes so much sense which is why I am surprised some people are surprised about this.
Furthermore, think of how profitable surveillance on certain individuals could be. Could a military contractor not have a thriving side business funneling out intelligence on innocent American targets? Could multiple people not be hired just to clear tracks and make justifiable excuses for accessing the records?
If Snowden, as one person, accomplished what he did with (the majority of Americans would say) good intentions, imagine what a team of people, who were just as smart if not smarter, could do if they didn't have morals guiding their actions. Imagine how well a team could cover their tracks, dot all the I's, and cross all the T's, compared to just one guy.
This is all common sense, but it bears repeating.
If Snowden, as one person, accomplished what he did with (the majority of Americans would say) good intentions, imagine what a team of people, who were just as smart if not smarter, could do if they didn't have morals guiding their actions. Imagine how well a team could cover their tracks, dot all the I's, and cross all the T's, compared to just one guy.
This is all common sense, but it bears repeating.
I'd just play the market. Find dirt on CEOs, short their stock and pass the dirt to the media.
Brought to you by meta data commodities and pattern recognition limited - don't be evil. just be data.
I believe its really telling that, rather than take the tactic of fully promoting open society, the inclination is towards more and more secrecy. Like, dire, utter secrecy. Kill someone-style, secrecy.
Imagine the other end of the scale - where in fact every detail about everyones lives is wide open and available for everyone and anyone to access. Willingly. Freely. A new order of celebrity: total telepathy.
Do you think we'd be dealing with terrorism, then? Would there be the idealist, killing souls, for a little private time?
Imagine the other end of the scale - where in fact every detail about everyones lives is wide open and available for everyone and anyone to access. Willingly. Freely. A new order of celebrity: total telepathy.
Do you think we'd be dealing with terrorism, then? Would there be the idealist, killing souls, for a little private time?
Maybe there was an earlier release I missed but this report looks to be the first to show the extent of the call record data being collected. SIM, IMEI, Lat/Lng and more. Logged for every American, every day, all since 2007. And now shared outside of the military context it was gathered and authorized under to civilian agencies.
It's possible the program has been reigned in since these slides were authored but that seems unlikely.
It's possible the program has been reigned in since these slides were authored but that seems unlikely.
I am just not surprised about this at all. They have more information than I could ever imagine, wouldn't any rational person assume there is a search engine to index, sort, parse, and return results?
seems like a lot until you consider how many indexed pages Google has: http://i.imgur.com/EqIJAoL.jpg
why not throw in grains of sand or atoms in the universe?
seems like a lot until you consider how many indexed pages Google has: http://i.imgur.com/EqIJAoL.jpg
why not throw in grains of sand or atoms in the universe?
Nice comparison, though I hate it when graphics look like they were drawn to scale, but actually aren't. (The Google indexed pages bar would have to be more than 10 times larger.)
More on topic: Whether you can compare the index sizes or not mostly depends on their signal to noise ration. Given the number of Google search request per month and assuming and average of 5 viewed results, you can put a lower bound of 99% of pages that are never seen by anyone. As the search results should greatly overlap the actual number of relevant sites is probably orders of magnitude less than 1%.
More on topic: Whether you can compare the index sizes or not mostly depends on their signal to noise ration. Given the number of Google search request per month and assuming and average of 5 viewed results, you can put a lower bound of 99% of pages that are never seen by anyone. As the search results should greatly overlap the actual number of relevant sites is probably orders of magnitude less than 1%.
To me, this explains why the five eyes nations are pushing for (meta)data retention legislation. It's a condition of participating in the scheme.
I don't think it remained "under the radar" like some are commenting. There's mention of it in Snowden's disclosures and if you search around, lots of government recruiting related info, e.g., http://www.socnet.com/archive/index.php/t-108034.html
I think I'm more surprised that this was a surprise to people.
Of course they built a search engine. Wouldn't you? Don't you have similar at your workplace? We use them all the time. Think about web interfaces built on top of ElasticSearch, for example. Is that not a 'search engine'?
Of course they built a search engine. Wouldn't you? Don't you have similar at your workplace? We use them all the time. Think about web interfaces built on top of ElasticSearch, for example. Is that not a 'search engine'?
It's not the building a search engine part.
It's the opening it up to LEO. Taking homeland security-justified warrantless searches and turning those over to criminal investigations is far beyond unconstitutional.
It's the opening it up to LEO. Taking homeland security-justified warrantless searches and turning those over to criminal investigations is far beyond unconstitutional.
> Taking homeland security-justified warrantless searches and turning those over to criminal investigations is far beyond unconstitutional.
Except, that it's not.
It's all about how the government comes into possession of information that makes it the fruit (or not) of the "poisonous tree".
Once the government legally learns of some bit of information, there's no requirement by itself that they have to "pretend not to know" something is going on. That's why a third party voluntarily divulging information to the government doesn't cause that evidence to get thrown out of court: There may be a cause for civil or even criminal action against that third party, but that doesn't invalidate the evidence.
Now, Congress has passed statute laws limiting information sharing between intelligence agencies and law enforcement precisely because of the threat of having this all-seeing eye subvert democratic government, but the limits were never complete exclusions, even before 9/11. And the reason statute laws were needed is because there was no barrier to this activity from the judicial side alone.
Parallel construction came in not because evidence was illegally gathered, but because giving the (fully legal!) chain of custody for evidence derived from intelligence sources would have quite naturally have "burned" that source or method, so the NSA would require law enforcement agencies using that data to use alternate (again though, legal) means to make a case in court, in order to protect their intelligence source. It's the old apocryphal dilemma about whether to let Coventry be bombed to protect ULTRA, applied to the post-9/11 world.
Except, that it's not.
It's all about how the government comes into possession of information that makes it the fruit (or not) of the "poisonous tree".
Once the government legally learns of some bit of information, there's no requirement by itself that they have to "pretend not to know" something is going on. That's why a third party voluntarily divulging information to the government doesn't cause that evidence to get thrown out of court: There may be a cause for civil or even criminal action against that third party, but that doesn't invalidate the evidence.
Now, Congress has passed statute laws limiting information sharing between intelligence agencies and law enforcement precisely because of the threat of having this all-seeing eye subvert democratic government, but the limits were never complete exclusions, even before 9/11. And the reason statute laws were needed is because there was no barrier to this activity from the judicial side alone.
Parallel construction came in not because evidence was illegally gathered, but because giving the (fully legal!) chain of custody for evidence derived from intelligence sources would have quite naturally have "burned" that source or method, so the NSA would require law enforcement agencies using that data to use alternate (again though, legal) means to make a case in court, in order to protect their intelligence source. It's the old apocryphal dilemma about whether to let Coventry be bombed to protect ULTRA, applied to the post-9/11 world.
[deleted]
Is anyone else puzzled that stories about the NSA are accepted as gospel fact when posted on Hacker News?
The Reality Distortion Field appears, and people believe, because they want to believe.
It depresses me, the lack of intelligent discourse.
Most tech people I meet actually believe that the NSA records and stores all telephone calls. It's depressingly stupid, but I have given up arguing, logic and sense are not welcome when the NSA is the topic.
The Reality Distortion Field appears, and people believe, because they want to believe.
It depresses me, the lack of intelligent discourse.
Most tech people I meet actually believe that the NSA records and stores all telephone calls. It's depressingly stupid, but I have given up arguing, logic and sense are not welcome when the NSA is the topic.
They do record and store (foreign and domestic) phone calls. Your comment proves you're simply an ignorant or a shill.
Proof:
* NSA Collects ‘Word for Word’ Every Domestic Communication, Says Former Analyst http://www.pbs.org/newshour/bb/government_programs-july-dec1...
* NSA collecting phone records of millions of Verizon customers daily http://www.theguardian.com/world/2013/jun/06/nsa-phone-recor...
* NSA surveillance program reaches ‘into the past’ to retrieve, replay phone calls http://www.washingtonpost.com/world/national-security/nsa-su...
* NSA spying flap extends to contents of U.S. phone calls http://www.cnet.com/news/nsa-spying-flap-extends-to-contents...
Proof:
* NSA Collects ‘Word for Word’ Every Domestic Communication, Says Former Analyst http://www.pbs.org/newshour/bb/government_programs-july-dec1...
* NSA collecting phone records of millions of Verizon customers daily http://www.theguardian.com/world/2013/jun/06/nsa-phone-recor...
* NSA surveillance program reaches ‘into the past’ to retrieve, replay phone calls http://www.washingtonpost.com/world/national-security/nsa-su...
* NSA spying flap extends to contents of U.S. phone calls http://www.cnet.com/news/nsa-spying-flap-extends-to-contents...
I'm a "shill"? Really? This is the level of debate?
Also, none of your links in any way prove that the NSA stores all phone calls. None of them. 1 is hearsay from someone, another talks about metadata, and another says that the NSA can record any phone call (yes we know that). Nowhere is there any proof that all calls are being recorded.
This is what I am talking about. It doesn't seem possible to have a conversation about the NSA on Hacker News that is based on logic, reasoning or sense.
Also, none of your links in any way prove that the NSA stores all phone calls. None of them. 1 is hearsay from someone, another talks about metadata, and another says that the NSA can record any phone call (yes we know that). Nowhere is there any proof that all calls are being recorded.
This is what I am talking about. It doesn't seem possible to have a conversation about the NSA on Hacker News that is based on logic, reasoning or sense.
"MYSTIC is a voice interception program used by the National Security Agency. The program recorded every phone call made within a non-specified country for thirty days."
https://en.wikipedia.org/wiki/MYSTIC_(surveillance_program)
https://en.wikipedia.org/wiki/MYSTIC_(surveillance_program)
Again, this is moving the goalposts.
The original claim was "the NSA records all phone calls".
This is saying, "the NSA records all phone calls, in a certain, non-domestic country, for a 30-day window", which is nowhere near the same.
You would think that simply accusing the NSA of what it's actually doing what be good enough, but that's only rarely the case here at HN.
The original claim was "the NSA records all phone calls".
This is saying, "the NSA records all phone calls, in a certain, non-domestic country, for a 30-day window", which is nowhere near the same.
You would think that simply accusing the NSA of what it's actually doing what be good enough, but that's only rarely the case here at HN.
Perhaps proof to the contrary would be more useful than the veracity or logic of an argument?
Unfortunately, with the NSA, independently verifiable proof of anything, especially doing the right thing, is impossible to attain.
Unfortunately, with the NSA, independently verifiable proof of anything, especially doing the right thing, is impossible to attain.
You're asking the parent commenter to prove a negative? Is this what logic has come to at HN?
Yes, ever since June 2013.
[1] https://www.techdirt.com/articles/20140203/11143926078/paral...