XKeyscore: NSA program collects 'nearly everything a user does on the internet'(theguardian.com)
theguardian.com
XKeyscore: NSA program collects 'nearly everything a user does on the internet'
http://www.theguardian.com/world/2013/jul/31/nsa-top-secret-program-online-data
612 comments
Wow. Just Wow.
For years all of this was in the back of my mind as being capable but my not wanting to think like a conspiracy crackpot just dismissed the thought as it couldn't be possible. A conspiracy takes a lot of co-operation from within large corporations who must also remain it a secret. Surely someone would have a conscious and leak it? Or one of companies we all look up to as a modern example of do-good company would say "Hell NO" to the attempt and then let the world know what was attempted. Guess that was eventually proven true with Snowden (a real hero imo), just shocked they were able to operate to the scale they did for so long before a Snowden came along.
In my mind, this is not so much a shock to me regarding the NSA as well as the current evil government we have had in place. Doesn't take a genius to realize the president lies to our face on TV about trivial issues/promises, so expected for top secret stuff.
What is the BIG stomach churning shock to me is the very companies that we have come to know that are multi-billion dollar conglomerates providing service/products for millions for every day use has been a part of it. A part of this secret web while all the while proclaiming privacy for it users. I guess at end of day profits still rule the roost. "Just do this for us, turn a blind eye, and you get to go on making your billions". I wonder how many CEO's knew of all this. Gates? Zuckerburg? Etc etc.
I feel like I have no outs now. There are no alternatives to current establishment of companies that make our lives easier. Should we all wipe our PC's and use Linux, sell our phones and use Ubuntu Phone, not pay for SSL certs anymore (another mafia), etc?
For years all of this was in the back of my mind as being capable but my not wanting to think like a conspiracy crackpot just dismissed the thought as it couldn't be possible. A conspiracy takes a lot of co-operation from within large corporations who must also remain it a secret. Surely someone would have a conscious and leak it? Or one of companies we all look up to as a modern example of do-good company would say "Hell NO" to the attempt and then let the world know what was attempted. Guess that was eventually proven true with Snowden (a real hero imo), just shocked they were able to operate to the scale they did for so long before a Snowden came along.
In my mind, this is not so much a shock to me regarding the NSA as well as the current evil government we have had in place. Doesn't take a genius to realize the president lies to our face on TV about trivial issues/promises, so expected for top secret stuff.
What is the BIG stomach churning shock to me is the very companies that we have come to know that are multi-billion dollar conglomerates providing service/products for millions for every day use has been a part of it. A part of this secret web while all the while proclaiming privacy for it users. I guess at end of day profits still rule the roost. "Just do this for us, turn a blind eye, and you get to go on making your billions". I wonder how many CEO's knew of all this. Gates? Zuckerburg? Etc etc.
I feel like I have no outs now. There are no alternatives to current establishment of companies that make our lives easier. Should we all wipe our PC's and use Linux, sell our phones and use Ubuntu Phone, not pay for SSL certs anymore (another mafia), etc?
Makes me Wonder, if the Internet in this widespread form, was allowed so that they can snoop (so easily)?
When I was a kid, my father, had told me a story that in Russia people are scared to speak their minds, for fear of being snooped via any hidden gadgets in the walls.
When I was a kid, my father, had told me a story that in Russia people are scared to speak their minds, for fear of being snooped via any hidden gadgets in the walls.
Slide 23: "Show me all the Microsoft Excel spreadsheets
containing MAC addresses coming out of Iraq
so I can perform network mapping"
Does MS Excel store your MAC address in the xlsx file?
Does MS Excel store your MAC address in the xlsx file?
It's been being weirdly suppressed on reddit: http://www.anonmgur.com/up/17832a6eafb09376d012090ff1b06dbe....
Every time a thread on this hits the top it gets mod-deleted.
Every time a thread on this hits the top it gets mod-deleted.
http://s3.documentcloud.org/documents/743252/nsa-pdfs-redact...
Missing: How much did this cost? Did the government (taxpayers) overpay?
Missing: How much did this cost? Did the government (taxpayers) overpay?
I wonder if we should try to put together a programme to try to drain the NSA of technical talent ... offering jobs or other incentives to try to persuade developers currently working for the agencies and their various contractors to resign?
It is interesting on slide 17 that the NSA can decrypt all VPN traffic.
Does this indicate that they have broken HTTPS, or simply that they own VPN companies like Private Internet Access?
Does this indicate that they have broken HTTPS, or simply that they own VPN companies like Private Internet Access?
Wow... Govt is into big data. I wonder what they use for analyzing all this data
Nice pre-emptive "attack" by Greenwald today, just before the NSA hearings.
This bit both somewhat limits the impact and makes Greenwald et. al.'s claims that most everything is being Hoovered up a lot more credible:
"The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.""
Of course, as the article goes on to detail, anything that's found to be of interest in that window can be saved permanently, and NSA analysis do that a lot.
"The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.""
Of course, as the article goes on to detail, anything that's found to be of interest in that window can be saved permanently, and NSA analysis do that a lot.
Well then they are going to need that big data center. That is an unimaginable amount of data...
Imagine if storage limitations weren't holding back the NSA.
Those 60TB density HAMR[1] drives that are due in 2016 are really going to take invasive to a whole new level.
[1] http://storageeffect.media.seagate.com/files/2012/03/perpham...
Those 60TB density HAMR[1] drives that are due in 2016 are really going to take invasive to a whole new level.
[1] http://storageeffect.media.seagate.com/files/2012/03/perpham...
I wonder how they store all that. Surely a side benefit of this could be NSA contributions to CS journals about database techniques.
Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? I think it's more likely this claim is made simply to generate FUD in the general population.
Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? I think it's more likely this claim is made simply to generate FUD in the general population.
The Guardian strongly implies this system is used to intentionally target US citizens in violation of the law, but then admits that would be "illegal." I wonder if the leaked presentation touches on this point.
That UI looks awfully similar to a theme I've seen used in SharePoint Portal Server. I hope that's not what they use for the front end, but I wouldn't put it past them.
One of the screen shots:
http://static.guim.co.uk/sys-images/Guardian/Pix/audio/video...
says: Top Secret Comm(?) REL() to USA, AUS, CAN, GBR, NZL
confirming the previous suspicions that many other governments are on board.
Der Spiegel actually has reported a few weeks back about XKeyscore [1] and that it is used by the BND (Germany's NSA). I.e. all this data is also available to the NSA equivalents of Australia, Candana, Great Britain and New Zealand.
Many Americans trust their government (unfortunately), will they also trust the other governments?
[1]:
http://www.spiegel.de/international/world/german-intelligenc...
http://www.spiegel.de/international/germany/german-intellige...
http://static.guim.co.uk/sys-images/Guardian/Pix/audio/video...
says: Top Secret Comm(?) REL() to USA, AUS, CAN, GBR, NZL
confirming the previous suspicions that many other governments are on board.
Der Spiegel actually has reported a few weeks back about XKeyscore [1] and that it is used by the BND (Germany's NSA). I.e. all this data is also available to the NSA equivalents of Australia, Candana, Great Britain and New Zealand.
Many Americans trust their government (unfortunately), will they also trust the other governments?
[1]:
http://www.spiegel.de/international/world/german-intelligenc...
http://www.spiegel.de/international/germany/german-intellige...
This is brilliant, I love the screenshots:
Foreignness factor:
The person has stated that he is located outside the U.S.
Human intelligence source indicates person is located outside the U.s.
The person is a user of storage media seized outside the U.s.
Foreign govt indicates that the person is located outside the U.s.
Phone number country code indicates the person is located outside the U.s.
Phone number is registered in a country other than the U.S.
SIGINT reporting confirms person is located outside the U.S.
Open source information indicates person is located outside the U.s.
Network, machine or tech info indicates person is located outside the U.s.
In direct contact w/ tgt overseas no info to show proposed tgt in U.S.
It's quite easy to lose the protections of a U.S. citizen indeed!
Foreignness factor:
The person has stated that he is located outside the U.S.
Human intelligence source indicates person is located outside the U.s.
The person is a user of storage media seized outside the U.s.
Foreign govt indicates that the person is located outside the U.s.
Phone number country code indicates the person is located outside the U.s.
Phone number is registered in a country other than the U.S.
SIGINT reporting confirms person is located outside the U.S.
Open source information indicates person is located outside the U.s.
Network, machine or tech info indicates person is located outside the U.s.
In direct contact w/ tgt overseas no info to show proposed tgt in U.S.
It's quite easy to lose the protections of a U.S. citizen indeed!
Interesting; it appears someone failed to redact some data from the slides. In the Facebook chat example, the message is "to" 1536051595.
Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595
Which leads us to the Facebook profile (https://www.facebook.com/arash.gorjipour.5) of an individual, real or contrived, named "Arash Gorjipour". His email address and phone number are all exposed in one of his uploaded photos: http://i.imgur.com/0UUk5cB.jpg
I wonder what the reason for this man being in these slides is.
Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595
Which leads us to the Facebook profile (https://www.facebook.com/arash.gorjipour.5) of an individual, real or contrived, named "Arash Gorjipour". His email address and phone number are all exposed in one of his uploaded photos: http://i.imgur.com/0UUk5cB.jpg
I wonder what the reason for this man being in these slides is.
At what point do the mathematical limits of data mining kick in here? How useful is all this information?
I'm not an expert in this area of mathematics, so I could be wrong, but my impression is that as the haystack becomes larger the problem of false positives becomes more and more severe.
As a data miner, what you want is the maximum number of "hits" (of whatever you're trying to hit) with the minimum number of misses and the minimum number of false positives. My impression is that this becomes progressively harder-- the golden region between too many false positives and too many false negatives becomes smaller and smaller and harder to hit.
Eventually you either miss important hits, namely the next terrorist attack, or you get swamped with false positives that you have to manually investigate and rule out.
I'd love someone who does know more here to chip in, but my personal suspicion is that this actually has a pretty huge pork angle to it. How much money are the contractors getting for building this stuff?
I'm not an expert in this area of mathematics, so I could be wrong, but my impression is that as the haystack becomes larger the problem of false positives becomes more and more severe.
As a data miner, what you want is the maximum number of "hits" (of whatever you're trying to hit) with the minimum number of misses and the minimum number of false positives. My impression is that this becomes progressively harder-- the golden region between too many false positives and too many false negatives becomes smaller and smaller and harder to hit.
Eventually you either miss important hits, namely the next terrorist attack, or you get swamped with false positives that you have to manually investigate and rule out.
I'd love someone who does know more here to chip in, but my personal suspicion is that this actually has a pretty huge pork angle to it. How much money are the contractors getting for building this stuff?
Just like suspected. If you use encryption like PGP, you become person of interest.
Snowden deserves the Nobel Peace Prize and the Vatican should consider canonizing him.
This is overwhelming. Even when you always hear the claims about we knew this was going on, somehow it is still shocking when you see it all laid out infront of you with screenshots and the capabilities described.
I can see how they get HTTP information, since they would intercept at transit hubs - but how are they getting all Facebook private messages and Gmail?
I was also looking for another unique ID that users are identified by - perhaps a machine or browser fingerprint or some form of intel that can 'glue' different browsers together and make a best guess if they are the same person (Facebook does this with device and user cookies) but couldn't find anything. It seems they rely solely on email addresses, IP addresses, cookies and HTTP headers.
So if you are browsing via 16 tor circuits and a browser that defaults to incognito with session histories being wiped, they couldn't reconstruct your history.
Users of PGP/encryption products being singled out is terrifying. The sooner we have the whole world using decent encryption tools, the better.
Edit: Gmail messages must only be captured when they leave the Google network. They are the only provider to support server-to-server TLS: https://twitter.com/ashk4n/status/346807239002169344/photo/1
They must only be getting a slice of the Facebook chat data, since the transport there is also https.
Facebook Messenger, on the other hand, uses MQTT, so it transmits and stores in plaintext. It has support for encrypted + signed messages with OTR if you are using an alternate client such as Adium or Pidgin.
Really need to go out an audit all of these services and let users know which are better.
I can see how they get HTTP information, since they would intercept at transit hubs - but how are they getting all Facebook private messages and Gmail?
I was also looking for another unique ID that users are identified by - perhaps a machine or browser fingerprint or some form of intel that can 'glue' different browsers together and make a best guess if they are the same person (Facebook does this with device and user cookies) but couldn't find anything. It seems they rely solely on email addresses, IP addresses, cookies and HTTP headers.
So if you are browsing via 16 tor circuits and a browser that defaults to incognito with session histories being wiped, they couldn't reconstruct your history.
Users of PGP/encryption products being singled out is terrifying. The sooner we have the whole world using decent encryption tools, the better.
Edit: Gmail messages must only be captured when they leave the Google network. They are the only provider to support server-to-server TLS: https://twitter.com/ashk4n/status/346807239002169344/photo/1
They must only be getting a slice of the Facebook chat data, since the transport there is also https.
Facebook Messenger, on the other hand, uses MQTT, so it transmits and stores in plaintext. It has support for encrypted + signed messages with OTR if you are using an alternate client such as Adium or Pidgin.
Really need to go out an audit all of these services and let users know which are better.
Slide 6 of the presentation clearly shows that pretty much every government is in on the program, with heavy concentration in western Europe.
One question, how did the dot in China get there?
http://www.theguardian.com/world/interactive/2013/jul/31/nsa...
One question, how did the dot in China get there?
http://www.theguardian.com/world/interactive/2013/jul/31/nsa...
From the slides
http://www.theguardian.com/world/interactive/2013/jul/31/nsa...
"Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users"
Does this mean using VPN is not very safe from dragnet?
"Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users"
Does this mean using VPN is not very safe from dragnet?
Holy shit... Apparently, the only way to ensure privacy is to go Stallman. Funny how yesterday's "conspiracy crackpot" became today's visionary.
You have to admit these guys are working on some cool problems. If you don't have a problem with the legality of it or potential for misuse it looks like a really interesting place to work.
Another data point on the relationship between government and terrorism:
I live in Columbia, South Carolina. A mile from my house there is a prominent statue of Ben Tillman. Tillman was an explicit advocate of terrorism, and indeed personally engaged in it [1], which drove his popularity and ensured his election to the governorship and the United States Senate.
Government programs such as the NSA's exist to protect the interests of the powerful. Same as it ever was.
http://en.wikipedia.org/wiki/Benjamin_Tillman
I live in Columbia, South Carolina. A mile from my house there is a prominent statue of Ben Tillman. Tillman was an explicit advocate of terrorism, and indeed personally engaged in it [1], which drove his popularity and ensured his election to the governorship and the United States Senate.
Government programs such as the NSA's exist to protect the interests of the powerful. Same as it ever was.
http://en.wikipedia.org/wiki/Benjamin_Tillman
Noticed one of the screenshots have a URL. It's a little blurry, but I suppose it's an intranet URL since the TLD looks like .nsa
URL looks like: https://gamut-wakefield.ein.nsa/utt/UTT/do/FRNewSelector#sel...
URL looks like: https://gamut-wakefield.ein.nsa/utt/UTT/do/FRNewSelector#sel...
Hard for me to fathom anyone taking a job, helping to build systems like this. I get that many of the components of a system like this could be seen as harmless. However, a system of this complexity must have some talented engineers bringing it all together and making it work. How can they feel good about what they are doing?