A Team at Amazon Is Listening to Recordings Captured by Alexa(buzzfeednews.com)
buzzfeednews.com
A Team at Amazon Is Listening to Recordings Captured by Alexa
https://www.buzzfeednews.com/article/nicolenguyen/amazon-employees-listening-to-alexa-echo-recordings
12 comments
See https://news.ycombinator.com/item?id=19629513 for more discussion on this and the original Bloomberg article
Large discussion already occurring:
https://news.ycombinator.com/item?id=19629513
https://news.ycombinator.com/item?id=19629513
Am I supposed to be surprised? This is just logical. Of course they employ people to grade the quality of assistants' voice recognition and responses.
Does that excuse Amazon's behavior? None of us were surprised when we learned about the NSA's Echelon or PRISM programs after all.
I would NEVER put an Alexa in my home - but has Amazon done anything wrong here?
You agreed to the terms of use, they’re doing expected work to anyone that has any idea how this all works, there seems to be no malicious intent even implied. IDK, if you don’t want someone potentially listening you, don’t put an Echo in your home.
You agreed to the terms of use, they’re doing expected work to anyone that has any idea how this all works, there seems to be no malicious intent even implied. IDK, if you don’t want someone potentially listening you, don’t put an Echo in your home.
>has Amazon done anything wrong here?
There's the contract that was actually signed, which Amazon didn't break, and then there's the contract people think they signed. It's generally considered your fault when you sign a contract with the devil and get tricked, but the devil is still evil. In other words, Amazon is evil in this situation and their customers made a mistake. If it had been made clear to non-technical people that they would get listened to by the listening device, then it would all be on them. However, this comes as a surprise to the average person, because it doesn't take that much work to encourage a false belief in the mind of a non-expert.
There's the contract that was actually signed, which Amazon didn't break, and then there's the contract people think they signed. It's generally considered your fault when you sign a contract with the devil and get tricked, but the devil is still evil. In other words, Amazon is evil in this situation and their customers made a mistake. If it had been made clear to non-technical people that they would get listened to by the listening device, then it would all be on them. However, this comes as a surprise to the average person, because it doesn't take that much work to encourage a false belief in the mind of a non-expert.
Fundamentally for a voice assistant to assist you it needs to listen to you. I don't see how this could possibly be surprising to anyone.
The headline in this case is that humans are also involved in the listening, which is technically advantageous but not technically necessary. It is also worth mentioning that the recording gets uploaded out of your house, which is again technically advantageous but not technically necessary.
I disagree that it is conceptually possible for most consumers to have an adequate understanding of future implications of this technology such that they can possibly give informed consent for this type of data capture.
There are just too many ways it can eventually be used maliciously, so much so that we can’t even contemplate or imagine them ahead of time. In that case, it should be illegal for a company to capture the consumer-generated private data, much the same as age of consent laws... even if a minor appears to have a self-aware and complete understanding of something they are consenting to, it is just legally defined to be impossible for the minor to actually be capable of granting consent.
The same should be true for collecting user behavior and non-aggregated user activity data, across the board. “Consenting” to it, whether explicitly or through site usage terms or EULAs, is just not a logically possible thing.
There are just too many ways it can eventually be used maliciously, so much so that we can’t even contemplate or imagine them ahead of time. In that case, it should be illegal for a company to capture the consumer-generated private data, much the same as age of consent laws... even if a minor appears to have a self-aware and complete understanding of something they are consenting to, it is just legally defined to be impossible for the minor to actually be capable of granting consent.
The same should be true for collecting user behavior and non-aggregated user activity data, across the board. “Consenting” to it, whether explicitly or through site usage terms or EULAs, is just not a logically possible thing.
I somehow disagree. In a free democratic society every citizen should be able to operate with the assumption that very invasive practice is banned, unless you opt into it with clear intent.
Most people who got an Alexa have probably no idea how it works and whether it is invasive or not and they quite certainly operate it under the assumption that no real human will listen in to the conversations they have in the presence of their electronic assistant.
Unless Amazon made a strong effort to communicate this in the clearest possible way (e.g. by putting a "our employes may listen in to any conversation" front and center on every place they sell it), they are in my opinion at least guilty in a ethical sense here. Legal is a different thing, but you can be operating totally legal, while still beeing ethically wrong.
Most people who got an Alexa have probably no idea how it works and whether it is invasive or not and they quite certainly operate it under the assumption that no real human will listen in to the conversations they have in the presence of their electronic assistant.
Unless Amazon made a strong effort to communicate this in the clearest possible way (e.g. by putting a "our employes may listen in to any conversation" front and center on every place they sell it), they are in my opinion at least guilty in a ethical sense here. Legal is a different thing, but you can be operating totally legal, while still beeing ethically wrong.
>In a free democratic society every citizen should be able to operate with the assumption that very invasive practice is banned
The words free society and everything you don’t agree with should be banned seem like opposite things.
The reality is if you put an always on microphone in your home - you should absolutely expect someone could be listening. At not point did they promise you that wouldn't happen.
The words free society and everything you don’t agree with should be banned seem like opposite things.
The reality is if you put an always on microphone in your home - you should absolutely expect someone could be listening. At not point did they promise you that wouldn't happen.
> they’re doing expected work to anyone that has any idea how this all works
For sure. But lots of people dont't know how this stuff works and should not be expected too.
I wonder how many average consumers of Alexa products would feel differently about their purchase if it said on the box "some people at amazon are likely to listen to what you say". Sure, everyone agrees to the terms of use, but that doesn't seem to count for much in the way of "informed consent" these days. Maybe to the letter, but not the spirit.
For sure. But lots of people dont't know how this stuff works and should not be expected too.
I wonder how many average consumers of Alexa products would feel differently about their purchase if it said on the box "some people at amazon are likely to listen to what you say". Sure, everyone agrees to the terms of use, but that doesn't seem to count for much in the way of "informed consent" these days. Maybe to the letter, but not the spirit.
>But lots of people don't know how this stuff works and should not be expected too.
How does caveat emptor not apply here? If you put an internet connected microphone in your home - it seems very reasonable to assume someone could listen in on that.
How does caveat emptor not apply here? If you put an internet connected microphone in your home - it seems very reasonable to assume someone could listen in on that.
I don't see it as a wrong or right, more so a matter of informing the public. Everyone suspects that those devices are listening to them but it's usually brushed off as tin-foil hat stuff. Stories like this are just making people aware of the reality of these technologies. You decide if it's acceptable or not.
Any time Alexa is lit up, it may be listening to you. Any time Alexa is not lit up, your conversations can't be overheard. The tin foil hat aspect comes in when people say it's listening ALL THE TIME, which it isn't.
None of my VERY non-technical family think that their assistant devices aren't transmitting recordings of their voices to some remote server somewhere, because it's obvious enough. Even my 71 year old grandfather understands this.
None of my VERY non-technical family think that their assistant devices aren't transmitting recordings of their voices to some remote server somewhere, because it's obvious enough. Even my 71 year old grandfather understands this.
Yeah, but do they know that other human beings are listening to it?
You will be surprised how many customers don't expect it to happen. You only know it because you have an idea of how the whole system works. Almost every user just agrees to terms of use without reading it.
We really have to go back to a model where most computation is done on the client and data mainly goes from server to client and not the other way. They shouldn’t be using real conversations for training their models or at least only in a very limited and open manner. Soon these companies will see everything we do, hear everything we say and read everything we write. That’s not a good future.
> Soon these companies will see everything we do, hear everything we say and read everything we write.
Seriously, in 2019, you are actually surprised this is happening? After all the news that keeps popping out, proving that almost every single effin' company out there is after as much client data as possible, regardless of morals and sometimes even laws? People actually pay for devices that keep monitoring them, hilarious (and sad tbh). To save few clicks and feel 'modern'?
I fail to see the proper added value in ones life of all these assistants, not in the light of all the technical f_ckups that shouldn't have happened by simple design decisions.
Seriously, in 2019, you are actually surprised this is happening? After all the news that keeps popping out, proving that almost every single effin' company out there is after as much client data as possible, regardless of morals and sometimes even laws? People actually pay for devices that keep monitoring them, hilarious (and sad tbh). To save few clicks and feel 'modern'?
I fail to see the proper added value in ones life of all these assistants, not in the light of all the technical f_ckups that shouldn't have happened by simple design decisions.
I am not really surprised as such but I am worried that my worst suspicions seem to always come true. If that trend continues then our future will be very bleak and very suppressive.
There's a team of people at Mozilla working on adding an offline speech recognition API [1] to Firefox. This is particularly exciting to me because I have a SpeechRecognition-based "assistant" framework that I would love to make 100% locally served.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1248897
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1248897
There are actually a few projects that follow this model much closer than Alexa/Siri/Google Assistant, or at the very least let you control host the computation on your desktop so you know where all your traffic is going.
https://mycroft.ai/ https://snips.ai/
https://mycroft.ai/ https://snips.ai/
Is this newsworthy?
I know people who work at Apple. Their job is to listen to recordings taken by Siri, regardless of what the intention is. E.g. they listen to text messages people say to Siri - to send, searches, anything said to Siri.
I know people who work at Apple. Their job is to listen to recordings taken by Siri, regardless of what the intention is. E.g. they listen to text messages people say to Siri - to send, searches, anything said to Siri.
[deleted]
I think it’s newsworthy to remind normal people that technology isn’t magic. In the case of always on microphones in your home, that is important imo.
[deleted]
Nobody should be surprised by this. To actually test your voice recognition software in a definitive manner you need to compare you kind of have to compare your software's analysis to human analysis and that requires humans to listen to the audio or at least snippets of it. The software is competing with a human after all, you need to test its performance compared to the humans.
With the exception of the fact that this is going to sell clicks...
Is this really news? I thought they confirmed this years ago? Same with Google and Apple...
I am fine with this, if the purpose is solely for product development. I have a problem with the data not being anonymous and their mocking some of the clips in a shared internal forum. The data should be anonymized and management should stop the mocking of clips on internal forums.
I'm wondering if they are recording some fun conversations or maybe telling family/friends: "hahahah, hey Mike, you wouldn't believe what this weirdo said to Alexa... and he lives there ..., yep there ... just three blocks away, I know because he ordered toilet paper just after that..."
The smart solution would've been for Amazon to make it an opt-in program and offer people a dollar credit for some servic they'll never use in exchange for signing up. Amazon would've had millions of users consenting to any amount of data collection and be free and clear in the press.
Buzz Feed of all sources for something just about everyone knows. What’s the goal other than driving clicks and trying to stir up pointless outrage?
Buzzfeed cites Bloomberg as the source in this article.
I think this sort of process, assuming the privacy and security teams have their say, is an absolutely legitimate part of product improvement.