Decoding Vibrations From Nearby Keyboards With Mobile Phone Accelerometers [pdf](cc.gatech.edu)
cc.gatech.edu
Decoding Vibrations From Nearby Keyboards With Mobile Phone Accelerometers [pdf]
http://www.cc.gatech.edu/~traynor/papers/traynor-ccs11.pdf
18 comments
>Page 6 states that there was a significant training period for their machine learning algorithm that involved sending the bluetooth from the keyboard through the phone to train it.
You shouldn't need to train it via this method, you pretty much know exactly how often each letter of the alphabet is used so once you differentiate between keys it's just standard substitution encryption cracking = very easy.
The keyboard type should be irrelevant, if you can uniquely identify keys being pressed it's a easy game over.
You shouldn't need to train it via this method, you pretty much know exactly how often each letter of the alphabet is used so once you differentiate between keys it's just standard substitution encryption cracking = very easy.
The keyboard type should be irrelevant, if you can uniquely identify keys being pressed it's a easy game over.
No, in the real world adversaries have been perfecting (i.e., 'training') these techniques for longer than most of us have been alive.
http://www.nsa.gov/public_info/_files/cryptologic_spectrum/t...
http://www.nsa.gov/public_info/_files/cryptologic_spectrum/t...
what if it's your own phone? Like you place it near someone's keyboard and use it to spy on them?
(I confess, i haven't read the paper yet)
btw would this work for laptops? wouldn't the signal be a bit different?
(I confess, i haven't read the paper yet)
btw would this work for laptops? wouldn't the signal be a bit different?
>btw would this work for laptops? wouldn't the signal be a bit different?
Yes, and that's the major problem with its practicality. The researchers had to make use of a long training period on one specific keyboard (I don't believe they specify how many typists were involved in the training process, though). The rub, is that when they take this phone to another keyboard or have another typist at the keyboard, it will need to be trained again, just to reach 80% accuracy. If you want to spy on someone with your phone there are far better/more reliable/efficient ways to do it that do not involve training a machine learning algorithm to process key strokes from a specific keyboard with, at most, a couple of typists on it (hopefully they are all the typists you want). Not to mention what happens if the phone gets moved. A few inches and their training period would probably be borked eight ways to Sunday.
Yes, and that's the major problem with its practicality. The researchers had to make use of a long training period on one specific keyboard (I don't believe they specify how many typists were involved in the training process, though). The rub, is that when they take this phone to another keyboard or have another typist at the keyboard, it will need to be trained again, just to reach 80% accuracy. If you want to spy on someone with your phone there are far better/more reliable/efficient ways to do it that do not involve training a machine learning algorithm to process key strokes from a specific keyboard with, at most, a couple of typists on it (hopefully they are all the typists you want). Not to mention what happens if the phone gets moved. A few inches and their training period would probably be borked eight ways to Sunday.
It's definitely far from being an easy attack. But it raises an interesting question in that the current smartphone security models don't require special permissions for accelerometer access whereas they do for cameras and other things.
With this kind of stuff and other advances in activity recognition, allowing any app to record anything using the accelerometer might become a privacy threat.
With this kind of stuff and other advances in activity recognition, allowing any app to record anything using the accelerometer might become a privacy threat.
Both Android and iPhone expose an accelerometer API even to web pages without user input, although I'm not sure what the sample rate is.
Hmm, this is very interesting - a web page that you commonly use can also easily track large amounts of what you type on your computer, and use it as the training data to correlate with the accelometer on your phone; and if you use that web page often, then you're somewhat likely to have their app installed on your phone.
sounds like one can be tracked even without GPS, just by integrating accelerometer data and even one fixed point of reference may be not necessary as the pattern of movements can, in many cases, be uniquely aligned with the pattern of streets/etc...
Interesting, I could see this working with an app that didn't have GPS permissions especially in rural areas where the roads might have a unique fingerprint. Through the mobile web APIs the phone's screen has to be on for the JavaScript event loop to run so that would be a harder way to do it.
I think the reason GPS is used is that measurement error in the accelerometer leads to pretty bad drift. If you could resynch the movements periodically it might work though.
Yes I am very relived they published this. I assumed this was possible acoustically many years ago, but never told anyone ahahha. When I saw this headline, I feared seismic (accelerometers) might do an even better job at it, but fortunately, it looks like they don't.
I have some bad news for you =P.
https://www.schneier.com/blog/archives/2009/03/sniffing_keyb...
https://www.schneier.com/blog/archives/2009/03/sniffing_keyb...
Modern SCIFs vibrate window panes to prevent laser microphones.
They also use EMF shielding to prevent TEMPEST attacks. Most interesting to me is that they were aware of TEMPEST leakage as early as WWII.
They also use EMF shielding to prevent TEMPEST attacks. Most interesting to me is that they were aware of TEMPEST leakage as early as WWII.
You could train it with a similar setup and then use the training data to target another.
I'd use a dictionary and Bayesian net. You're given word lengths and keystroke signatures. You are not told which key matches which signature. It essentially becomes an encryption cracking problem, kinda like "e is the most common letter so that's the vibration sig we'll see the most".
I think the backspace key would be the best key to try to find a reference for. It is often pressed multiple times in a row and would allow for recognition of common typos.
You could also get a sketch map of the keyboard from key press volume, louder means closer, find backspace, model phone position in relation to keyboard position, take it from there.
And the spacebar key, when hit, sounds different than the other keys. I'm not good with onomatopoeia but normal keys generally go tack-tack-tack while the spacebar goes tchick-tchick-tchick. It's noticeably different. This could actually prove quite scary since if you know the length of the 'tack', you're basically doing stupidly simple linear cryptanalysis.
Would this be a template attack? (pdf: http://saluc.engr.uconn.edu/refs/sidechannel/chari02template...)
Hm. Not an expert on the subject but I think by some readings of that you could consider it to be one. After all, if you duplicate the setup you're going to bug that gives you exactly the same kind of advantage that a template attack gives you, one where you have many more samples than you'd have if all you had to use was the target itself. You could even try to parallelize this to determine sensitivity to conditions by creating a number of setups all slight variations on the target. This should help to establish how reliable the output of the actual side-channel is.
What one could do though is greatly limit the search space for a password. It could definitely tell you how many characters and give at least areas of the keyboard.
Sure, it probably has not much real world impact. But you have to remember this is academic research, not industry security research. The point of academic research is to come up with non-standard ideas that are sort-of feasible, caveats are okay - but might become important in the future.
Right, and when someone made the first transistor, it was no smaller than a vacuum tube. Clearly impractical.
Give it time. A proof of concept does not represent the upper limit of what a technology can do.
Give it time. A proof of concept does not represent the upper limit of what a technology can do.
I don't see why you need realtime "decoding" of the vibrations. You record the motion, then later analyze it. I'm sure different kinds of work involve different patterns of typing (e.g. writing code uses different typing patterns than transcribing legal documents) and those patterns can be recognized after the fact to reconstruct most of the typing.
That said, there are much easier methods to go about obtaining keystrokes from a victim ...
That said, there are much easier methods to go about obtaining keystrokes from a victim ...
Our application instead detects and decodes keystrokes by measuring the relative physical position and distance between each vibration. We then match abstracted words against candidate dictionaries and record word recovery rates as high as 80%.
So we're all going to have to use constantly vibrating keyboards now to fool the accelerometers. Oh great.
So we're all going to have to use constantly vibrating keyboards now to fool the accelerometers. Oh great.
I'd be interested to see what happens if the keyboard is put on rubber grommets or a sponge mousepad.
It'd be tricky to get enough damping without making the typing action horrible, I think.
It'd be tricky to get enough damping without making the typing action horrible, I think.
Put your phone on something soft? I like to leave mine on a sponge mousepad so the vibration doesn't amplify through the table.
Actually, you probably want to do something similar to what Danny Hillis did with Babble. In his case emit speech sounding zero info sounds. In this case, vibrate similarly to a keyboard writing Lorem Ipsum (though not exactly that - then it could be filtered out). Best thing would be to train the babble-keyboard with the pattern of the specific user.
Or switch on the aircon, or play some music.
Sounds like a good time to me..
Random vibrations.
This is a real problem. You should be scared. Also, you should buy a VibraDamp Laptop/Keyboard mat from me. Protect yourself today! For added protection I will also make up and sell a VibraDamp Mini to keep your phone on!
Made out of the finest hatter's tinfoil!
I wouldn't submit such thing without checking how the accuracy changes when the phone is relocated, when the desk is changed or even when a different specimen of the same keyboard type is used. But I strongly agree that the sensor access privilege is deeply underrated -- Android doesn't even mention it!
This made me install an Android app (Seismograph) to test to see how my Nexus 4 picked up my key presses. I had it set to the maximum sample rate, with my phone on the desk as close to my laptop as possible whilst not touching it, and it couldn't pick up anything. I suspect therefore this is more of an issue for to-the-desk peripheral keyboards.
It does make me wonder though if the sound of each individual key press is subtle and unique enough to be identified.
It does make me wonder though if the sound of each individual key press is subtle and unique enough to be identified.
killnine: btw you have been hell-banned for the past 18 months. pretty harsh.
Perhaps it was the submission of tea party article about a forged birth certificate.
It's surprising to see so many hellbanned accounts that don't seem to know or care, and continue to post. I wonder if their posts get fake upvotes.
A very interesting generalization of Tempest [1].
Has anyone tried to recover keypresses from a video of someone typing? That would be an interesting challenge, I would think, especially if the video is at a poor angle.
[1] https://en.wikipedia.org/wiki/Tempest_(codename)
Has anyone tried to recover keypresses from a video of someone typing? That would be an interesting challenge, I would think, especially if the video is at a poor angle.
[1] https://en.wikipedia.org/wiki/Tempest_(codename)
Seems like a rare case where using this technique gives you much more entropy from a good passphrase than from a good password (although I guess over the shoulder watching is similar)
Ok, who wants to be the first person to build a usb powered 'thumper' which sits on your desk and raises the noise floor for vibrations and defeats this attack?
I would love to see how much better the detection would be if the accelerometer input was combined with the microphone.
[deleted]
Hopefully a disclosure like this prevents at least one patent.
That's won't work with the touchscreen keyboard.
I thought about a similar idea some weeks ago :d
In the real world, if you have compromised someone's phone and have access to their key strokes through bluetooth, why do you need this method at all? Even in some contrived circumstance where it was more feasible, it would probably be better 100% of the time to try to compromise the phone's bluetooth, audio, camera, or wifi capability before targeting the accelerometer and then hoping that they happen to leave their phone close enough to the keyboard and hope that your algorithm can decode keystrokes on a keyboard that it has never heard before, which may also be a type of keyboard that it has not heard before.
All in all, it's a cool experiment which shows that we can do some interesting and non-conventional stuff with accelerometers, but it's not practical in the slightest.