Extracting My Data from the Microsoft Band(jeffhuang.com)
jeffhuang.com
Extracting My Data from the Microsoft Band
http://jeffhuang.com/extracting_my_data_from_the_microsoft_band.html
30 comments
Is there any band that doesn't upload everything to some cloud? It's super crappy that everything I looked at had some stupid cloud feature.
I've always used a polar heart rate monitor when I play hockey to help me find my VO2 max and my target heart rate range. They're one of the few really accurate heart rate monitors out there. These also have a myriad of other functions.
http://www.polar.com/us-en/products/get_active/fitness_cross...
You can find a Polar Loop bracelet with the H7 heart rate monitor strap for around $150.00, which is a lot less than the Microsoft Band:
http://www.ebay.com/sch/i.html?_odkw=polar+h7&_from=R40&_osa...
I apologize in advance for the janky links.
http://www.polar.com/us-en/products/get_active/fitness_cross...
You can find a Polar Loop bracelet with the H7 heart rate monitor strap for around $150.00, which is a lot less than the Microsoft Band:
http://www.ebay.com/sch/i.html?_odkw=polar+h7&_from=R40&_osa...
I apologize in advance for the janky links.
I don't have experience with any, but after some searching, it appears the various Mio[1] bands might be a good bet? Appears to be more open that the polar products, unless I'm missing something. I also found:
https://github.com/mlt/schwinn810
Does anyone have any experience with either?
[1] http://www.mioglobal.com/en-uk/compare-mio-heart-rate-monito...
https://github.com/mlt/schwinn810
Does anyone have any experience with either?
[1] http://www.mioglobal.com/en-uk/compare-mio-heart-rate-monito...
And by "one of the few" you mean "one of the many". Heart rate monitoring is not terribly difficult or magical.
The Garmin Fenix can be mounted as a mass-storage device. It outputs in both "FIT" and GPX formats. If the proprietary Garmin software isn't used, it just appears like a USB key.
I just mount it with a Linux box and copy it whereever I want (including uploading to Garmin's cloud service).
I just mount it with a Linux box and copy it whereever I want (including uploading to Garmin's cloud service).
It is not just your health information. It is what you are doing.
It has a GPS and I suppose Wifi triangulation so they can track you outside and inside your house.
Are you running? climbing? walking? Your heart rate goes up with a very specific pattern.
Are you making love with this thing on? Also a clear heart rate pattern. If you remove this thing when you do it, it is also a very clear signal, normally you will put the device near you when you do, like on a table. Ohh, you know this thing has also a microphone?.
They can also track private conversations, like Snowden used to do in the NSA with smartphones. But smartphones use to be way farther than clockwatches to the mouth.
If you have a health problem, like a heart murmur, they also know, before you do.
Don't get me wrong. I would love to have something like this. But something I control, not someone else.
It has a GPS and I suppose Wifi triangulation so they can track you outside and inside your house.
Are you running? climbing? walking? Your heart rate goes up with a very specific pattern.
Are you making love with this thing on? Also a clear heart rate pattern. If you remove this thing when you do it, it is also a very clear signal, normally you will put the device near you when you do, like on a table. Ohh, you know this thing has also a microphone?.
They can also track private conversations, like Snowden used to do in the NSA with smartphones. But smartphones use to be way farther than clockwatches to the mouth.
If you have a health problem, like a heart murmur, they also know, before you do.
Don't get me wrong. I would love to have something like this. But something I control, not someone else.
> I wanted to track my heart rate while I run. I didn't want to let a large company have direct access to my health information.
This! FFS this!
I have yet to find a simple fitness tracker that doesn't literally hold your own data ransom, if they let you export your data at all, and many don't!
Every solution that I'm aware of involves MITM-ing the traffic, or something equally nontrivial. I'm looking at you FitBit!
FitBit does expose an API but IMO this is insufficient for exactly the reason parent stated: I don't want to let them have direct access to my health information.
Everyone in my circles who has a fitness tracker would prefer to be able to export their data without any additional fees, why do none of the trackers cater to this demographic?
This! FFS this!
I have yet to find a simple fitness tracker that doesn't literally hold your own data ransom, if they let you export your data at all, and many don't!
Every solution that I'm aware of involves MITM-ing the traffic, or something equally nontrivial. I'm looking at you FitBit!
FitBit does expose an API but IMO this is insufficient for exactly the reason parent stated: I don't want to let them have direct access to my health information.
Everyone in my circles who has a fitness tracker would prefer to be able to export their data without any additional fees, why do none of the trackers cater to this demographic?
I was just complaining the other day that I have to pay $59 to get my data out of fitbit. Jawbone claims that you can download your data for free though
“a large company” and their affiliates, business partners, anyone who acquires them, anyone who hacks them, and anyone who bribes or blackmails an employee to get access to the data.
(And, of course, the NSA. And their partners, employees, etc.)
(And, of course, the NSA. And their partners, employees, etc.)
Agreed; what I think to be an ideal solution would be a band with a USB interface that presents itself as a standard mass storage device, logs its data to one or more files stored on it, and its configuration is also accessible in the same way. NAND flash is cheap - and one of these could also double as a USB drive in the same way as a lot of media players. (Incidentally, I think adding a few extra sensors to an existing wearable media player platform wouldn't be all that difficult; now you get a band that can play music too...) That in no way precludes supplying an app that can automatically parse that data and generate pretty graphs etc.
Instead, the only products available from the big companies seem to require some proprietary app to interface them and give very little control of their data - the user's own health data - to the user, which is certainly disappointing and agrees with the trend of devices getting more and more locked-down against their users. They seem to be going out of their way to make it hard to "extract" the data from their devices, almost as if "plug it in and copy the data over" scares them.
I specifically mentioned "big companies" because what I find somewhat ironic is that it's the cheap, unbranded far-Eastern manufacturers who are more likely to come up with products based on existing standards than invent their own proprietary protocols. For example http://www.amazon.co.uk/Multifunction-Personallized-Signatur... is very similar to the concept I have in mind, and it's a completely non-cloud-based device, but the reviews show users complaining that it's "too difficult to use".
Instead, the only products available from the big companies seem to require some proprietary app to interface them and give very little control of their data - the user's own health data - to the user, which is certainly disappointing and agrees with the trend of devices getting more and more locked-down against their users. They seem to be going out of their way to make it hard to "extract" the data from their devices, almost as if "plug it in and copy the data over" scares them.
I specifically mentioned "big companies" because what I find somewhat ironic is that it's the cheap, unbranded far-Eastern manufacturers who are more likely to come up with products based on existing standards than invent their own proprietary protocols. For example http://www.amazon.co.uk/Multifunction-Personallized-Signatur... is very similar to the concept I have in mind, and it's a completely non-cloud-based device, but the reviews show users complaining that it's "too difficult to use".
If you go to the "Running" tab within the app, then swipe left to show the splits, you can touch the grey heart icon to see your heartrate during your run.
Of course, being able to download this data so you can slice/dice it yourself would be way better. It would also be nice for them to release an SDK so you can write applets for the band.
Of course, being able to download this data so you can slice/dice it yourself would be way better. It would also be nice for them to release an SDK so you can write applets for the band.
Well, theoretically you could simply add that to your hosts file permanently or patch their app to send the info to your server instead then write your own web application to handle it. Or fully reverse engineer the interaction with the hardware and write a new smartphone app instead.
Yeah, you can either mitm or just block prodwus0sts.blob.core.windows.net to prevent data from being PUT to Microsoft servers. In that case you'd want to decode the data stream (or access the BTLE sensor access protocol that some people have already reversed, but doesn't seem to be publicly documented anywhere yet)
It seems to use a standard odata format over ssl with oauth token security. I wonder if it's possible to simply attach an Excel worksheet to the data feed (https://support.office.com/en-us/article/Connect-to-an-OData...)
"Clearly, to get sleep events, the app is constructing a REST call."
This is gold :) Nice write-up, Jeff.
This is gold :) Nice write-up, Jeff.
I'd notice those URL parameters anywhere.
There is a very strong chance that it fully conforms to the OData spec: http://www.odata.org/documentation/odata-version-3-0/url-con... - although I'm not sure which version (most recent MSFT stuff has been 3.0).
So what you could do is hit a URL such as:
There is a very strong chance that it fully conforms to the OData spec: http://www.odata.org/documentation/odata-version-3-0/url-con... - although I'm not sure which version (most recent MSFT stuff has been 3.0).
So what you could do is hit a URL such as:
https://prodphseus.dns-cargo.com//v1/Events?$filter=AverageHeartRate gt 90
The source code he posted dealt with the ODataRequest class, so that's not a stretch :)
Also, the most recent OData version is 4.0: http://www.odata.org/documentation/odata-version-4-0/
Disclaimer: MS employee, not on OData team however.
Also, the most recent OData version is 4.0: http://www.odata.org/documentation/odata-version-4-0/
Disclaimer: MS employee, not on OData team however.
Where's the money here? It's not a subscription based service, so what financial sense does it make to dump this data straight to a server?
I can't imagine there's complex data processing being done that a smartphone can't handle, so I assume the data is being sent back because it's somehow useful or valuable to Microsoft.. but how?
I can't imagine there's complex data processing being done that a smartphone can't handle, so I assume the data is being sent back because it's somehow useful or valuable to Microsoft.. but how?
The network effect. Fitbit has a huge advantage right now because the user base is so large. A lot of people use its social features to compete with their friends, office mates, etc. This encourages more people to buy the hardware to participate.
From a user perspective, it makes sense to upload the data so that you're not constrained to your phone for viewing your info.
And Microsoft gets a good use case for Azure - I imagine they might use this in promoting Azure services.
As he already decompiled the app, wouldn't a more promising route be to figure out the Bluetooth communication between the app and the band? Using this knowledge, you could eventually write your own (private) app and bypass Microsofts's servers completely. (Provided they don't use some crazy authentification and/or encryption schemes in the Bluetooth protocol.)
I started poking around w/ mitmproxy the other day as well, since I had started to get a little tired of waiting (Microsoft has promised an open API/SDK of some sort, but there haven't been any updates to any of the software since release) w/ similar results. (I did this against the iOS app).
So I'll just post a couple notes:
* auth appears to be using OAuth WRAP (deprecated as a spec, but Microsoft appears to use it for Live logins), so I'm sure could be pretty easily extracted for an API library
* As mentioned the API mostly talks to an endpoint on and the returns are gzipped JSON except for a PUT to prodwus0sts.blob.core.windows.net for the binary log of your actual data (there's a subsequent PUT that then sends the UploadId and some other metadata to the API server)
People have mentioned wanting to avoid sending your data to the cloud completely, and that should be completely possible. The easy way atm is that you could just mitm the endpoints and sync as normal w/ the app.
However, there are at least a couple of people that have successfully reverse-engineered the BTLE protocol, although I haven't seen anything fully published yet. This appears to mostly/primarly be based on digging through the Windows client's DLL.
Pic of source w/ some of the BT protocol: https://twitter.com/JustinAngel/status/527955001436418048
Some BT functions: https://twitter.com/JustinAngel/status/528383467742957571
Methods extracted from the dll: https://twitter.com/JustinAngel/status/529876592479047682
(On OSX, strings gives you significantly less useful information, although apparently it was built by 'ianhowle' and there's a native Objective-C "CargoKit" library)
Note, there's one open source project that has theming and plans on building live sensor output: http://unband.nachmore.com/
And there's a closed source phone already that does access all the sensor data in realtime: http://www.windowsphone.com/en-us/store/app/band-sensor-moni...
I'm not too familiar with Windows Phone, but I believe you can access and decompile an unencrypted XAP if you have a rooted Windows Phone to see what it's doing.
I don't really have much experience/use/access to Windows stuff in general, but for someone w/ that kind of experience, I can't imagine it being very hard to deconstruct.
So I'll just post a couple notes:
* auth appears to be using OAuth WRAP (deprecated as a spec, but Microsoft appears to use it for Live logins), so I'm sure could be pretty easily extracted for an API library
* As mentioned the API mostly talks to an endpoint on and the returns are gzipped JSON except for a PUT to prodwus0sts.blob.core.windows.net for the binary log of your actual data (there's a subsequent PUT that then sends the UploadId and some other metadata to the API server)
People have mentioned wanting to avoid sending your data to the cloud completely, and that should be completely possible. The easy way atm is that you could just mitm the endpoints and sync as normal w/ the app.
However, there are at least a couple of people that have successfully reverse-engineered the BTLE protocol, although I haven't seen anything fully published yet. This appears to mostly/primarly be based on digging through the Windows client's DLL.
Pic of source w/ some of the BT protocol: https://twitter.com/JustinAngel/status/527955001436418048
Some BT functions: https://twitter.com/JustinAngel/status/528383467742957571
Methods extracted from the dll: https://twitter.com/JustinAngel/status/529876592479047682
(On OSX, strings gives you significantly less useful information, although apparently it was built by 'ianhowle' and there's a native Objective-C "CargoKit" library)
Note, there's one open source project that has theming and plans on building live sensor output: http://unband.nachmore.com/
And there's a closed source phone already that does access all the sensor data in realtime: http://www.windowsphone.com/en-us/store/app/band-sensor-moni...
I'm not too familiar with Windows Phone, but I believe you can access and decompile an unencrypted XAP if you have a rooted Windows Phone to see what it's doing.
I don't really have much experience/use/access to Windows stuff in general, but for someone w/ that kind of experience, I can't imagine it being very hard to deconstruct.
Interesting they store so little on the device. Does the Band generate a lot of data?
Even if it did, the interesting stuff can be filtered from the raw data and saved on the device.
Making customers depend on online service is a really hostile move. I wonder if the protocol between app and band can be reverse engineered to allow connecting other apps...
Making customers depend on online service is a really hostile move. I wonder if the protocol between app and band can be reverse engineered to allow connecting other apps...
Right. He's only reverse engineered the phone-to-server data, not the band-to-phone data.
Exercise equipment like treadmills should be able to read band data. They can use it for feedback to control speed. That's a common treadmill feature, but it's usually not wireless.
Exercise equipment like treadmills should be able to read band data. They can use it for feedback to control speed. That's a common treadmill feature, but it's usually not wireless.
The most amusing part for me is the domain name dns-cargo.com
Seems like a random choice. Wonder if this was just some spare throwaway domain they had laying around.
A nice hack and a good list of the things you may want to fiddle with if you want to explore phone applications calling home just in general.
i knew something had to be up when syncing between band and app required internet access! my fitbit always synched with just bluetooth.
So far despite the decent specs the Microsoft Band is disappointing to pretty much everyone I've spoken to who bought one. Now that I know how the data storage functions additional disappointment abounds. This is clearly no exception to equating the MS Band as the windows 8 of smartwatches. I am fanatically thrilled I couldnt find one when I wanted to buy one.
I wanted to track my heart rate while I run. I didn't want to let a large company have direct access to my health information.