Map Your Google Location History(nbviewer.ipython.org)

nbviewer.ipython.org

http://nbviewer.ipython.org/github/chrisalbon/code_py/blob/master/matplotlib_map_your_google_data.ipynb

22 comments

tghw12년 전

For those who don't want to do it themselves: https://maps.google.com/locationhistory/b/0

Actually, the Google-provided version restricts you to only 30 days. If you're interested in everywhere you have been, ever (according to Google), then this is quite awesome and you get much more options with how you display that data and what else you can incorporate with it.

thekos12년 전

You can go back further by modifying the startTime parameter in the download link.

https://maps.google.com/locationhistory/b/0/kml?startTime=14...

jacquesm12년 전

Oh my, a hacker. Better be careful.

http://theblot.com/man-jailed-because-feds-dont-understand-c...

Tiksi12년 전

I did something similar with that data a few months ago using qGIS, postGIS, and OSM data:

https://packetfire.org/projects/qgis/img/lines.crop.png

https://packetfire.org/projects/qgis/img/points.and.lines.cr...

https://packetfire.org/projects/qgis/img/too.many.points.png

The data is definitely not completely consistent. Trips I took are missing, even though Google plus made auto-awesome stories with pictures taken there, and got the locations correct in those. There are also some points that shouldn't be there. It shows me in Michigan a few separate times (a few hundred data points at various times), to which I've never been. It also puts me at 0 Lat/0 Long a few times, but the reason for this is pretty obvious, just confusing when I first saw it mapped. Also there's quite a bit of "teleporting" from location to location, even though I had my phone with me and was using it when I was traveling between them.

The third image, on the other hand, is seemingly way too much data at one place. It's my workplace at 1 meter resolution. You'd think they could merge it down to a few points a day at the most, but for whatever reason, there are on average 200 points per day there. It just astonishes me how much data there is at some places, yet in others massive chunks seem to be missing.

I started working on a write up of how I did all this, including the bash scripts I wrote to parse, clean and transform the data, but one thing came up after another, and I haven't gotten around to finishing it. If anyone is interested, I'll finish that up and post it this week. I still have to finish up my scripts to clean the data of the "teleporting" and the places I haven't actually been. It was definitely a fun project, and now I have an interactive map of (some of) my past travels and habits. qGIS was interesting to play with, learning bits as I went along trying to get it to display properly and actually render what I wanted to see. Definitely gave me a new level of respect for anyone that does cartography.

Bassetts12년 전

I have messed around with mapping my location history and had some similar problems. Apparently I have been to Essex, UK although I have never been there in my life.

I tracked down the teleporting problem to the fact my tablet was at home and turned on. It seems you can't set location reporting per device, it is either on or off, so you end up being in two locations at once. I was going to look in to trying to remove problem points be setting a distance or speed threshold. That brings the problem of knowing which data point to keep and which to drop though, which I have not got a solution for yet apart from requiring user interaction.

Tiksi12년 전

In my case, my phone is the only device should report location data, and the times on either side of the teleport were correct, it just seems to be missing the trip itself. As far as removing the points, there seem to be plenty of points at the a similar time and location, so removing either one wouldn't be an issue. However it's currently one continuous track, and removing a point would break that.

My idea was to use speed as the constraint, give the points on both sides a label, and have the line between them be transparent. I just need to figure out that last part.

edent12년 전

If you want to get your data programatically, Google do not provide an API.

You'll need to "borrow" your own cookies. I documented the process at http://shkspr.mobi/blog/2014/04/extracting-your-own-location...

Tiksi12년 전

I once tried to quickly hack together some bash using "Copy as curl" in firebug to do something similar and got this shameful monstrosity after a minute or two of fighting with awk:

       for i in {10..14};do for j in 1 7; do date -d "20$i-0$j-01T00:11:20.000Z" "+%s000";date -d "20$i-0$j-01T00:11:20.000Z" "+%s000"|tr "\n" " "; done;done|sed '1d;$d'|awk '{system("curl \047https://maps.google.com/locationhistory/b/0/kml?startTime=" $1 "&endTime=" $2 "\047 -H \047Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\047 -H \047Accept-Encoding: gzip, deflate\047 -H \047Accept-Language: en-US,en;q=0.5\047 -H \047Connection: keep-alive\047 -H \047Cookie:cookie-stuff\047 -H \047Host: maps.google.com\047 -H \047User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0\047 -o " $1 ".kml")}'

(And the above doesn't look quite as bad as what I was doing without the actual cookie data there, so that's saying something.)

At which point I took a few minutes to think deeply about life and what possessed me to write such an obscenity, and decided to use Takeout to get the data instead.

Edit: That's not to say it didn't work, it gave me all the data. It was just a bad way to do it.

adminprof12년 전

Thank you. I also spent a lot of time looking for a programmatic way to access the data after Latitude's demise, and decided that using another app was easier.

dochtman12년 전

You can just use Google Takeout, IIRC.

edent12년 전

Takeout does not provide an API.

sahaskatta12년 전

I used a heatmap API with 2+ years worth of Google Location History data to make this last year: https://plus.google.com/+sahaskatta/posts/5bbn9D9ZzuP

IneffablePigeon12년 전

This inspired me to do the same. If anyone wants to try, I put my implementation up on github: https://github.com/olane/locationheatmap

dochtman12년 전

I built my own version of this about two years ago:

https://dirkjan.ochtman.nl/writing/2012/11/28/tracing-a-path...

Python code available here: https://github.com/djc/galatna

acadien12년 전

Very cool script! I just tried it out myself and it seems that Google's location history is actually really spotty. It missed several of my trips (during which the phone was active and responsive) and it also included some spots that I've never been to. I double checked the JSON and the problem is on Google's end and not the script.

ChrisArchitect12년 전

I've done this before by just downloading location history as KML and then hosting somewhere and directing google maps at it in an embed etc. works fine.

shaurz12년 전

So apparently I've been to Hong Kong... first I've heard of it. Also there seems to be no record of my visit to Canada last year.

dzhiurgis12년 전

Am I the only one whose location data is missing?

icebraining12년 전

Maybe you've disabled the feature?

https://support.google.com/gmm/answer/3118687?hl=en

dalek2point312년 전

how does python know what map data to use?

markveronda12년 전

It's fed in by the JSON file that was downloaded in Step 1 from Google when you request an archive of your data.

From the site: # Create a dataframe from the json file in the filepath raw = pd.io.json.read_json('/Users/chrisralbon/Downloads/Location History/LocationHistory.json')