Check out this video that goes into a very deep technical explanation about how the satellites can be used as a Synthetic Aperature Radar to build a realtime representation of the entire globe at meters of resolution: https://youtube.com/watch?v=jbp3kdJZ1_A
It's interesting that this feels awkward to you, because when apps don't function this way it feels broken and odd to me. When I tap a PDF attachment in an email I expect the back button to go back to the email I was just viewing, not the list of PDFs on my phone. If I wanted to view all the PDFs on my device, I would start at the PDF viewer and tap into PDFs from there.
I wonder what experience made this feel more awkward for you (and conversely, why it feels more natural for me). What a weird/complex world we live in!
I love seeing all the great comments in here about the different APIs and the features they do and don't offer, but I want to point out that the underlying data for addresses is incredibly hard to find. The reason the commercial geocoding providers won't let you store their data is because they're worried you'll store enough data to build your own geocoder.
To help with this, a group of folks (including me) started OpenAddresses (https://openaddresses.io/ and https://github.com/openaddresses/openaddresses/) with the goal of finding every open address dataset in the world. We produce a zip file with 100M's of addresses that several of the APIs mentioned in this thread use as a major part of their dataset. We've been going for well over 10 years now, but it would be great to have more eyes looking for more address sources. Check us out!
It boils down to the fact that the United States does not have (public domain) knowledge of every address in the country.
The USPS knows about deliverable addresses but won't give that information to the federal government because then it'd be public domain and they would lose several of their primary data moats (Zipcodes, addresses, delivery routes, for example). The Census has very complete knowledge of every address, but won't give it up because it's illegal (see Title 13 of the US Code). There is an ongoing attempt by the DOT to collect a National Address Database (https://www.transportation.gov/gis/national-address-database) by collecting information from the address assigning authorities (usually county governments), but it's incomplete and unlikely to ever be complete because of holdout/underfunded local governments.
There are several address datasets that are private (Google has a fairly complete one, FedEx/UPS probably have the most complete, TomTom, CostQuest, etc.). I started https://openaddresses.io/ to try and collect them (NAD is based off this idea) into an open-licensed dataset.
The broadband companies have records that say "this address is connected to this network, which could theoretically have this service level", but (a) they won't/can't tell you where they think the address is and (b) won't spend the time to match their address string format with the government's address because both are private data.
Finally, without the address -> location data, even if we could get broadband providers to tell us what service is available at each address, we couldn't put that service level on a map because we don't know where the address is.
----
The Markup published some work in 2022 where they used OpenAddresses to use ISP's own tools to gather per-address service offerings and put them on a map. This is what the FCC's broadband map should be doing, but can't for the above (and political) reasons: https://themarkup.org/show-your-work/2022/10/19/how-we-uncov...
Interestingly, it wasn't the Obama administration that tried to build the high speed rail. It was the Republican governor of Wisconsin (Tommy Thompson) that pushed for it. Scott Walker used it as a wedge to split the electorate and win the election. Wisconsin Public Radio did a great podcast about it here: https://www.wpr.org/derailed/wisconsins-high-speed-rail-saga...
OSM editors are prevented from using Google Maps because OSM prefers that we have explicit permission to use data sources. Since we don't have explicit permission to use Google Maps, we can't use it.
Separately, Google Maps has a terms of use that prevent reuse of Google Maps data. You agree to those terms when visiting Google Maps or using Google Maps API.
You wouldn't be breaking copyright law when copying from Google Maps to OSM, you'd be breaking the terms of use contract with Google and community Norma expectations in OSM.
As someone who has spent a considerable amount of time on data that has ended up on this page, I think the fact that the list hasn't grown says more about the priorities of other companies than of AWS. Amazon doesn't (yet) have time to build and maintain these datasets themselves: they work with others to build and maintain it and then fund the storage and transmission fees.
I helped build the Terrain Tiles dataset as part of Mapzen, which recently shut down. The OpenStreetMap data exists on the AWS Public Datasets page because it's useful to Humanitarian OpenStreetMap Team. If you're able to convince your company to generate and work with a public dataset, consider reaching out to the AWS and Google public datasets teams to get it hosted and publicized.
Hello. I am the creator of alltheplaces and Max is absolutely correct. Alltheplaces is a bunch of scrapers right now because it was the fastest way to get the data, but if you have data in a machine readable format already then I don't have to scrape it. I can just download your data file and distribute it to whomever wants to use it (people interested in importing to OSM, geocoders/search engines, etc.).
It'd be great to have this data in Internet Archive. If you check out our GitHub repo [0] you can see where we coordinate finding data sources. The data I download gets composited into tiles for display on maps, so we're updating ~4 billion objects in S3.
The source data is probably the better thing to include in IA, and the GitHub repo is probably the best place to find how to mirror it. If you've got time to spend on it, you might post an issue in there and I can help point you in the right direction.
I want to give another example: I'm currently updating the Terrain Tiles dataset [0] and a very significant portion of my time was spent working around terrible government data download websites or incompatible distribution formats.
For example, the UK Government flew some excellent LIDAR data missions generating a very high resolution elevation model for most of the country and then put it behind a terrible website where you have to click 3 or 4 times to get a small piece of the data. After a couple hours I built a script to download all the pieces and put them back together into usable sized downloads [1]. Mexico's INEGI has a similar situation, so I had to dig through that to build a scraper [2]. USGS's EarthExplorer uses a terrible shopping cart metaphor for download [3].
All that is to say that the interesting piece with Earth on AWS is that this is public data that smart people are putting in a more easily accessible place for mass consumption and AWS is footing the bill. In return AWS is getting people more interested in AWS products and a set of customers that are more knowledgeable about how to process data "in the cloud".
Check out OpenAddresses: we're trying to solve the address/location data aggregation problem so you don't have to. We do all the data scraping and collating for you so you can work on making a geocoder. https://openaddresses.io/
All addresses are pretty hard for computers to understand, so the people building the software to make them understand (geocoders) usually make them work for the most 'in-demand' address formats. Just like every other bit of software, finding the special cases and figuring out how to understand them is tricky.