For elevation data, we store our DEM/DSM in S3 as LERC [1] COGS, LERC has a WASM bundle which I think can be used in the browser. We found LERC COGs to be one of the most space efficient ways of storing highresolution DEM/DSM data [2], If you wanted to you could fetch LERC tiles directly out of a remote COG and use that directly for the terrain heights.
I am more focused on storage/archiving/publishing of our LiDAR capture program [3] than web based visualizations of it though, so I am unsure if a LERC COG would even be better for you than a PNG TerrainRGB.
Your browser has a very powerful image decoder built into it, offloading the PNG decoding into Javascript is very resource hungry.
Using maplibre (or any map viewer) you can load blobs of image data out of a tiff and use `Image` or `Canvas` to render the data onto a map.
Its even easier if the tiffs are already Cloud optimized as they perfectly align to a 1-to-1 map tile and they don't need to be rescaled, you can then just render the images onto the map. eg here is a viewer that loads webps out of a 15GB tiff and uses Canvas to render them onto a map [1]
Unless you are trying to layer all your maps together, you also could stop reprojecting them into webmercator, or if your goal is to layer them, then storing them in webmercator would save a ton of user's compute time.
There are a bunch of us that talk web maping and imagery in the #maplibre and #imagery slack channels in OSMUS's slack [2]
I don't think geojson is a great format for anything with more than a few MB of data.
I wanted to see exactly how bad it is with a largeish datasets, so I exported the New Zealand address dataset[1] with ~2.5M points as a geopackage (750MB) QGIS loads this fine its a little slow when viewing the entire country but when zoomed into city level it is almost instant to pan around.
Using ogr2ogr I converted it to ndgeojson (2.5GB), It crashed my QGIS while trying to load it. Using shuf I created a random 100,000 points geojson (~110MB) it was unbearably slow in QGIS while panning around 5+ seconds.
I currently use and recommend flatgeobuf[2] for most of my working datasets as it is super quick and doesn't need sqlite to read (eg in a browser).
It is also super easy to convert to/from with ogr2ogr
if you also wanted to pull in a node framework for deserilization you could import something like `zod` to have the similar level of nice error messages when invalid input is encountered.
If you were in typescript, zod would also automatically generate your interfaces too.
New Zealand regularly uses flying companies to take aerial imagery photos of most of the country, which is generally taken between 30cm and ~2cm, Land Information New Zealand (LINZ) then releases this imagery imagery completely free (CC-BY)[1,2] to the public and can be downloaded as GeoTIFFs [3]
I think LINZ decided that <0.05m might have some privacy issues due to being able to distinguish people in it, and have held back releasing some <0.05m or they may have reduced the quality of it.
disclaimer: I work at Land Information New Zealand
Doing S3 put requests for 260M files every week would cost around $1300 USD/week which was too much for our budget
> or in ZIP format?
We looked at zip's but due to the way the header (well central file directory) was laid out it mean that finding a specific file inside the zip would require the system to download most of the CFD.
The zip CFD is basically a list of header entries where they vary in size of 30 bytes + file_name length, to find a specific file you have to iterate the CFD until you find the file you want.
assuming you have a smallish archive (~1 million files) the CFD for the zip would be somewhere in the order of 50MB+ (depending on filename length)
Using a hash index you know exactly where in the index you need to look for the header entry, so you can use a range request to load the header entry
offset = hash(file_name) % slot_count
Another file format which is gaining popularity recently is PMTiles[1] which uses tree index, however it is specifically for tiled geospatial data.
This is basically exactly what we do we have created a cloud optimised tar (cotar)[1] by creating a hash index of the files inside the tar.
I work with serving tiled geospatial data [2] (Mapbox vector tiles) to our users as slippy maps where we serve millions of small (mostly <100KB) files to our users, our data only changes weekly so we precompute all the tiles and store them in a tar file in s3.
We compute a index for the tar file then use s3 range requests to serve the tiles to our users, this means we can generally fetch a tile from s3 with 2 (or 1 if the index is cached) requests to s3 (generally ~20-50ms).
To get full coverage of the world with map box vector tiles it is around 270M tiles and a ~90GB tar file which can be computed from open street map data [3]
> Though even that would only work with a subset of compression methods or no compression.
We compress the individual files as a work around, there are options for indexing a compressed (gzip) tar file but the benefits of a compressed tar vs compressed files are small for our use case
New Zealand has 8G/8G to some homes within bigish cities for around $270nzd/mo (~180usd/mo)[1] and most of the population has access to at least 1000up/500down fibre for about $100nzd/mo (~65usd/mo).
It how ever is not the cheapest country in the world, so it somewhat depends on your definition of affordable
Looks to be a service to remove the HTTP referer header when linking to other sites.
Say your on example.com and click a a link to foo.com the browser will send the http header `Referer: example.com` in the HTTP get to foo.com, this means foo.com can then track how you came to their site.
This is pretty much exactly what we do to serve aerial/satellite imagery maps.
We convert the imagery into Cloud optimised geo tiffs and store them in S3 https://www.cogeo.org/ then the browser can request the tiles directly from S3.
For elevation data, we store our DEM/DSM in S3 as LERC [1] COGS, LERC has a WASM bundle which I think can be used in the browser. We found LERC COGs to be one of the most space efficient ways of storing highresolution DEM/DSM data [2], If you wanted to you could fetch LERC tiles directly out of a remote COG and use that directly for the terrain heights.
I am more focused on storage/archiving/publishing of our LiDAR capture program [3] than web based visualizations of it though, so I am unsure if a LERC COG would even be better for you than a PNG TerrainRGB.
[1] https://www.npmjs.com/package/lerc
[2] https://github.com/linz/elevation/tree/master/docs/tiff-comp...
[3] https://linz.maps.arcgis.com/apps/MapSeries/index.html?appid...