A simple optimization can probably make things much better. If you think of it, the delta between panaromas corresponding to two adjacent points is very little. So, compressing image data across panaromas instead of each separately, I think, will make it _so_ much better.
Interesting thought exercise, thanks.