I run into this pretty frequently with Docker on Mac OS X. Passing the `--rm` flag helps, but for cleaning up old/unused containers, I usually set these two commands on a cron:
# Remove un-tagged or un-named images
for im in $(docker images | awk '/<none>/ { print $3 }'); do docker rmi $im; done
# Remove stopped containers
docker rm $(docker ps -a -q)
Google has been around for 17 years and employees roughly 10,000+ software developers. I think it's reasonable to assume that the 2B LOC metric is accurate...
He mentions towards the end that there's a load-balancer that pauses the connection while the migration occurs. When the container comes back up, the traffic is redirected to the new instance and the connection resumes.
It also doesn't help that the Hadoop ecosystem is unstable, insecure, and overkill for the vast majority of the use-cases out there. Cloudera can also be thrown into the same bucket when they will no doubt file next year.
IMHO Hadoop just doesn't have a viable business model behind it. The people that really need a big data solution will run it themselves, and the people that don't (who will have it shoved down their throats all in the name of big data anyways) will unsuccessfully run it for a year or two before scrapping it for a better, more appropriate solution.