That said, we're not actually doing the 'chasing db' config. Instead we run a HA Neo4j DB deployment as a Marathon service pegged to a handful of nodes each with local persistent volumes allocated to Neo4j. I.e. we can allocate a % of a node's resources to 'static' Neo4j deploys, and then let Marathon dynamically manage any remaining free resources on the nodes. Our other services then use the Marathon service DNS to look up the Neo4j service for read/write.
Portworx looks cool too -- will need to investigate.
> "Does Mesosphere reschedule your DB to another node that has an equivalent persistent and reserved storage(SSD etc.) volume configured on it?"
Yes/it can, but in that config you're booting up new/empty storage volumes. Obviously not what you want for many core persistence requirements though great for caches. We'll probably opt for this config near-term for our web-server SSR cache.
Currently running stateful apps on a DC/OS cluster + HA DBs. Fairly straight forward to get stateful working now + there are libs to migrate data around to chase your apps/services when/if Mesos/Marathon relocates them (e.g. after a service restart/crash) if you need your data co-located with a managed app/service.
The thing I'm most interested in exploring with K8+DC/OS is having DC/OS manage a couple of K8 instances so we can isolate 'virtual' clusters for various envs/apps. I suppose you could do this already with Marathon (the DC/OS built-in container manager), but we're not. Psyched to benefit from the K8 community + have the underlying DC/OS VM control/management plane.
Accidental release should be a major consideration. Also, there are a lot of crazy people out there who do a lot of crazy things for crazy reasons - their logic would either not make sense or be incomplete, but it wouldn't matter to them.
That said, I think biotech is an amazing force for good in the world, we just need to be responsible with how we deal with it. Which means defining and justly implementing 'responsible' is a core tech challenge.