There are a lot of developments around using Kubernetes as an IaC platform for the reasons in your comment. The combination of a standard API model in CRDs + the controller model maps nicely to managing infrastructure and exposing resources to developers.
<https://crossplane.io> just graduated to CNCF Incubation and each of the cloud providers are working on K8s controllers and code generators (like Amazon Controllers for Kubernetes, Google Config Connector, and the Azure service operator).
Before you leave, I'd love to talk to you. I'm in STL and looking for engineers to work in a professional, supportive, and inclusive environment. [email protected]
In this case the task resource http://converge.aster.is/0.5.0/resources/task/ might help, as it allows you to create a directed graph using any kind of interpreter (for example, Python or Ruby) instead of having to use the DSL.
We have started work on exposing Hashicorp Vault secrets via FUSE and Docker volumes. The expectation is your containers will just mount secrets via a mount like /secret in the container.
tl;dr: mesos/marathon run the tasks, consul exposes the tasks in DNS.
Both Mesos and Marathon store different views of task state on the cluster.
How do we use this data to make it easy for jobs to find one another? Can we use this information to automatically configure things like Load balancers?
This is where tools like consul and mesos-dns come in. They populate a DNS store, so that task-name becomes something like task-name.example.com. If you are running 10 copies of a service across different hosts, DNS will have 10 entries.
If a container moves to another system, DNS is updated on the fly.
We can embed health checks with consul, so that if a service is unhealthy, it gets pulled out of DNS.
Consul also is nice because the edges perform the checks (instead of a central server), so the load is distributed.
Why you should care about this project:
You wouldn't spend months building your own Linux distribution, so why spend time reinventing the wheel with chef/puppet/ansible?
We're not building a PaaS. Like a Linux distribution, we're integrating open source building blocks (logging, service discovery, scheduling) so that you can focus on app development and analytics instead of sysadmin tasks.
Here's how you would deploy it in microservices-infrastructure:
1. You'd deploy the cassandra mesos framework, giving you HA cassandra. Instead of setting IPs, you'd connect to this as cassandra.service.consul
2. You'd launch a Kong container in marathon. It would show up in dns as kong.service.consul for other apps to find, so you don't have to hard code IPs in your config.
We like Kubernetes (and are looking to add it to our project), but our goal is to integrate building blocks that allows us to run many different types of workloads. Think of our project more like an Ubuntu for distributed systems.
Kubernetes may eventually spread out beyond Docker, but for today we need to support things like Kafka and Spark.
As others have noted, we've had things like CloudFoundry, OpenShift and Heroku, and these all-in-one frameworks tend not to extend outside their original domain.
The real problem is going from tutorial to something you would use in production. Throw in logging, security and service discovery and you can have a few engineers hacking away for months.
We're trying to make it super easy to deploy these tools. For example every time you launch a docker container, it will register with consul and be added to haproxy. The nice thing about using Mesos is we can support data like workloads Cassandra, HDFS, and Kafka on the same cluster your run Docker images on.
We use terraform to deploy to multiple clouds so you don't get locked in to something like cloudformation.
Our goals for this tool is to make it dead simple to deploy and run. The binary and a json config file is all you need to check a host. When you combine it with consul, we can integrate service discovery with host monitoring and push checks out to the edge nodes instead of relying on something like Nagios.
We were already using consul as our service discovery engine and as a data store for dynamic configurations via consul template. We feel Vault will help us manage secrets on our infrastructure and inside containers.
As someone who is contributing to this project, what we are focusing is making it easy to run and connect multiple containers and applications. We do this by:
1. Building in service discovery. This means that containers register automatically with DNS (and mesos tasks will register with consul in our next version)
2. Distributed scheduling. We use mesos to manage the running of containers and other processes. This means you can just use a an API call to launch a container, and that container will register itself in DNS.
3. Framework support. One of the coolest features of Mesos is the ability to launch containers that easily run things like Kafka, HDFS and Cassandra clusters without having to use tools like chef or ansible.
Please let me know if you have any questions. I'm really excited about this project!
I've been using gitblit for a few months, and have been very pleased with how easy it is to set up and manage. It's great to set up http sharing of git repositories inside an organization.
It has nice features like ACL's and federation for backup.
Each card (not core) is running an embedded version of Linux. Binaries are almost all linked to busybox. You can mount NFS shares to share data.
The embedded systems need specially compiled binaries. With the Intel compiler, this is usually achieved by turning on a flag at compile time. So yes, you can install and run your own software on the nodes.
<https://crossplane.io> just graduated to CNCF Incubation and each of the cloud providers are working on K8s controllers and code generators (like Amazon Controllers for Kubernetes, Google Config Connector, and the Azure service operator).