Deis 0.8.0 – CoreOS Integration(deis.io)
deis.io
Deis 0.8.0 – CoreOS Integration
http://deis.io/deis-0-8-0-coreos-integration/
11 comments
I speak for the whole Deis team when I say we're extra excited about this release as it lays the groundwork for our 1.0. Happy to answer any questions here.
What is a good(best) way to deploy n (dedicated)servers that consist of:
1.1-3 processes of 1 python program(these are worker-nodes,no webserver) (needs to be monitored so it's restarted if it fails)
2.a firefall to block everything incoming except me(the server-guy)
3.some enviroment-variables? for database connections and stuff
4.some kind of log management
5.server monitoring(disk,cpu,ram,network)
And a way to update the python program and the variables in all servers. Is that for saltstack / fabric ansible?
I asked you because this is just a part of Paas.
Thanks
1.1-3 processes of 1 python program(these are worker-nodes,no webserver) (needs to be monitored so it's restarted if it fails)
2.a firefall to block everything incoming except me(the server-guy)
3.some enviroment-variables? for database connections and stuff
4.some kind of log management
5.server monitoring(disk,cpu,ram,network)
And a way to update the python program and the variables in all servers. Is that for saltstack / fabric ansible?
I asked you because this is just a part of Paas.
Thanks
You can do everything you mentioned with Deis 0.8.0.
- Process management is handled the same way it is on Heroku, with a `deis scale` command. The fleet scheduler ensures processes are automatically restarted on host failure.
- How you secure network traffic is done differently depending on your environment (EC2, metal, etc). It's just a matter of configuration.
- Environment variables are managed using the `deis config` command, which creates new Docker images under the hood. We also support rollbacks of new builds and configuration changes using `deis rollback`.
- Log management is handled automatically for deployed applications. Developers can see their app's aggregated logs with `deis logs`. For platform/host logs, you can configure `journalctl` on CoreOS to drain whereever you like (Logstash, Splunk, etc)
- Server monitoring is often done at the hosting provider level (EC2 CloudWatch, etc). You can also configure agent-style host monitoring at the CoreOS level. We're currently working on some packaged solutions in this area.
There are some gaps in our documentation re: things like firewalling/SSL, monitoring, and draining platform logs. We are working to fill those documentation holes. But like I said, everything you mentioned is possible with Deis today. If you have further questions, hit me up at gabriel _at_ opdemand.com.
- Process management is handled the same way it is on Heroku, with a `deis scale` command. The fleet scheduler ensures processes are automatically restarted on host failure.
- How you secure network traffic is done differently depending on your environment (EC2, metal, etc). It's just a matter of configuration.
- Environment variables are managed using the `deis config` command, which creates new Docker images under the hood. We also support rollbacks of new builds and configuration changes using `deis rollback`.
- Log management is handled automatically for deployed applications. Developers can see their app's aggregated logs with `deis logs`. For platform/host logs, you can configure `journalctl` on CoreOS to drain whereever you like (Logstash, Splunk, etc)
- Server monitoring is often done at the hosting provider level (EC2 CloudWatch, etc). You can also configure agent-style host monitoring at the CoreOS level. We're currently working on some packaged solutions in this area.
There are some gaps in our documentation re: things like firewalling/SSL, monitoring, and draining platform logs. We are working to fill those documentation holes. But like I said, everything you mentioned is possible with Deis today. If you have further questions, hit me up at gabriel _at_ opdemand.com.
This is a great example of how to do release announcements.
First sentence gives a broad summary of what is new. Next paragraph explains what Deis is. It could be a little longer, but by the end you get the gist what it does. Then the features, what changes could affect you, then known issues, roadmap, and finally a "Here's how you can help".
First sentence gives a broad summary of what is new. Next paragraph explains what Deis is. It could be a little longer, but by the end you get the gist what it does. Then the features, what changes could affect you, then known issues, roadmap, and finally a "Here's how you can help".
Awesome! I've been playing with all the components of the new Deis (CoreOS, etcd, fleet, docker) over the last few weeks and I -really- like them. This might be the future of how we do devops (I hope so, at least).
Thanks for this great update!
Thanks for this great update!
CoreOS is already composed of etcd, fleet, docker - can you say more about how Deis is different?
Docker, etcd, fleet are powerful tools, but they don't magically solve problems for software teams the way a platform like Heroku does.
Deis is designed to tie all of the above into a streamlined workflow for developers. Specifically, we provide developers with the ability to:
-Create discrete applications
-Push code or existing Docker images
-Implement Docker-based CI/CD pipelines
-Set environment variables
-Manage releases and rollbacks
-Run ephemeral containers
-View aggregated logs
-Scale containers
-Collaborate with a team
...all without having to think about underlying tooling or infrastructure.
Deis is designed to tie all of the above into a streamlined workflow for developers. Specifically, we provide developers with the ability to:
-Create discrete applications
-Push code or existing Docker images
-Implement Docker-based CI/CD pipelines
-Set environment variables
-Manage releases and rollbacks
-Run ephemeral containers
-View aggregated logs
-Scale containers
-Collaborate with a team
...all without having to think about underlying tooling or infrastructure.
Absolutely fantastic. Thank you!
"Documentation for .... bare-metal provisioning are forthcoming."
That's kind of a downer, hoping for a sooner release.
"Documentation for .... bare-metal provisioning are forthcoming."
That's kind of a downer, hoping for a sooner release.
It's definitely a priority! The focus on 0.8.0 was the rewrite of our scheduler to use CoreOS. My focus for 0.9.0 is thinking about high availability and ensuring the provisioning process is as smooth as possible.
That said, anywhere CoreOS can run, Deis can run. There are existing CoreOS guides for PXE booting[1] and iPXE booting[2]. I'm planning to adapt these into Deis provisioning scripts, but if you want to take a stab at provisioning Deis on bare metal in the meantime, let us know how we can help!
[1] http://coreos.com/docs/running-coreos/bare-metal/booting-wit...
[2] http://coreos.com/docs/running-coreos/bare-metal/booting-wit...
That said, anywhere CoreOS can run, Deis can run. There are existing CoreOS guides for PXE booting[1] and iPXE booting[2]. I'm planning to adapt these into Deis provisioning scripts, but if you want to take a stab at provisioning Deis on bare metal in the meantime, let us know how we can help!
[1] http://coreos.com/docs/running-coreos/bare-metal/booting-wit...
[2] http://coreos.com/docs/running-coreos/bare-metal/booting-wit...
I've been playing with Deis the past few weeks and really like it. There has been a lot to learn (CoreOS, Etcd, Fleet, Docker), but the documentation makes it quite easy to get a cluster up and running. The Deis folks have also been very friendly and helpful on Github and IRC. If you're having any trouble I strongly recommend reaching out to them. Keep up the good work!
Definitely going to re-evaluate deis. I like chef but I like it for SCM - using it for cluster management via databags was interesting but that problem is better solved elsewhere (i.e fleet, flynn's layer-0 or mesos).
When we first released Deis, we were most interested in answering questions around workflow and how you should design a PaaS that leverages Docker from the ground up. We used node convergence and Chef databags because it worked and alternatives weren't available.
Fast forward to today... CoreOS/Fleet and Mesos are finally stable enough to build upon. They offer simpler, faster and more reliable ways for managing containers across distributed systems. Meanwhile, we've been able to hone our workflow and tooling using the legacy cluster management implementation. We're a better platform because of it.
TLDR; we _love_ Chef -- just not for managing containers across a distributed system.
Fast forward to today... CoreOS/Fleet and Mesos are finally stable enough to build upon. They offer simpler, faster and more reliable ways for managing containers across distributed systems. Meanwhile, we've been able to hone our workflow and tooling using the legacy cluster management implementation. We're a better platform because of it.
TLDR; we _love_ Chef -- just not for managing containers across a distributed system.
Love the idea of the "easy-fix" tag on GitHub to encourage developer involvement in the project. Best of luck with the project!
What is the recommended way to handle database migrations with deis? In this case I don't want to automatically deploy on push.
You can handle database migrations the same way as you would on Heroku. Deploy the application, then do the migrations afterwards with `deis run`.
Looks awesome!
With this release, what would be the difference between Deis and Flynn?
I think the biggest difference is that Flynn is building a scheduler modeled after Google's Omega paper (what they call Layer-0) along with the Heroku-style platform components (which they call Layer-1) -- all written from scratch in Go.
Deis, on the other hand, is laser focused on the platform problem. We're also not shy about using off-the-shelf components where they make sense. For example, we believe the distributed scheduler/init system is best left to others like CoreOS/Fleet.
Deis, on the other hand, is laser focused on the platform problem. We're also not shy about using off-the-shelf components where they make sense. For example, we believe the distributed scheduler/init system is best left to others like CoreOS/Fleet.
gabrtv is correct that Flynn is built on a scheduling framework (think Mesos, but based on the Omega paper), which allows us to support running stateful services (like databases) inside of the platform.
Flynn develops most core technologies internally. We try to minimize our external dependencies, as the majority of instability we've run into has historically come from dependencies.
Building modular, reusable, and repurposable components is also a major design goal of ours (for example, Deis uses a few Flynn components).
Flynn develops most core technologies internally. We try to minimize our external dependencies, as the majority of instability we've run into has historically come from dependencies.
Building modular, reusable, and repurposable components is also a major design goal of ours (for example, Deis uses a few Flynn components).
Great release, deis team!