Scaling Mesos: A Perspective Featuring Mesosphere, Airbnb, Ericsson and Others(meetup.com)
meetup.com
Scaling Mesos: A Perspective Featuring Mesosphere, Airbnb, Ericsson and Others
http://www.meetup.com/Bay-Area-Mesos-User-Group/events/216433322/
7 comments
I'm very interested in how Mesos can serve up http in a Highly Available way to all these services that run internally in the cluster. Has that problem been solved and will that be on topic at the meetup?
Marathon is a Mesos framework for long-running "always on" apps that can serve up http among other things.
It ensures that an app stays up even when machines or entire racks fail.
It ensures that an app stays up even when machines or entire racks fail.
We'll talk briefly about this for sure and, time permitting, may demo an example.
As it happens, we're in the process of writing a more in-depth doc about this topic and it should be live later this week at http://mesosphere.com/docs/getting-started/service-discovery.... Ping us at [email protected] if you have any specific questions after it goes live!
As it happens, we're in the process of writing a more in-depth doc about this topic and it should be live later this week at http://mesosphere.com/docs/getting-started/service-discovery.... Ping us at [email protected] if you have any specific questions after it goes live!
Awesome, curious to see how Airbnb used this with Hadoop
[deleted]
[deleted]
Schedule:
6:30 - 7:00 Networking and Light Dinner
7:00 - 7:10 Announcements
7:10 - 7:40 Mesosphere: Deploying Docker Containers at Scale with Mesos and Marathon
Abstract: The norm these days is to operate apps at web scale. But that’s out of reach for most companies. Marathon is an open source project that makes it considerably easier to deploy Docker containers at web-scale using Apache Mesos.
See how Mesos and Marathon can help deploy and manage Docker containers at scale and how the Mesos cluster scheduler builds highly-available, fault-tolerant web-scale apps.
Bio: Sunil studied Computer Science at the University of Cambridge, where he wrote his dissertation on delay-tolerant networking systems. After four months spent cycling across Africa he returned to London to join a major investment bank and then Last.fm as a Data Engineer. Last year his interest in robotics brought him to UC Berkeley, where he completed a Master of Engineering degree with a focus on distributed computing and unmanned aerial vehicles, working in his spare time on an UAV startup. After leaving Cal, he joined Mesosphere as a Distributed Applications Engineer and is currently working on easy deployment of Mesosphere clusters on Google Cloud Platform.
7:40 - 8:10 Airbnb: How we're using Mesos at Airbnb today
Abstract: Mesos is the operating system for Airbnb's data platform. We use Chronos, Marathon, Hadoop, and other frameworks for operating a complex system of data warehouse and analytics tools. In this talk, we'll discuss how some of the day-to-day operational problems are solved using these systems.
Bio: Brenden Matthews is a software engineer at Airbnb, working on the data infrastructure team. He is an Apache Mesos contributor, the primary maintainer for Chronos and Hadoop on Mesos, and has also contributed to related frameworks such as Storm, Spark and Marathon. Previously, he has given talks at about Mesos at Twitter, QCon, and MesosCon.
8:10 - 8:30 Ericsson: On the use of Mesos and Marathon for Multi-Datacenter Service Orchestration
Abstract: With the availability of numerous geographically distributed datacenters, there is an increasing need for highly distributed service orchestration. Apache Mesos and Marathon are valuable starting points for further exploration towards a system enabling such service orchestration while achieving high datacenter utilization and providing high availability. However, both frameworks are designed and used for (resp. on) a single datacenter.
In this talk I will present prospectives on leveraging Mesos and Marathon as building blocks to orchestrate Docker-based services across multiple datacenters.
Bio: Eugen is a Senior Research Engineer at Ericsson Research Silicon Valley Lab. He currently builds systems which leverage real-time big-data analytics for improved resource management in large-scale datacenters. Prior to that, he was a postdoc in the Data Intensive Systems Group of the Advanced Computing for Science Department at the Lawrence Berkeley National Lab (LBNL). He holds a PhD in Computer Science from the University of Rennes 1, France which he received while working in the Myriads INRIA project-team. During his PhD he worked on systems and algorithms for autonomic and energy-efficient resource management. Eugen received his Bachelor and Master of Science degrees in Computer Science from the Heinrich Heine University Duesseldorf (HHU), Germany. He was a student research assistant in the HHU operating systems group, interned in the Myriads INRIA project team and was a summer intern at the LBNL.
8:30 - 9:00 Common Q&A
6:30 - 7:00 Networking and Light Dinner
7:00 - 7:10 Announcements
7:10 - 7:40 Mesosphere: Deploying Docker Containers at Scale with Mesos and Marathon
Abstract: The norm these days is to operate apps at web scale. But that’s out of reach for most companies. Marathon is an open source project that makes it considerably easier to deploy Docker containers at web-scale using Apache Mesos.
See how Mesos and Marathon can help deploy and manage Docker containers at scale and how the Mesos cluster scheduler builds highly-available, fault-tolerant web-scale apps.
Bio: Sunil studied Computer Science at the University of Cambridge, where he wrote his dissertation on delay-tolerant networking systems. After four months spent cycling across Africa he returned to London to join a major investment bank and then Last.fm as a Data Engineer. Last year his interest in robotics brought him to UC Berkeley, where he completed a Master of Engineering degree with a focus on distributed computing and unmanned aerial vehicles, working in his spare time on an UAV startup. After leaving Cal, he joined Mesosphere as a Distributed Applications Engineer and is currently working on easy deployment of Mesosphere clusters on Google Cloud Platform.
7:40 - 8:10 Airbnb: How we're using Mesos at Airbnb today
Abstract: Mesos is the operating system for Airbnb's data platform. We use Chronos, Marathon, Hadoop, and other frameworks for operating a complex system of data warehouse and analytics tools. In this talk, we'll discuss how some of the day-to-day operational problems are solved using these systems.
Bio: Brenden Matthews is a software engineer at Airbnb, working on the data infrastructure team. He is an Apache Mesos contributor, the primary maintainer for Chronos and Hadoop on Mesos, and has also contributed to related frameworks such as Storm, Spark and Marathon. Previously, he has given talks at about Mesos at Twitter, QCon, and MesosCon.
8:10 - 8:30 Ericsson: On the use of Mesos and Marathon for Multi-Datacenter Service Orchestration
Abstract: With the availability of numerous geographically distributed datacenters, there is an increasing need for highly distributed service orchestration. Apache Mesos and Marathon are valuable starting points for further exploration towards a system enabling such service orchestration while achieving high datacenter utilization and providing high availability. However, both frameworks are designed and used for (resp. on) a single datacenter.
In this talk I will present prospectives on leveraging Mesos and Marathon as building blocks to orchestrate Docker-based services across multiple datacenters.
Bio: Eugen is a Senior Research Engineer at Ericsson Research Silicon Valley Lab. He currently builds systems which leverage real-time big-data analytics for improved resource management in large-scale datacenters. Prior to that, he was a postdoc in the Data Intensive Systems Group of the Advanced Computing for Science Department at the Lawrence Berkeley National Lab (LBNL). He holds a PhD in Computer Science from the University of Rennes 1, France which he received while working in the Myriads INRIA project-team. During his PhD he worked on systems and algorithms for autonomic and energy-efficient resource management. Eugen received his Bachelor and Master of Science degrees in Computer Science from the Heinrich Heine University Duesseldorf (HHU), Germany. He was a student research assistant in the HHU operating systems group, interned in the Myriads INRIA project team and was a summer intern at the LBNL.
8:30 - 9:00 Common Q&A