Lambdoku – AWS Lambda with Heroku-like Experience(github.com)
github.com
Lambdoku – AWS Lambda with Heroku-like Experience
https://github.com/kubek2k/lambdoku
15 comments
Related: Apex https://www.github.com/apex/apex
I'm wondering if it would be possible to create something like Lambda, but optimized for multiplayer servers with game loops? What if there was a platform that let you write pure functions to create player accounts, spawn game instances, and handle the transformation of player and game state from tick to tick? (You'd have to choose a particular architecture for synchronization with the client, of course. Also, before someone makes the mistake of assuming I'm as stupid as they are -- I'm not proposing that a function be transmitted, compiled, then evaluated at each tick over the network! Functions would be installed into virtualized servers.)
Clojure, Javascript, and Golang would be good languages for this. The environment possibly would let you inject and replace functions live, lending to a Light Table like experience, but while developing a multiplayer game.
Clojure, Javascript, and Golang would be good languages for this. The environment possibly would let you inject and replace functions live, lending to a Light Table like experience, but while developing a multiplayer game.
Your basic description sounds like LambdaMOO (written in 1990, the name is a coincidence).
The LambdaMOO server has a main event loop which handles user events, then schedules tasks which execute functions ('verbs' in LambdaMOO parlance) to modify the game state. Verbs are not pure per se but can be thought of as a transaction which executes in an atomic, consistent, and isolated manner. Verbs are attached to objects in the game, and can be modified in-game on the fly by players with their programmer bit set. (Programmers can only modify verbs on objects they own.)
The LambdaMOO language is a lot like JavaScript with a Luaesque syntax. It has prototypical inheritance, which meshes well with its concept of objects as physical things. For example, to get a new instance of a duck one would create a Generic Duck named "My Duck" ('Generic' being the conventional prefix for an object specifically intended to be used as a prototype.)
There was a time in the late 1990s when it regularly supported 300 active users at once (with a certan amount of lag); on modern hardware it could probably handle significantly more than that. If you want a modern scalable system then LambdaMOO itself is probably not for you; the server is single-threaded (in fact the whole thing is remarkably similar to NodeJS with async/await support) and state is saved in-memory and checkpointed periodically (hence the Durability missing from the above description of verbs).
However, the basic design (prototypal inheritance, in-database verbs, ACI[D] tasks, etc) is extremely well proven, and would lend itself well to being copied into a modern distributed system like AWS Lambda. As long as you had an ACID datastore to back it up, you could scale the task execution as far as you wanted, and with a bit of clever work you could probably come up with a partitioning scheme to make the database scale as well.
In fact, I've partially implemented this already; what's stopped me has primarily been that I'm more interested in the technical aspects and have no idea what I would actually use it for once I built it. If you have a use case for this sort of thing I'd love to hear about it.
The LambdaMOO server has a main event loop which handles user events, then schedules tasks which execute functions ('verbs' in LambdaMOO parlance) to modify the game state. Verbs are not pure per se but can be thought of as a transaction which executes in an atomic, consistent, and isolated manner. Verbs are attached to objects in the game, and can be modified in-game on the fly by players with their programmer bit set. (Programmers can only modify verbs on objects they own.)
The LambdaMOO language is a lot like JavaScript with a Luaesque syntax. It has prototypical inheritance, which meshes well with its concept of objects as physical things. For example, to get a new instance of a duck one would create a Generic Duck named "My Duck" ('Generic' being the conventional prefix for an object specifically intended to be used as a prototype.)
There was a time in the late 1990s when it regularly supported 300 active users at once (with a certan amount of lag); on modern hardware it could probably handle significantly more than that. If you want a modern scalable system then LambdaMOO itself is probably not for you; the server is single-threaded (in fact the whole thing is remarkably similar to NodeJS with async/await support) and state is saved in-memory and checkpointed periodically (hence the Durability missing from the above description of verbs).
However, the basic design (prototypal inheritance, in-database verbs, ACI[D] tasks, etc) is extremely well proven, and would lend itself well to being copied into a modern distributed system like AWS Lambda. As long as you had an ACID datastore to back it up, you could scale the task execution as far as you wanted, and with a bit of clever work you could probably come up with a partitioning scheme to make the database scale as well.
In fact, I've partially implemented this already; what's stopped me has primarily been that I'm more interested in the technical aspects and have no idea what I would actually use it for once I built it. If you have a use case for this sort of thing I'd love to hear about it.
There was a time in the late 1990s when it regularly supported 300 active users at once (with a certan amount of lag)
I had a pseudo-roguelike server first implemented in Clojure, then in Go that updated 12 frames a second. It could support 250 simultaneous users. It even had Conway's life as an area attack. I had it posted to Show HN.
However, the basic design (prototypal inheritance, in-database verbs, ACI[D] tasks, etc) is extremely well proven, and would lend itself well to being copied into a modern distributed system like AWS Lambda.
All that's needed is pure functions, soft realtime, and good tooling and APIs. ACID takes some doing, in terms of implementation and will also make scaling complicated, as you mention above. Inheritance? Nice but not necessary. Just let developers inject functions into a game loop, then tell them their load and when their simulation tick rate starts to drop.
If you have a use case for this sort of thing I'd love to hear about it.
Yes. Something like that could be packaged into MMOGAAS -- Massively Multiplayer Online As A Service.
I had a pseudo-roguelike server first implemented in Clojure, then in Go that updated 12 frames a second. It could support 250 simultaneous users. It even had Conway's life as an area attack. I had it posted to Show HN.
However, the basic design (prototypal inheritance, in-database verbs, ACI[D] tasks, etc) is extremely well proven, and would lend itself well to being copied into a modern distributed system like AWS Lambda.
All that's needed is pure functions, soft realtime, and good tooling and APIs. ACID takes some doing, in terms of implementation and will also make scaling complicated, as you mention above. Inheritance? Nice but not necessary. Just let developers inject functions into a game loop, then tell them their load and when their simulation tick rate starts to drop.
If you have a use case for this sort of thing I'd love to hear about it.
Yes. Something like that could be packaged into MMOGAAS -- Massively Multiplayer Online As A Service.
> Something like that could be packaged as Massively Multiplayer Online As A Service
That already exists, see Photon Engine's multiple PaaS options https://www.photonengine.com
Different ones have different APIs tuned for different types of gameplay and or Unity3D integration (by mimicking Unity's original networking programming API).
I've implemented them for an MMO I freelanced for a while back and it was pretty painless.
That already exists, see Photon Engine's multiple PaaS options https://www.photonengine.com
Different ones have different APIs tuned for different types of gameplay and or Unity3D integration (by mimicking Unity's original networking programming API).
I've implemented them for an MMO I freelanced for a while back and it was pretty painless.
So Photon is operating a farm of servers that are running game loops, which can have pure functions injected into them? (Also, the per message pricing!)
No, Photon is doing "Massively Multiplayer Online As A Service" (did you even read their page?).
And actually they are running game loops, but you can't inject code into them unless you run their software on-premise, or use a separate system I recall them mentioning that offered server-side scripts that got injected in their loops.
And cost per message make perfect sense because of the networking model, which is fully message based. Paying per message is paying for bandwidth + cpu usage as a single combined rate.
And actually they are running game loops, but you can't inject code into them unless you run their software on-premise, or use a separate system I recall them mentioning that offered server-side scripts that got injected in their loops.
And cost per message make perfect sense because of the networking model, which is fully message based. Paying per message is paying for bandwidth + cpu usage as a single combined rate.
No, Photon is doing "Massively Multiplayer Online As A Service" (did you even read their page?).
And actually they are running game loops, but you can't inject code into them unless you run their software on-premise
So you are seemingly contradicting yourself. 1st you said that "they are doing that." Now you are saying they are also doing "Massively Multiplayer Online As A Service" -- but in a quite different way than what I specified. So yes, I did read what you posted. Apparently you only carelessly read mine.
And actually they are running game loops, but you can't inject code into them unless you run their software on-premise
So you are seemingly contradicting yourself. 1st you said that "they are doing that." Now you are saying they are also doing "Massively Multiplayer Online As A Service" -- but in a quite different way than what I specified. So yes, I did read what you posted. Apparently you only carelessly read mine.
I think you need to read my post again then. I'm not contradicting myself in any way.
You acted like no one offers a PaaS for MMOs (and gave it a ridiculous name).
I pointed out just one of many PaaS options for MMOs (or ...MMOAAAS as you insist on calling it.).
In fact, when working for that MMO we did evaluate options the did let you use their game loop with your cloud functions being called on triggers, but you'd have to look that up yourself, that was a few years ago (even Photon has added new services since I did that research).
If you think you have an approach that can enter the space, by all means go ahead, but PaaSes for MMOs have existed for ages.
You acted like no one offers a PaaS for MMOs (and gave it a ridiculous name).
I pointed out just one of many PaaS options for MMOs (or ...MMOAAAS as you insist on calling it.).
In fact, when working for that MMO we did evaluate options the did let you use their game loop with your cloud functions being called on triggers, but you'd have to look that up yourself, that was a few years ago (even Photon has added new services since I did that research).
If you think you have an approach that can enter the space, by all means go ahead, but PaaSes for MMOs have existed for ages.
I didn't log in for a while and my account was deleted circa 1999, I was pissed, since I'd had it since about 93, although there was little I could do about it. What a great mud though. :)
I only discovered it around 2012, so I never got to see it in its heyday. I had great fun programming different things, though I haven't found the time in a couple of years. I still log in occasionally just to wander around and marvel at things--there's all sorts of weird stuff lying around, though I can never seem to find something when I go looking for it. I don't think they reap accounts anymore, since there's no longer a massive influx of players putting pressure on the database.
If I was building a MUD or MMO from scratch today I'd try with Elixir (the language built on Erlang). Functional programming with hot code replacement, implementing the actor model for distributed, fault-tolerant, soft-real-time massively concurrent applications, does that sound similar to what you're looking for?
Growing that into a FaaS product would be hard though; Erlang processes all share a trust domain, from the infosec point of view, so containerizing each function/process for invocation lambda-style on otherwise shared infrastructure may prove difficult.
Growing that into a FaaS product would be hard though; Erlang processes all share a trust domain, from the infosec point of view, so containerizing each function/process for invocation lambda-style on otherwise shared infrastructure may prove difficult.
implementing the actor model
Each of my server processes in Go looks a little like an Erlang actor. There's a switch statement where a queue of input messages is processed, then each instance is run in turn. Instances also have a similar structure and maintain a "space" using an R-Tree. When it comes to efficiency, game loops are still a great way to do concurrency. Game loops per-instance allow you to sidestep many pitfalls of multi processor concurrency, much as per-process GC does with Erlang.
Each of my server processes in Go looks a little like an Erlang actor. There's a switch statement where a queue of input messages is processed, then each instance is run in turn. Instances also have a similar structure and maintain a "space" using an R-Tree. When it comes to efficiency, game loops are still a great way to do concurrency. Game loops per-instance allow you to sidestep many pitfalls of multi processor concurrency, much as per-process GC does with Erlang.
You could, but the latency issues (network hops) would make the solution untenable. Function as a Service platforms are ideal for solutions where scalability is a huge concern, latency not so much. These functions are all running inside of containers that have been optimized to be "warmed up" extremely quickly, but that warm-up takes time, and the more robust the solution, the longer the warm-up.
You'd probably want architecture extremely specialized to a game loop, and probably your game loop specifically, if you wanted low-latency, high-availability FaaS for this purpose.
You'd probably want architecture extremely specialized to a game loop, and probably your game loop specifically, if you wanted low-latency, high-availability FaaS for this purpose.
You could, but the latency issues (network hops) would make the solution untenable.
...ok... Did you read my comment?
These functions are all running inside of containers that have been optimized to be "warmed up" extremely quickly, but that warm-up takes time, and the more robust the solution, the longer the warm-up.
Yeah, of course that's not going to work.
I've basically built a specialized "container" system, but instead of using kernel virtualization, it's actually a specific Go "object" residing in an already running server process. These "containers" (really instances) start running in an environment where we already have a virtual server with a process with a server/game loop up and running. I have a cluster of processes that does this, and idempotently spawns an instance of star system in a multiplayer game consisting of 2^87 star systems.
www.emergencevector.com
Such a system can still be built on Docker style containers, it's just that those Docker style containers represent an entire sub-cluster of many server processes that can each host many "instance" containers which are where the functions that transform from tick n to tick n+1 reside.
You'd probably want architecture extremely specialized to a game loop
That is what I said in my comment.
and probably your game loop specifically
I'm saying that the game loop could be customized/built up by a developer by injecting functions into such a system. (Or, possibly, by injecting such functions into a dev version running locally, then exporting them to a server in the cloud.)
...ok... Did you read my comment?
These functions are all running inside of containers that have been optimized to be "warmed up" extremely quickly, but that warm-up takes time, and the more robust the solution, the longer the warm-up.
Yeah, of course that's not going to work.
I've basically built a specialized "container" system, but instead of using kernel virtualization, it's actually a specific Go "object" residing in an already running server process. These "containers" (really instances) start running in an environment where we already have a virtual server with a process with a server/game loop up and running. I have a cluster of processes that does this, and idempotently spawns an instance of star system in a multiplayer game consisting of 2^87 star systems.
www.emergencevector.com
Such a system can still be built on Docker style containers, it's just that those Docker style containers represent an entire sub-cluster of many server processes that can each host many "instance" containers which are where the functions that transform from tick n to tick n+1 reside.
You'd probably want architecture extremely specialized to a game loop
That is what I said in my comment.
and probably your game loop specifically
I'm saying that the game loop could be customized/built up by a developer by injecting functions into such a system. (Or, possibly, by injecting such functions into a dev version running locally, then exporting them to a server in the cloud.)
This is great. I was going to create something similar to manage lambda functions across environments. Awesome to see you're already using the newly released environment variables for lambdas.
thanks! - please give it a try and let me know in the issues if something doesn't work as expected.
Another nice project that helps to ease lambda workflows is https://github.com/garnaat/kappa
There is similar project called Zappa.
pip install zappa
pip install zappa
Zappa actually uses some of Kappa under the hood, but with many many more features. Learn more here: https://github.com/Miserlou/Zappa
Is Zappa a fork of Kappa? I have not used Kappa at all but it looks like it a whole lot.
For a second there, I thought someone implemented a push-to-deploy experience for Heroku (with some of the other Heroku niceties). I actually wrote a blog post about that this week: http://josediazgonzalez.com/2016/11/21/push-to-deploy-lambda...
really good read. thx! :bow:
I didn't plan to do the push-to-deploy thing, mostly because that would totally complicate things. The rationale was more that we would use the CI that would automatically build and deploy things to DEV env that would be then promoted around.
I didn't plan to do the push-to-deploy thing, mostly because that would totally complicate things. The rationale was more that we would use the CI that would automatically build and deploy things to DEV env that would be then promoted around.
You could easily prefix your special environment variable with 'lambdoku_' so you don't have to fear that people could use it..
Has anyone characterised server less architectures? I'm interested in testing / exploring the limits with some predictive modelling work and implementing (mathematical) functions as FAAS? No doubt, it 'depends' (e.g. nature of data and its locality or otherwise) but interested to hear if anyone is looking at very fine-granularity server less systems?
How is this any different from Serverless? (not trolling, I could not find any differences in the README)
I don't mind being trolled :)
Serverless is way much more advanced machinery than this tool. My point was to create a tool, not a whole ecosystem around AWS Lambda.
When it comes to serverless - my main concern (as it was with Lambda without env variables), is with storing configuration data along the code in the repo. Additionally the reason for creating was mostly pipelines.
Serverless is way much more advanced machinery than this tool. My point was to create a tool, not a whole ecosystem around AWS Lambda.
When it comes to serverless - my main concern (as it was with Lambda without env variables), is with storing configuration data along the code in the repo. Additionally the reason for creating was mostly pipelines.
Very cool project, but it has me thinking what/if it would be possible to build out more of the infrastructure management that Heroku has to make it even easier (domain setup, addon store, etc.)
Looks cool! I like the release message stuff, I wanted to add similar to apex(1) at some point, not having a clue what the releases actually are in Lambda makes rollbacks a little tricky.
thats a wonderful improvement of apex, thats for sure :)
The guide on the README doesn't tell me how am I supposed to push and update a function.
Also, does it help with dependencies or something like that? Does it manage packaging files?
Also, does it help with dependencies or something like that? Does it manage packaging files?
the initial idea was that CI deploys lambda, and you promote it from stage to stage. I might consider upload sometime.
From what you write I think apex would be better suited for you
From what you write I think apex would be better suited for you
You acted like no one offers a PaaS for MMOs (and gave it a ridiculous name).
I was talking something specifically like Lambda, but where pure functions could be injected into an already running game loop. The name was intentionally ridiculous. Also, you suggested that my proposal would suffer from the latency of lambda requests going over the network, which shows you didn't read my original comment.
with your cloud functions being called on triggers
Yeah, I pretty much covered that this was not what I was proposing, when I wrote "(You'd have to choose a particular architecture for synchronization with the client, of course. Also, before someone makes the mistake of assuming I'm as stupid as they are -- I'm not proposing that a function be transmitted, compiled, then evaluated at each tick over the network! Functions would be installed into virtualized servers.)" As far as I knew (which might be out of date) Photon was developing something that would be suitable for a persistent world MMORPG, but not quite ready with it. What I'm proposing would be suitable for an MMORPG, not just MMO game sessions.
I was talking something specifically like Lambda, but where pure functions could be injected into an already running game loop. The name was intentionally ridiculous. Also, you suggested that my proposal would suffer from the latency of lambda requests going over the network, which shows you didn't read my original comment.
with your cloud functions being called on triggers
Yeah, I pretty much covered that this was not what I was proposing, when I wrote "(You'd have to choose a particular architecture for synchronization with the client, of course. Also, before someone makes the mistake of assuming I'm as stupid as they are -- I'm not proposing that a function be transmitted, compiled, then evaluated at each tick over the network! Functions would be installed into virtualized servers.)" As far as I knew (which might be out of date) Photon was developing something that would be suitable for a persistent world MMORPG, but not quite ready with it. What I'm proposing would be suitable for an MMORPG, not just MMO game sessions.
We detached this subthread from https://news.ycombinator.com/item?id=13062880 and marked it off-topic.
BoorishBears(2)
what I'd really like is ability to take a swagger spec and click a button to deploy it on AWS Lambda.
This is great for internal facing REST Api. I don't know about scaling but hypothetically serverless means it should handle public requests fine.
This is great for internal facing REST Api. I don't know about scaling but hypothetically serverless means it should handle public requests fine.
I guess you mean the API-Gates + Lambda. Not sure - I think thats something that should be handled by some separate tool (APIGatesgger? ;) - lets hope its not another dirty word)
that's right swagger -> api gateway triggering lambda functions.
I was using Chalice but seems like Zappas is another one a HNer pointed out. I couldn't get Chalice to work and sort of gave up on the serverless-api-gatway-lambda idea.
I was using Chalice but seems like Zappas is another one a HNer pointed out. I couldn't get Chalice to work and sort of gave up on the serverless-api-gatway-lambda idea.
[1] https://github.com/poly/stdlib
[2] https://www.portal.reinvent.awsevents.com/connect/sessionDet...