World Models (the long version)(adgefficiency.com)
adgefficiency.com
World Models (the long version)
https://adgefficiency.com/world-models/
11 comments
For a hobbyist, you definitely don't want to use the cloud, buy yourself a good GPU (I got a great deal 2nd hand in eBay) and you'll save a lot of cash.
What you need the cloud for is parallelism. If you want to do any sort of hyperparameter tuning doing it on a single computer is going to leave you waiting for days.
What you need the cloud for is parallelism. If you want to do any sort of hyperparameter tuning doing it on a single computer is going to leave you waiting for days.
I can't speak to the author's workflow but I also do ML research and I often need heavy bursts of computation with long stretches of nothing in between. So I might spend a few hundred bucks in a couple of days by using multiple very powerful instances at the same time. Even if the cost ends up being the same as a workstation over a year, when it matters the cloud gives me results faster than a workstation would and this enables fast iteration over ideas.
Bursting is definitely a point. It wasn't clear to me from the data at first glance whether this was used here.
I did appreciate them detailing these costs.
World Models is a good paper, but is definitely not the top three most significant RL papers.
To be fair, the post does not say that. It says that it's "one of the papers significant to the author" - a subjective ranking.
[deleted]
I loved this paper. It's a good example of a VAE+RNN style model learning for MBRL. It's well exposed. Sure the results aren't the best but it's a well worth read.
Which are the top three mist significant RL papers?
Other than the AlphaZero papers and DQN, maybe TRPO? (assuming we're talking about deep RL)
Total spend was $3.648 over 11 months. For that amount you can tailor a pretty powerful workstation these days, saving you quite a bit of AWS headaches and overheads (and yes, potentially trading those for others, I know).
Still, this seems like a decent candidate for omitting the cloud as at first sight I do not detect any asks that scream out cloud to me.
Thoughts?