The bot has actually never seen Destroyer's big drones during training even once, so I found it somewhat surprising that it even works as well as it does!
Completely agree that adding something like the "League" used by AlphaStar would be one of the top priorities if you wanted to push this project further. I don't think CodeCraft is sufficiently complex to really allow for several very distinct strategies in the same way as StarCraft II, but I would still expect training against a larger pool of more diverse agents to increase robustness quite a bit.
Thank you for the kind words! I am also quite excited about the new points in game design space that RL will unlock and am planning write another blogpost on that topic.
CodeCraft is a programming game which you can "play" by writing a Scala/Java program that controls the game units. It's not actively developed anymore but still functional: http://codecraftgame.org/
The constraining factor in this case is CPU throughput. The CPU cannot execute instructions fast enough to keep up with the data being read in from memory.
Of course you are right that this is not the whole picture, it is possible to be bottlenecked on neither memory bandwidth nor CPU throughput.
With respect to memory, two additional considerations are read amplification and latency.
Read amplification:
Read amplification basically means that you are reading data but not actually using it.
Suppose we have a table stored in row format with four columns C1, C2, C3, C4 which store 8 byte integers.
Row format means that the values for the columns are interleaved, so the memory layout would look like this, where cn refers to some value for column Cn:
One property of conventional memory is that we can only read memory in blocks of 64bytes (the size of a cache line).
So if we have a loop that sums of all of the values in C1, we will actually also load all of the values for columns C2, C3, C4.
This means we have a read amplification of factor 4, and our usable bandwidth will be a quarter of the theoretical maximum.
Latency:
Suppose we also don't have any read amplification and make use of all values for each loaded cache line.
Then after we finish processing a cache line, and load the next cache line, we might have to wait a long time (100s of cycles) before the new cache line actually arrives!
So then we would actually be stalled because of memory latency during some parts of program execution.
One of the big advantages of using columnar layout (which stores all values for a column sequentially) is that it all but eliminates read amplification.
Similarly, by accessing data sequentially we make it possible for the CPU to predict what data will be accessed in the future, and have it loaded into the cache before we even request it.
Of course there's a number of reasons why things might not work out quite so perfectly in practice, and those may why performance in this benchmark starts to degrade before the (usable) memory bandwidth reaches the theoretical maximum memory bandwidth. But as you can see it still possible to get very close, so it's safe to assume that queries which read much less data than the maximum bandwidth are constrained on CPU.
CodeCombat is aimed at people with no, or very little, programming experience.
My game will be targeted towards people who already have at least basic programming knowledge, and are looking for a less restrictive environment.
In that respect, it will resemble CodeCombat's zero sum mode[1].
I am hoping that the focus on slightly more experienced programmers will allow me to create a game with more strategic depth.
You raise a really good point about thinking through the game mechanics before diving into the coding.
As it happens, I have actually spent a lot of time on this and even implemented something similar some years ago.
I probably could made that clearer in the blogpost, but I mainly wanted to give a concise overview my goals and plans for this project.
My next updates should have a bit more substance :)
I have never used Clojure myself, so I don't know too much about it.
Can it easily call into Java libraries?
In that case, Clojure should work as well.