Depends on the way you play can be a time sink, or session-like game. It is extremely deep and complex to learn from scratch though.
I've made some of the best friends playing it when I had time, friendship formed out of high stakes in this game (you regularly lose hours of grind or real money if you pay for the game - in seconds) and respect you have for each other skill.
Outstanding work! I've participated in the challenge, but didn't get far. One of the questions I had at the time was - if I'm going to use ML to detect ink, could it invent hallucinated letters, or even parts of text, and how to prevent that?
How much time do you lose when doing things like "verify plan with a second clean agent" instead of just reading and fixing it yourself in 5 min? How much understanding do you lose? How do you manage to treat it "as an engineer" where it's clearly not there yet? How much time do you lose when it makes almost the same mistake, invents stuff or tries to gaslight you over and over? What about blood pressure?
I did vibe code jam 59 entry with friends, the spirit of the rules there's a lot more lax. We didn't even get to top 100, but that's mostly due to gamedesign errors, not tech. This is the first entry in years which was vibecoded 100%, and I have very mixed feelings about it. It's no doubt anymore - 1.5x-2x speedup, which makes not using it (if allowed) a complete no-go. But psychologically it's tough losing control, and changing workflow to managerial one substantially, it diminishes the craft.
But how is that less work and allows you to do that in Disneyland with your kids? For me, personally, there is little difference between "speccing out everything in excruciating detail in spec docs" and "writing actual implementation in high-level code". Speccing in detail requires deep thought, whiteboard, experimentation etc. All of this cannot be done in Disneyland, and no AI can do this at good level (that's why you "spec out everything in detail", create "beads" and so on?)
I have a feeling this is somehow different. The tool is broad enough, that I don't have to think myself in a wide variety of tasks, not just one. Which hurts my intelligence way more.
Most critical piece of information I couldn’t find is - how many shot was this?
Could it understand the solution is correct by itself (one-shot)? Or did it have just great math intuition and knowledge? How the solutions were validated if it was 10-100 shot?
This is really interesting, because I do gamejams from time to time - and I try every time to make it work, but I'm still quite a lot faster doing stuff myself.
This is visible under extreme time pressure of producing a working game in 72 hours (our team scores consistenly top 100 in Ludum Dare which is a somewhat high standard).
We use a popular Unity game engine all LLMs have wealth of experience (as in game development in general), but the output is 80% so strangely "almost correct but not usable" that I cannot take the luxury of letting it figure it out, and use it as fancy autocomplete. And I also still check docs and Stackoverflow-style forums a lot, because of stuff it plainly mades up.
One of the reasons is maybe our game mechanics often is a bit off the beaten road, though the last game we made was literally a platformer with rope physics (LLM could not produce a good idea how to make stable and simple rope physics under our constraints codeable in 3 hours time).
But they definitely could and were [0]. You just employ multiple, and cross check - with the ability of every single one to also double check and correct errors.
LLMs cannot double check, and multiples won't really help (I suspect ultimately for the same reason - exponential multiplication of errors [1])
Not possible, because can't be guardrailed with 100% accuracy. You'll ask it something outside of the Warcraft world (e.g. US politics), and it'll happily oblige. I imagine NPCs will generate really weird immersion breaking stuff even if you cannot freeform interact with them anyway.
Used to live near Japan, in a 5-story concrete building. We had so many earthquakes, the things were getting thrown off the shelves multiple times a year. There is nowhere to leave for a week. And there is a very slim chance anything would collapse, because the buildings are specifically built to withstand up to around 9 richter.
If we can create a human-level intelligence in the computer - it would already be superintelligence. No human on Earth is capable of reading and remembering Internet scale corpus of data, or doing math at GHz speeds, etc.
Start with game jams. When you're constrained to make a complete game in 48/72 hours, suddenly you start to understand what's important and what's not. Granted, I haven't done any doors in my jam games yet =)
There are far more games that were made with Unity, not Unreal, I would even say an order of magnitude more. So I don't really get your point? Unity is vastly easier to jump on, so a lot more people actually make games and test things (not knowing the quirks of the engine inside out, like a game engine developer would).
He actually mentions the tool [0] that has all of this, in the end. It records your screen and compiles a demo that runs in any browser by the looks of it.
Yeah I'm aware of the criticism of that tweet, it may not completely make sense mathematically, but I just liked it because it's how I feel about GPT4.
It "diverges" while my human mind seemingly is different in some way - I can keep going at the math problem forever (for much longer?) and I won't hallucinate incorrect proofs (at least very unlikely, and I can keep re-checking them).
Of course this all in the area of feelings and faith - we just don't know much about cognition I guess.
It definitely can "understand" things is some way, however I'm pretty sure ReAct or similar will give it just a nudge forward and the underlying problem of it hallucinating and not being "lucid enough" is not so easily solved.
In the original ReAct paper it falls apart almost immediately in ALFWorld (this is a classical test for AI systems - to be able to reason logically - and it still isn't generally solvable due to combinatorial explosion).
For now it requires human correction looped or not, or else it "diverges" (I like Yann Lecun explanation [0]).
In my own experiments (I haven't played with LangChain or ReAct yet) it diverges irrecoverably pretty quickly. I was trying to explain to it the elementary combinators theory, in the style of Raymond Smullyan and his birds [1] and it can't even prove the first theorem (despite being familiar with the book). A human can prove it knowing almost nothing about math whatsoever, maybe it will take a couple of days thinking, but the correct proof is not that hard - just two steps.
I've made some of the best friends playing it when I had time, friendship formed out of high stakes in this game (you regularly lose hours of grind or real money if you pay for the game - in seconds) and respect you have for each other skill.