Programming principles from the early days of id Software(gamasutra.com)
gamasutra.com
Programming principles from the early days of id Software
http://www.gamasutra.com/view/news/279357/Programming_principles_from_the_early_days_of_id_Software.php
28 comments
Maintain flow at all costs.
To anyone that hasn't checked out the source code for Doom 3, it's some seriously beautiful C++.
https://github.com/id-Software/DOOM-3-BFG
https://github.com/id-Software/DOOM-3-BFG
I was just looking at where the cheats are tracked and I suddenly thought, what did IDDQD and IDKFA stand for?
https://github.com/id-Software/DOOM-3-BFG/blob/master/doomcl...
https://github.com/id-Software/DOOM-3-BFG/blob/master/doomcl...
Dave Taylor (ddt) and I and another friend had some Q's on our college transcripts (from dropping classes after the tuition refund deadline). We joked that we should have a fraternity for losers like us. We called it delta q delta. Dave named the iddqd cheat after that joke. This is also why my vanity domain name is dqd.com. Dave did end up graduating, but I and the other friend didn't.
ID-Kick-Fucking-Ass, ID-Delta-Quit-Delta was a made up frat of college dropouts
Keys Firearms Armor, IIRC.
Sometimes reported as Keys & Full Ammo, but this neglects all firearms granted to the user.
There's a subhed that refers to tooling:
> Great tools help make great games. Spend as much time on tools as possible.
But the text under it is more related to testing of code and bug-squashing. Anyone have any links/references to id's process and prioritization of building tools?
> Great tools help make great games. Spend as much time on tools as possible.
But the text under it is more related to testing of code and bug-squashing. Anyone have any links/references to id's process and prioritization of building tools?
Tools drive everything in games. Your normal AAA team structure is something along the lines of 1:10 dev:art/design. Every improvement you make in tools has a 10x factor in the people using it.
Additionally everything about game development is about iteration. No one really knows what's fun(it's kinda one of those things where "I know what it is when I see it"). So the core of gamedev is iterating and trying as many crazy ideas as you can until something sticks. The right tools that enable that rapid iteration are critical. The faster you can make changes the more shots you get at capturing that lightning in the bottle(which is why 10% of games make 90% of the revenue).
Additionally everything about game development is about iteration. No one really knows what's fun(it's kinda one of those things where "I know what it is when I see it"). So the core of gamedev is iterating and trying as many crazy ideas as you can until something sticks. The right tools that enable that rapid iteration are critical. The faster you can make changes the more shots you get at capturing that lightning in the bottle(which is why 10% of games make 90% of the revenue).
I don't doubt that. I am particularly interested in how that mentality compares to the priorities of devs in other fields. For example, web development. I'm thinking that games development might be substantially more bespoke than webdev, necessitating the need to spend time on developing bespoke tooling?
There's really just 10-50x more "content" involved in gamedev then say web development. You might have 3-4 designers on a front end where a 60M budget game will have 6-8 animators alone, not even considering environment art, character art, level design, gameplay mechanic designers, etc.
Now if you get into indie things are slightly different there but you still see just about everyone using an off-the-shelf engine because no one has the time to invest the man-years needed for proper tooling.
Now if you get into indie things are slightly different there but you still see just about everyone using an off-the-shelf engine because no one has the time to invest the man-years needed for proper tooling.
"proper tooling" is an interseting choice of words.
Are you saying that the off-the-shelf engines are somehow inadiquate, a necessary evil compared to hand rolled tools?
Are you saying that the off-the-shelf engines are somehow inadiquate, a necessary evil compared to hand rolled tools?
Not really, just each game usually has unique needs and it's hard to build tools that address every possible game style and type. Usually when you use UE3/UE4/Unity you end up making significant modifications/extensions to meet those needs.
> Tools drive everything in games.
Isn't that true for everything in development?
Isn't that true for everything in development?
Sure, it's turtles all the way down in some aspects. However you usually don't have your front-end web designer working a custom toolset to author images for webpages.
There's no common standard in games(hell even 3D interchange formats are a mess) so there's quite a bit of custom tooling done per-title based on gameplay needs in addition to the standard content pipeline(export->import->annotate/tune->bake->console) that you see.
There's no common standard in games(hell even 3D interchange formats are a mess) so there's quite a bit of custom tooling done per-title based on gameplay needs in addition to the standard content pipeline(export->import->annotate/tune->bake->console) that you see.
I noticed that too.
The tools principle dovetails with "Use a superior development system". Infocom's game building tool (the ZIL compiler) was originally a PDP-10 program. LucasArts developed many games using the SCUMM suite of tools, which they developed on Unix-based Sun workstations. (The SCUMM compiler used YACC!)
Even today, you can use a compute cluster to render videos, "bake" assets, distribute compilation, run tests, run machine learning, ...
The tools principle dovetails with "Use a superior development system". Infocom's game building tool (the ZIL compiler) was originally a PDP-10 program. LucasArts developed many games using the SCUMM suite of tools, which they developed on Unix-based Sun workstations. (The SCUMM compiler used YACC!)
Even today, you can use a compute cluster to render videos, "bake" assets, distribute compilation, run tests, run machine learning, ...
[deleted]
> Write your code for this game only, not for a future game. You’re going to be writing new code later because you’ll be smarter.
This alone is worth the read for me. I wish I could shadow this team.
This alone is worth the read for me. I wish I could shadow this team.
I'd recommend checking out this post[1] by Wil Shipley (a longtime Cocoa dev), which includes a story about how Carmack doesn't re-use code. The whole blog post is a fun read.
[1] http://blog.wilshipley.com/2013/12/my-doom-20th-anniversary-...
[1] http://blog.wilshipley.com/2013/12/my-doom-20th-anniversary-...
Great link. Thanks for sharing.
Bear in mind the next thing the article says is
>(And of course there was no Unity or Unreal Engine to license.)
But advice like "don't reuse" and "use an external framework", actually have a common root because the difficult thing in programming is not banging out code, not even when you have your QA in place.
The really hard thing is getting knowledge: figuring out the right thing to do and how to do it.
Once you have figured it out in one app, you can do it again only better in the next app you write. You are re-using knowledge that is stored in your head.
But when someone writes an engine, they are also storing knowledge, but now inside code. Again, knowledge is being re-used.
>(And of course there was no Unity or Unreal Engine to license.)
But advice like "don't reuse" and "use an external framework", actually have a common root because the difficult thing in programming is not banging out code, not even when you have your QA in place.
The really hard thing is getting knowledge: figuring out the right thing to do and how to do it.
Once you have figured it out in one app, you can do it again only better in the next app you write. You are re-using knowledge that is stored in your head.
But when someone writes an engine, they are also storing knowledge, but now inside code. Again, knowledge is being re-used.
Choose the right platform for your goals. While id was banging out bare-metal code on a relatively unchanging PC platform, my old partner and I spent our time battling API deprecation, a change of processor, endianness issues, performance issues, just on and on and on, a whole litany of insults on the Macintosh platform.
We were attempting to "just have a successful shareware game business on the Macintosh" but after 20 years of this self-flagellation I bailed. We each now work at rather successful studio/agency development firms and I've probably contributed to more meaningful projects in the last year and a half than in all the years previously.
When you choose the wrong platform, you will not only be running against the wind, but when the wind dies down you will be so tired that you miss the new opportunities. We should have been positioned to capitalize on the iOS store back in the mid-2000s with our substantial codebase, but due to burnout I witnessed others openly rake in $100,000 paydays on projects similar to our own that they only spent a couple of months making. That could have been us, but I can't blame the competition or Apple because in hindsight it was my own sheer inability to stop beating a dead horse that put me in the universe that didn't work out. Really you should always be working in a sustainable way that's not too easy and not too hard so that you can see the forest for the trees, and after a while those small successes often accumulate into something unbelievable.
We were attempting to "just have a successful shareware game business on the Macintosh" but after 20 years of this self-flagellation I bailed. We each now work at rather successful studio/agency development firms and I've probably contributed to more meaningful projects in the last year and a half than in all the years previously.
When you choose the wrong platform, you will not only be running against the wind, but when the wind dies down you will be so tired that you miss the new opportunities. We should have been positioned to capitalize on the iOS store back in the mid-2000s with our substantial codebase, but due to burnout I witnessed others openly rake in $100,000 paydays on projects similar to our own that they only spent a couple of months making. That could have been us, but I can't blame the competition or Apple because in hindsight it was my own sheer inability to stop beating a dead horse that put me in the universe that didn't work out. Really you should always be working in a sustainable way that's not too easy and not too hard so that you can see the forest for the trees, and after a while those small successes often accumulate into something unbelievable.
How does one choose the right platform ahead of time? It seems that that would be something you could only judge in hindsight.
For instance, did you know that, over the course of your endeavour, that the powers that be were going to deal you a bunch of API deprecation? Did Id Software know that their respective platform providers, or the market in general, were going to keep their platform a relatively stable development target?
I am thinking about developing a game in my spare time. I don't have any grand plans with it. I thought to develop it on Android (though I hear that's a mess, getting apps to run on various phones and OSes), but I want to try to keep open the possibility that it could be ported to other platforms (iOS, console, etc) relatively painlessly.
Being a web developer, how could I glean enough information in areas I'm really unfamiliar with to judge which is the right platform?
If I decide on Android, for whatever reason, and I become un-enamored with all of its warts, how do I know that the grass truly is greener on the other side?
For instance, did you know that, over the course of your endeavour, that the powers that be were going to deal you a bunch of API deprecation? Did Id Software know that their respective platform providers, or the market in general, were going to keep their platform a relatively stable development target?
I am thinking about developing a game in my spare time. I don't have any grand plans with it. I thought to develop it on Android (though I hear that's a mess, getting apps to run on various phones and OSes), but I want to try to keep open the possibility that it could be ported to other platforms (iOS, console, etc) relatively painlessly.
Being a web developer, how could I glean enough information in areas I'm really unfamiliar with to judge which is the right platform?
If I decide on Android, for whatever reason, and I become un-enamored with all of its warts, how do I know that the grass truly is greener on the other side?
It pains me to say it, but the big companies (even nvidia) are incentivized towards proprietary solutions, so your best bet would probably be to go with something like Unity.
My big mistake was to think that I could manage a game engine. This was in the late 90s and early 2000s before the current paradigm of github, stackoverflow, etc. I got to the point where I was cross-compiling with mingw and using libraries like libpng, libfreetype, libtheora, etc because frankly a lot of the system-level APIs at that time were not up to the demands that games placed on them (and probably still aren’t). I spent a ton of time in dependency hell because it was all C++ without a package manager, so it was a game of whack-a-mole just to update a single library. These were all huge red flags but I couldn’t see them because I was thinking to myself that if I could keep that mess together long enough, that I could put out a gold master of our flagship game on Mac, PC and iOS (and eventually Xbox etc) and everything would be all better. But what ended up happening that I spent the entirety of my time maintaining dependencies and trying to make rent.
Now id has certainly been down that road as well, and I’m not saying that the Windows platform is free of breaking updates. The big difference was that Windows was already the de facto gaming platform outside of consoles. So id could rely on a vast constituency of established game companies that would put Microsoft’s feet to the fire, or simply route around the damage with 1337 code snippets from the community. I always sensed a kind of reverence towards gaming, so that if Microsoft made a mistake, it was instantly under a magnifying glass. Contrast that with the Apple development community, where rampant fanboism has led to an Apple-knows-best mentality that prevents meaningful progress at even the most basic levels: OpenGL drivers are years behind, online documentation goes missing, iOS code signing is a hair raising process, etc etc etc etc etc.. I don’t want to be too hard on them but they’ve lacked a serious software engineering vision for so long that to fix it now would require a reevaluation of where they are going as a company.
I don’t know what to tell you as far as your own game development goals, but I know that if I was starting now, I would work at the highest levels that are still performant and likely to be maintained in the future. I prefer fundamentals over cookie cutter solutions, so I might opt to go with up-and-coming standards like WebGL and WebSocket running in Cordova to learn some fundamentals before applying them in Unity. The rule of thumb used to be that anything you were working on would be free and open source within 2-3 years, but that’s dropped to about 6 months today. So don’t stress too hard about it, just do what feels right and gives the shortest runway between planning and implementation. Often the conceptually correct solution and the most optimized one are the same or similar in the long run. Be wary of solutions like CocoaPods that seem to create their own diverging ecosystem. A few are ok, but if you find yourself knee-deep in packages that’s definitely a code smell. So is having too few, monolithic packages trying to take the place of the OS. Prioritizing your time gives you a roadmap on where to put your efforts and helps you remember “don’t be like that guy”.
My big mistake was to think that I could manage a game engine. This was in the late 90s and early 2000s before the current paradigm of github, stackoverflow, etc. I got to the point where I was cross-compiling with mingw and using libraries like libpng, libfreetype, libtheora, etc because frankly a lot of the system-level APIs at that time were not up to the demands that games placed on them (and probably still aren’t). I spent a ton of time in dependency hell because it was all C++ without a package manager, so it was a game of whack-a-mole just to update a single library. These were all huge red flags but I couldn’t see them because I was thinking to myself that if I could keep that mess together long enough, that I could put out a gold master of our flagship game on Mac, PC and iOS (and eventually Xbox etc) and everything would be all better. But what ended up happening that I spent the entirety of my time maintaining dependencies and trying to make rent.
Now id has certainly been down that road as well, and I’m not saying that the Windows platform is free of breaking updates. The big difference was that Windows was already the de facto gaming platform outside of consoles. So id could rely on a vast constituency of established game companies that would put Microsoft’s feet to the fire, or simply route around the damage with 1337 code snippets from the community. I always sensed a kind of reverence towards gaming, so that if Microsoft made a mistake, it was instantly under a magnifying glass. Contrast that with the Apple development community, where rampant fanboism has led to an Apple-knows-best mentality that prevents meaningful progress at even the most basic levels: OpenGL drivers are years behind, online documentation goes missing, iOS code signing is a hair raising process, etc etc etc etc etc.. I don’t want to be too hard on them but they’ve lacked a serious software engineering vision for so long that to fix it now would require a reevaluation of where they are going as a company.
I don’t know what to tell you as far as your own game development goals, but I know that if I was starting now, I would work at the highest levels that are still performant and likely to be maintained in the future. I prefer fundamentals over cookie cutter solutions, so I might opt to go with up-and-coming standards like WebGL and WebSocket running in Cordova to learn some fundamentals before applying them in Unity. The rule of thumb used to be that anything you were working on would be free and open source within 2-3 years, but that’s dropped to about 6 months today. So don’t stress too hard about it, just do what feels right and gives the shortest runway between planning and implementation. Often the conceptually correct solution and the most optimized one are the same or similar in the long run. Be wary of solutions like CocoaPods that seem to create their own diverging ecosystem. A few are ok, but if you find yourself knee-deep in packages that’s definitely a code smell. So is having too few, monolithic packages trying to take the place of the OS. Prioritizing your time gives you a roadmap on where to put your efforts and helps you remember “don’t be like that guy”.
Thanks for this informative reply!
My game idea is a kind of retro rogue-like, with a few modern conveniences. It's not going to bump up to any boundaries graphics-wise.
However, I've been disheartened to hear about the platform sprawl on android, which is my phone of choice. If it had any modicum of response, I would want to try to port it to iOS, and perhaps desktop platforms, which brings in another layer.
I feel that, I could embrace the game-design challenges, but platform issues would really kill my motivation to complete and launch the product. Especially when I don't own the platforms that people are reporting bugs on.
I know there are muli-platform simple games like Snakes and Agar.io that are pretty simple. Perhaps I should look into how those are written.
My game idea is a kind of retro rogue-like, with a few modern conveniences. It's not going to bump up to any boundaries graphics-wise.
However, I've been disheartened to hear about the platform sprawl on android, which is my phone of choice. If it had any modicum of response, I would want to try to port it to iOS, and perhaps desktop platforms, which brings in another layer.
I feel that, I could embrace the game-design challenges, but platform issues would really kill my motivation to complete and launch the product. Especially when I don't own the platforms that people are reporting bugs on.
I know there are muli-platform simple games like Snakes and Agar.io that are pretty simple. Perhaps I should look into how those are written.
Was hoping for Carmack, ended up with Romero.
I'd say all of these principles are still correct today, and apply outside of games, though perhaps only after appying an interpretive filter. Take:
> No prototypes – always maintain constantly shippable code. Polish as you go.
Romero then says that nowadays you will need prototypes. I'll believe him: but I think the right way to prototype is to reduce scope, not quality. So even the prototype is in some internal sense "constantly shippable" -- even if you are just shipping to the non-programmers within the company.
The alternative is to have a fragile prototype that gets kept because it seems almost ready and just needs a few bugs fixed and some small features added.
Two years and 500kloc later, you are still adding bugs faster than you can fix them.
> No prototypes – always maintain constantly shippable code. Polish as you go.
Romero then says that nowadays you will need prototypes. I'll believe him: but I think the right way to prototype is to reduce scope, not quality. So even the prototype is in some internal sense "constantly shippable" -- even if you are just shipping to the non-programmers within the company.
The alternative is to have a fragile prototype that gets kept because it seems almost ready and just needs a few bugs fixed and some small features added.
Two years and 500kloc later, you are still adding bugs faster than you can fix them.
> There was a lot of work that came before id's most famous game, Doom. Wolfenstein was his 87th game; Doom was around his 90th, he said.