top | item 14752123

Create your own Game Engine but don't use it

93 points| majikarp | 8 years ago |zeroequalsfalse.press | reply

109 comments

order
[+] thomascgalvin|8 years ago|reply
This can be said about plenty of things. Encryption is probably the most obvious; there's a lot of benefit from reading about and implementing various encryption methods, and absolutely foolish to use any of that work in a setting where it might actually be attacked.

The article is pointing out something that should be obvious; if you're making a widget as part of a larger task, your implementation will probably be lacking compared to one developed by a team solely focused on making the best widget they can. I could write an HTTP server or a threadpool or a database connection layer if I really wanted to, but thousands of talented people have invested millions of hours in those tasks already, and I'm not likely to do better than them on my own.

Getting over the "not invented here" mentality allows us to focus on doing things that are novel, or things that are custom. In my day job, I'm working on a (relatively) trivial problem that's never been done before, because there are about ten people in the world that need it. As a hobby I work on a text editor that is completely customized to my personal workflow, and which will probably never be used by anyone else.

[+] aaron-lebo|8 years ago|reply
Indeed, but there are a lot of situations where a focused specialized widget made by a small team will be better than a general "best widget" made by a large team. The further you stray from the use cases of the "best widget" the worse it gets.

Writing another HTTP server does seem like an inefficient expenditure of time, but we wouldn't have Nginx or smaller tools like Mongrel if the existing "best tools" were good enough. Game engines benefit you even more to be custom because the further your game strays from another Unity game, the further it stands out from the crowd of 100,000 smartphone games.

[+] aphextron|8 years ago|reply
>Getting over the "not invented here" mentality allows us to focus on doing things that are novel, or things that are custom.

I remember gradually coming to accept this as an intermediate programmer. Once you do, it's incredibly freeing. You can stop thinking in terms of "I'm gonna write SOOO much complex code to solve this problem!" to "I bet I can pipe together 5 different CLI programs and not write a single line of code".

You learn that it's all about layers of abstraction, and understanding that the human brain is far more useful and efficient at problem sets higher and higher on that scale of abstraction than it is at twiddling bits.

[+] octalmage|8 years ago|reply
I once wrote an OAuth client library for a language that didn't have one and learned so much. Now any time I do anything OAuth related I feel that I have a better understanding.
[+] lucasime|8 years ago|reply
Would you mind sharing your text editor?
[+] crucini|8 years ago|reply
There's no absolute answer to the "build vs buy" conundrum. A lot of us are biased towards "build" and its useful to recognize that bias in our decision making. But the "buy" side often has hidden costs. In the author's case, which involves targeting multiple platforms, the "buy" side gets more attractive. In general you are deciding whether to code on platform X or on platform Y built on top of X.

Fashions come and go, and if you invest heavily on top of Y it may become "obsolete". Will you be willing to stay on Y when the developer/community has abandoned it? You may have to reinvest to port onto Y', while X remained somewhat stable.

The higher level the platform, the more likely you are to hit surprising limitations late in the process. Your role gradually transitions from technology producer to technology consumer.

Programming an AVR chip in C gives you a great feeling of space and power because the platform is so clean, while using Arduino you rapidly hit limitations and spend your time getting around them. Higher level platforms tend to make the easy things easier and the hard things very hard.

I wouldn't rule out the subjective factors. For many of us building is more fun than sorting through what others have built. The brain works a lot better when having fun, which is why MBA-style decisions about programming don't always work.

[+] steeleduncan|8 years ago|reply
This blogpost gives the other point of view:

http://www.spacesofplay.com/blog/why-we-built-our-own-tech-f...

Essentially that the engine you use can constrain what you create, so write your own.

[+] thehardsphere|8 years ago|reply
I've considered a stack like that myself for less exotic looking games. It's nice to know someone who actually has gotten farther than I have is making similar choices and that those choices are working for them.

I think the choice boils down to project scope and the relative size and skill of the team. If a game of your scope is already well served by an existing engine, it makes sense to use it. If you have a large team with a particular skill set, it makes sense to cater to their skill set by picking the engine they know. For any other case, rolling your own makes sense, provided you know what the scope of the project is in advance.

The common mistake (that maybe the guys in the parent article also made) is that they assume "game engine" is some necessary prerequisite to making games, so they spend time working on all of these technical problems that are not actually their game they want to make.

[+] Kenji|8 years ago|reply
Thank you for presenting an encouraging counterpoint. I think own game engines can work, but you'll probably have to stick to 2D or 2.5D because 3D introduces such great amounts of additional complexity - and even though 3D is not extremely hard, it's a huge time sink.
[+] zellyn|8 years ago|reply
Other games with interesting/different visual “looks” and custom 3D engines are Reset and The Witness.
[+] karmadog|8 years ago|reply
There's two big hindrances for success right off the bat:

1. Refusing to just use C/C++

2. Attempting to support most of the platforms

Both problems compound each other.

All platforms (even Android at this point) have C APIs for everything you need. Creating another layer of abstraction here causes more work and overhead.

Trying to get Java running on iOS with 3D APIs is a huge task in and of itself. The same goes for C# (a task which Unity does for you). Languages that require JIT for performance don't work on mobile (unless supplied by the OS, e.g. JavaScriptCore). Debugging becomes extra-difficult.

C/C++ are not convenient to use, but not hard to use either. It causes discomfort at first, but you get used to it. Being forced to do memory management manually also helps with learning how to structure programs, with a bonus of not having to wrestle GC pauses.

Having said all that, using a commercial engine is totally fine, of course.

[+] Koshkin|8 years ago|reply
> Being forced to do memory management manually

It may be worth repeating that in C++ the need to manually manage memory is drastically reduced compared with C (due to the native support of the RAII pattern).

[+] davexunit|8 years ago|reply
I refuse to use C/C++. If that's what I have to use to make games, I just won't make games. Fortunately, there are other languages available with a C FFI.

I don't think forced memory management teaches anyone much of anything besides how to manage memory manually. It doesn't make programs better. I'm very grateful for garbage collection so that I can spend my cycles thinking about the problem I'm actually trying to solve.

edit: I think this came off a bit too dismissive, but I guess what I was after is that what technology you choose depends on what you value. Personally, I value using programming languages that are pleasant to work with, and I don't personally find C/C++ to be pleasant. I enjoy dabbling in game development, but it's not so important to me that I would use tools that I don't like to do it.

[+] pjmlp|8 years ago|reply
1. Refusing to just use Assembly.

Was the reply we got in the old days, when one used C, Pascal, Basic, Amos.

Thankfully many ignored that and moved the state of art to such languages.

A few years later the reply would be,

1. Refused to use C

When devs tried to use C++ or Delphi.

State of the art in games development only advances thanks to those that manage to produce entertaining games, while ignoring such advices.

[+] dakics|8 years ago|reply
Unity's multi-platform track record is quite solid at this point in time.

I wouldn't bet on exotic Java attempts of "write once run everywhere" though.

[+] thefoxbard|8 years ago|reply
I really don't think the issue was with "creating x from scratch", but from the purpose it was built for.

Game engines like Unity are general-purpose engines, they are not made for a specific type of game, and can be used for basically anything. The trade-off is flexibility and performance.

What they tried to do was to basically create a "clone" of Unity, that is, they tried to create a general purpose engine, and that does not make any sense at all for an indie developer.

If you're going to create a game engine + editor, then do it for a specific genre of games (e.g. FPS), and do it only if you're planning to create a series of games that belong to that genre only.

Unless you want to create a business out of selling game engines, then have a specific game in mind before creating a game engine, and tweak it for that purpose only.

[+] alol|8 years ago|reply
Having a broader understanding of your domain at a different level of abstraction is always extremely valuable.

For anyone interested in the detail of _how_ to create a game engine from scratch, I'd recommend checking out Handmade Hero https://handmadehero.org/

[+] fellellor|8 years ago|reply
Handmade Hero is over 380, hour or two long videos last i saw. How far are you into it?

Regardless, it's an amazing effort.

[+] CJefferson|8 years ago|reply
On the other hand, I find my game engines, particularly (unfortunately) open source ones, are in a terrible state.

They often try to support many platforms, then it turns out they usually support one platform well, and the others are various degrees of broken. Then, when you want to get your game working on Android, the engine is too massive and complex for me to have any chance of figuring out how to fix it.

[+] TillE|8 years ago|reply
Too massive and complex, plus if you want to do anything a little unusual (eg, procedural generation) you'll often find most engines are working against you.

I can't imagine writing a 3D game from scratch in this day and age, but for 2D games I think it's absolutely justifiable to just grab something like SDL2 and go.

[+] oneZergArmy|8 years ago|reply
This is why i prefer frameworks like LÖVE 2d. it's so simple that I can keep everything I need in my head. I like just staying in the text editor.
[+] crucini|8 years ago|reply
Seems to be a subset of libraries and frameworks in general. They are not up front with the limitations of their software, so you have to invest a lot of hours finding them.
[+] avaer|8 years ago|reply
Interesting contrast to the Carmack school of thought: _do_ build your own game engine and don't even reuse anything you made before, because you'll know better this time.
[+] sclangdon|8 years ago|reply
Also remember that id Software wrote an engine specific to the game they were creating (it's debatable whether you can call them engines at all, really). Things are a lot more streamlined if you take this approach because you rarely, if ever, have to answer the question "would this work for some other title?". Or worse, "would this work for any future title that may not even be in the same genre?"!

That, I think, is the mistake many game devs (who write their own engines) make. They make an engine for all possible games, then try to use it for their 2D match-3 puzzler, which uses about 5% of the code they've written, and even then they have to cludge some bits.

[+] thomascgalvin|8 years ago|reply
Most of us aren't John Carmack, though. When you're the guy that basically invented the genre of code you're talking about, there's a very good chance that your next project will advance the state of the art. When you're basically a hobby coder looking to get an game in the app store, it's a slightly different story.
[+] tomku|8 years ago|reply
It's worth pointing out that without id's engine licensing business that depended on other companies NOT following the Carmack school of thought, it would've been harder to justify writing a new engine for every game.
[+] FRex|8 years ago|reply
Where did you get that information from? Id tech is a clear engine family/line, they aren't named 1, 2, 3, 4, 5 and 6 for fun(except for the first one which really doesn't fit the rest).

1. Carmack himself says he wrote D3 renderer inside Q3 in C and then ported it to C++ (Doom 3 was first C++ id game). 2. Carmack himself says between Q1, 2 and 3 the move was "evolutionary" and lots of code stayed the same. 3. On Mac the main of D3 is even called quakeMain. 4. Radiant - GtkRadiant is shared between many games and various radiants (Net, Dark, Gtk, etc.) share a lot of code with each other, DarkRadiant is for example a fork of GtkRadiant meant for D3 and TDM. 5. The MegaTexturing code must definitely be shared between 4, 5 and 6 because it'd be crazy to reinvent this from scratch each time to do the same thing.

The only big/throw everything away jumps are from previous odd games (Commander Keen which was 2D, Wolf3D which was a super simple ray caster) to id tech 1 (Doom 1 and 2, pure software, fake 3D) to id tech 2 (Q1 and 2 and beyond, GL but often with software fallback, real 3D). And that kind of jump is like giving web devs shit for not writing Java browser applets and thus "not reusing what they made before". From 2 and beyond it's evolutionary with the data formats, editors, code, etc. and never a total rewrite from scratch. Case in point: the use of OpenGL (or wrapping other APIs in a GL like one when there is no GL), Carmack used to (90s) prefer it over DirectX but much later in life (2010s) switched his preference but the code stayed with OpenGL out of inertia. Even Rage and new Doom use OpenGL (id tech 5 and 6 respectively). Why would this happen instead of switching to DX if they threw everything away with each new engine and never reused code?

Granted: lots of stuff and the ENTIRE scripting system (and Carmack's opinion of scripting) change almost each iteration but "change most of stuff but it's very evolutionary and lots of code is same" is far from your claimed "don't even reuse anything".

As for his opinion on writing your own engine it's not super clear, few years back he admitted that Epic dominated that field, but he was happy to stop licensing his tech to outsiders because it was a burden and started almost by an accident ("It's interesting when you look at our technology licensing -- it was never really a business that I wanted to be in. In the very early days, people would pester us, and we'd just throw out some ridiculous terms, and we were surprised when people started taking us up on it."). He is also reportedly a very calm, reasonable and polite person so even if he thinks reusing engines is shit he won't tell us. One interviewer who interviewed him said he just turned around in his chair, didn't even say 'hi', answered all questions in length in detail with no BS and then just thanked for the interview and turned back away to his screen without a 'bye'. Embodiment of pragmatism and niceness, despite the lack of 'hi' and 'bye'.

Sources: http://fabiensanglard.net/doom3/renderer.php http://fabiensanglard.net/doom3/interviews.php http://icculus.org/gtkradiant/developers.html https://en.wikipedia.org/wiki/Id_Tech_6 http://www.gamasutra.com/view/news/125324/E3_ids_Carmack_Wil...

[+] ganoushoreilly|8 years ago|reply
I think a lot can be said for someone trying to go their way, and learning the value of a prebuilt solution. A lot of people would stubbornly keep trying to build their own "because they could build better" rather than focusing on the end outcome of a playable product.
[+] thehardsphere|8 years ago|reply
The mistake that most people make is thinking about whether or not they should make a "game engine" when what they really want to do is make a game.

If you want to make a game, make it. If it makes sense to use somebody else's engine to make it, then by all means do so. If you aren't going use someone else's game engine (which is a perfectly valid choice), skip straight to making your game. After you've made a game or two, you'll know whether making a "game engine" out of what you've written previously is a worthwhile effort for your next game.

[+] golergka|8 years ago|reply
I've worked with Unity from the beginning of my career as a developer. (Used to be a game designer and producer before that). I haven't quite understood what's so special about it before I tried other solutions, including open-source (Cocos) and in-house ones; turns out, a LOT of things that Unity devs take for granted (like asset pipeline and management) are lacking elsewhere.

Sure, a senior developer should be able to write his own game engine from the lowest level - but he almost never should actually do it on a real project.

[+] kbenson|8 years ago|reply
This is good knowledge for whatever your field is. I've created a few ORMs. I don't use any of the ones I wrote, but I learned a lot in doing so. I've created a few web routing systems. Again, you can learn a lot, but unless you actually want to spend a lot of time writing and extending one, just use what you learned to help you pick a good one.
[+] mherrmann|8 years ago|reply
Developing indie games is hard enough. Starting off by building your own engine is crazy (unless there are _very_ good reasons). They unnecessarily and gravely harmed their chances of success in a very competitive niche.
[+] advice_giver|8 years ago|reply
This blog post and the comments here are amazing. In no other software field but indie game dev does spending a year trying to build your own toolset instead of $ACTUAL_PRODUCT_THAT_MAKES_MONEY draw praise.
[+] thehardsphere|8 years ago|reply
It's cargo-cult word-thinking. "Game engine" is associated with "game" because of the success of id Software, Epic Games, Valve, CryTek, et al. So people who don't think the way programmers do, have the impression that a game engine is a required step in the software development process to have a successful game.
[+] krapp|8 years ago|reply
This is apparently what I've been doing for about two years.

Except I still haven't gotten to the "engine" part, it's more "a bunch of wrappers around Lua and SDL2 and some random stuff that seems useful, and probably the worst ECS ever."

I actually did almost get to an actual game, though. But I was so disgusted with it that I gave up and started everything over from scratch. Mind you, everything worked fine, I just hated every thing about the code.

And this is why I am an amateur programmer and a professional box-pusher.

[+] tisdy|8 years ago|reply
I'm interested in pushing boxes. It sounds relaxing. Well, relatively. Have you heard of webpack?
[+] sbov|8 years ago|reply
It seems like you can learn a lot of that by building a game without using a game engine at all.
[+] psyc|8 years ago|reply
Working on low level engine things for 15+ years lets me wield Unity like a sword now. It's a mid-level sword, but my technique makes up for that.
[+] aphextron|8 years ago|reply
I think a lot of developers who come into game dev have a mistaken view of Unity as the "budget" choice based on the poor quality of many of the indie titles shovleled out with it. But Unity is probably the most advanced game engine in existence today. Far beyond Unreal 4 in terms of both performance and tooling.

The reason the average quality is so low with Unity titles is because of just how easy the technical aspect of making a video game has become with modern engines. The barrier to entry for creating a game with AAA graphics is now a $20 shader pack and the ability to hack together some C# scripts.

[+] camus2|8 years ago|reply
> Far beyond Unreal 4 in terms of both performance and tooling.

No, that's untrue. First it's much much easier to develop a game from scratch with Unreal without a single line of code, which isn't possible with Unity.

Second Unreal "vanilla" is more advanced in every ways. Unity is a toy compared to it. Unity is "budget" and successful because it used to be much much more affordable than Unreal. That's barely the case today. There is very little reason to use it instead of Unreal.

I started game dev with all that C++ bullcrap and DirectX headers with its insane API and message pump, and there is absolutely nothing better than Unreal in my book today, both in terms of ease of use, pricing and performances.

[+] mikulas_florek|8 years ago|reply
Vanilla Unity tooling is a joke. Of course it's enough for most games. Store solves some issues, but not all.
[+] lentil_soup|8 years ago|reply
I don't know what tools you're talking about, Unity comes with hardly any tools at all. You have to buy them in the Asset Store if you want anything a little bit more advanced.

Then again, Unity IS way easier to get started with, easy to script in and get something running and brilliant for prototyping quick ideas.

[+] emodendroket|8 years ago|reply
I have pretty much no experience with game development, but why is UE4 such a popular choice for commercial games if that's the case?
[+] hacker_9|8 years ago|reply
Agree, Unreal Engine still has a lot of catching up to do in terms of ease of use and performance (even importing an FBX file in the editor is a slow affair).
[+] aaron-lebo|8 years ago|reply
A year is nothing in the grand scheme of things. If you can't invest a year into building tools that could be your foundation for the next 3 decades, then what you are building is just a quick-rich scheme that doesn't matter. Use whatever you want because it doesn't matter...and your game probably isn't going to sell anyway. Those are the odds.

But as they've pointed out, you kind of need to know that stuff anyway, so it's time well spent.

I don't have a ton of gamedev experience, but at least graphically that game isn't doing anything you can't do in a few hundred lines of C++. Maybe that's part of their problem. Why use Java if you want to make a mobile game and it isn't well supported on one of your primary platforms? That sounds like a bad fit.

Unity and Unreal are super cool but they are also bloated with features that you don't need and lacking in others you do. For the longest Unity required you to use 3rd party modules for a camera. A camera....something that takes a few lines of code for a basic implementation. Why didn't that come standard?

If you really want to hurry to market yet another game by all means use one of the existing engines. It might even work really well. But if you want to do anything interesting, you're gonna be doing custom stuff anyway.

[+] kevindqc|8 years ago|reply
>For the longest Unity required you to use 3rd party modules for a camera.

What do you mean? How can there not be cameras? How then can you view a scene without that 3rd party module?