top | item 21779583

The Outer Worlds: Fixing the “game thinks my companion is dead” bug

676 points| pkilgore | 6 years ago |twitter.com | reply

185 comments

order
[+] ergothus|6 years ago|reply
The real payoff to me is when someone asks why they didn't just try more ways of making companions unkillable without/before going through all that debugging. The response:

> that was our fallback, but

> -without actually knowing the cause, we couldn’t be sure that’d fix it

> -whatever the cause of the problem is might be causing OTHER problems, so we should find it

> -games near ship are fragile and it’s best to not make changes you don’t 100% understand

_THIS_ is someone that is practicing good software practices. I have no doubt this is why Outer Worlds is so much less buggy than, say, recent Bethesda _patches_. I saw a report this morning that the latest Fallout 76 patch causes some armors to degrade when a weapon reloads. This sounds exactly like the kind of bug you'd get from "fixing" something without understanding the actual cause, and thus just kicking up more emergent bugs because of your fix.

Complex interactions (like modern games), filled with shortcuts and assumptions (like modern games) means some bugs are inevitable. Requiring rigorous understanding before making changes can keep those bugs from multiplying.

...says someone with zero experience in coding games, but plenty of experience both fixing and generating bugs.

[Edit: fix formatting]

[+] _pmf_|6 years ago|reply
> _THIS_ is someone that is practicing good software practices.

There's nothing specific about software in there. My mechanic does the same type of reasoning.

[+] shultays|6 years ago|reply
It would be a good software development practice if they fixed it in a couple days top. It is not really acceptible if such a bug survived so long.
[+] mattmanser|6 years ago|reply
I don't buy this at all.

This is the antithesis of defensive programming. It's a game-breaking bug that's allowed to happen because of two things:

1. They allowed companions to take damage because they marked them with the wrong property

2. They never checked if the companions went somewhere silly to warp them back to where they should be

That's not defensive, that's like calling an API without a defensive try/catch if something goes unexpectedly wrong.

If they'd have done either of these things (and sent back error reports when it happened), they'd have caught it way sooner.

I don't write games and appreciate the interactions can be very complex. While I can understand your delight in them finding the root cause, I can't understand your support for failing to put in some defensive measures when they couldn't recreate a frequently reported bug.

[+] Danieru|6 years ago|reply
This thread is speaking to how regular programing and game play programming vastly differ.

In regular programming scopes are small, small enough for programmers to accomodate all expected logic. In games interaction scope is unlimited, or atleast that is what the game designers want.

If you try to program every expected system interaction you would never finish. So instead game programming is about making systems flexible. For example, as far as the gameplay code is concerned the inside of players ship is just yet another level. The bounds of any level are defined by what the map designers have layed out, not any pre-planned "ship area".

There are no meetings where project managers request "more playable area for level 7".

Likewise the systems programmer who made the furniture system was not the script writer who created the ladders. The script writer likely thought it was brilliant and elegant to split the entrances and exits, because in game programming it was indeed brilliant. It allowed the save system's existing code to handling what would have otherwise been special handling. It allowed the existing combat logic to handling what would otherwise be special handling.

Of course regular programming is different. In regular applications if a user's data is being sent to a server there is no secondary system which needs to be allowed to kill this data. Nor is there any save system which needs to be able to save and restore an in-progress action.

Games programming is all about figuring out ways to allow more varied and more complex interactions without exploding the implementation exponentially.

[+] konschubert|6 years ago|reply
This was an extreme insightful comment.

Originally, when I read the article, I thought that the issues described may be a result of bad software design.

But maybe it’s because applications are about strictness, and games are about freedom.

In application programming, you should be in a defined state most of the time, and make state transitions short and atomic.

In guess that in games, you are really in one huge, never-ending state transition.

[+] tomerv|6 years ago|reply
Why is this a difference between "regular" programing and game play programming? Any kind of programming could have complex interactions between component. A few well known examples: OS kernel and drivers, VM manager, internet browser. The list is long, and there is a lot of programming of complicated products in the world - not everything is web development or CRUD apps.
[+] qznc|6 years ago|reply
Regular programming is not different. You mention the case of users data being sent to a server. There is so much which can go wrong there when authentication, load balancing, session management, data storage, and network latencies interact.
[+] crimsonalucard|6 years ago|reply
It's not games that are the issue. Games like checkers or chess have less of this property.

The type of programs where things like this occur are called simulations.

[+] bipolar_lisper|6 years ago|reply
While bugs in game programming are quite obscene, and the task of making a game quite arduous, all this tweet did was demonstrate how really bad code design will bite you in the ass if you're making games.
[+] choeger|6 years ago|reply
Brilliant. It shows how your code is never encapsulated enough. You think physics should apply to all agents in your game and you think climbing a ladder is trivial and then someone thinks they can easily stop all NPCs from starting new Interactions. Boom.

In that particular case, what would be your favorite fix. I tend to say that the second step of climbing a ladder should probably not count as a new interaction.

[+] NoodleIncident|6 years ago|reply
I don't know if "encapsulation" is the right word, but the "undamageable" state should definitely have included fall damage. No point in having more than one way to reduce health points, damage is damage.
[+] jccooper|6 years ago|reply
At first I figured the physical solution, not being able to climb higher than the ladder in any case, would seem to be a reasonable one. And you probably want to do that. But that might somehow end up with characters stuck on a ladder if the dismount event never fires for some reason, so making the ladder a single event and\or exempting the dismount from the "blockable furniture actions" list, should also be done.

They should probably also consider adding fall damage as part of the considerations in invulnerable mode.

[+] ArchReaper|6 years ago|reply
> I tend to say that the second step of climbing a ladder should probably not count as a new interaction.

That might make sense when you view it through this specific lens, but that design decision allowed them to save character positions on ladders and make 'getting on' and 'getting off' be separate 'things' - just off the top of my head.

The '2 interactions' method actually makes the most sense in the context of their game engine - it's the same way getting 'on' or 'off' a chair works.

Their fix of just allowing interactions is probably the most correct option. That is essentially toggling a setting, as opposed to re-engineering how ladder mechanics work.

Games are complicated.

[+] gpm|6 years ago|reply
I'm not sure I understand why NPCs are prevented from interacting with the environment while you're talking... given that gameplay isn't stopped. Seems like an arbitrary decision that might lead to other surprising behavior too... e.g. talking to someone else to interrupt an NPC performing a scripted action.
[+] im3w1l|6 years ago|reply
My philosophy is that an important simple invariant shouldn't be maintained through the emergent behavior of a complex system.

If companions should be unkillable, then don't count on eliminating all sources of potential damage. Really make them unkillable. A ladder shouldn't ever take someone higher than the ladder goes so if some guy attempts to climb higher than the ladder goes, then freeze them in place or make them climb in place. Since these shouldn't trigger, crash the game if in dev mode, and if in prod mode check whether player allows telemetrics and if so phone home.

Those are the failsafe fixes.

Finally fix the root cause here by special casing that getting off a ladder during interaction is allowed.

[+] lazyjones|6 years ago|reply
> what would be your favorite fix

Not letting companions fall to their death, ever... Bethesda hasn't fixed these bugs in 5 years.

[+] rags2riches|6 years ago|reply
Bit of defensive programming would be a ceiling for the climbing action when it starts.
[+] kevin_thibedeau|6 years ago|reply
I had the same problem in a hardware product using a legacy wired protocol that was sent over a ZigBee connection. People would complain that volume would uncontrollably go to 100% when operating a remote.

Reproducing it was hard but the culprit was WiFi interference. The protocol had separate on/off commands and no provision for lost packets since it wasn't natively designed for wireless. If the off command dropped out everything would act as if a button was still pressed.

[+] fernandopj|6 years ago|reply
That'a well-worth read to everyone that enjoy reading about QA finding bugs, even if not interested in games Finding quest-related bugs is a nightmare in open-ended games...
[+] gambiting|6 years ago|reply
So I work in games as well, and one thing you learn really quickly is that this "one weird bug that QA has seen once and no one has been ever able to reproduce or even come up with an idea of how it could happen" will be reported thousands of times and cause uproar on forums once you sell 10+ million copies. It's just a statistical inevitably.
[+] loeg|6 years ago|reply
Ditto in enterprise, but that doesn't mean you're ever able to reproduce it in house before it ships. You do the best you can when you have a signal to follow. And there are probably other open bugs that impact more customers that deserve higher priority anyway.
[+] viraptor|6 years ago|reply
For all the flak people usually give the telemetry systems, this bug sounds like it could be made trivial if devs could push custom reporting to users. "When the companion dies, send me last few events and a minimal world-state". Having ready data collection/reporting when things go weird is super helpful, whether it's an own, server side software, or a remote client.
[+] lordleft|6 years ago|reply
CRPGs are my favorite genre of video game, and the complex interactions they feature make them both super fun and especially prone to glitches.

I will say that TOW was very stable for an Obsidian Game. I don't think I've experienced a bug yet.

[+] AdmiralAsshat|6 years ago|reply
Should've called this bug "Stairway to Heaven".
[+] iscrewyou|6 years ago|reply
But it ruins the story since that’s the punchline.
[+] pkilgore|6 years ago|reply
Don't skip the gif at the end!
[+] remarkEon|6 years ago|reply
Funny read.

The Outer Worlds is an unexpected joy, too. Definitely an open-world game I had no idea I needed. If you haven’t, check it out. It’s basically Fallout 4 but in space.

[+] Aromasin|6 years ago|reply
It very much nostalgically brought me back to the beginning of the decade, when fantastic "open-world" single-player RPG's like Mass Effect 2, Skyrim, Fallout: New Vegas, Witcher 2, Dragon Age: Origins etc were being realised in quick succession. For me personally, that was a golden era. It's great to have something like The Outer Words fulfilling that itch that has been present for so long. If you like any of the games mentioned, I also recommend checking it out.
[+] thrownblown|6 years ago|reply
...basically Fallout New Vegas in space... (obsidian made that)
[+] sdfin|6 years ago|reply
The combats quickly become too easy. The world and story are fine. I liked about New Vegas that it had a very elaborate world and the fights were more challenging.
[+] oneepic|6 years ago|reply
Lots of Bioshock vibes too.
[+] oniTony|6 years ago|reply
This reads a lot like the central premise in "You" by Austin Grossman -- unkillable NPCs are occasionally found to be dead and that bug is threatening the entire game studio.
[+] MattyRad|6 years ago|reply
Only tangentially related, sorry: all of the the companions in The Outer Worlds were uninteresting, I found it much more satisfying (and challenging) to beat the game solo. Effectively, each companion was an ordinary human, with some arbitrary specialities, joining your crew because the current mission required it. That said, The Outer Worlds was a lot of fun (albeit short-lived).
[+] doublement|6 years ago|reply
I don't play video games, but if someone created one that was entirely glitches like this, I might start.
[+] drewrv|6 years ago|reply
Check out Goat Simulator: https://en.wikipedia.org/wiki/Goat_Simulator

"The game, initially developed as a joke prototype from an internal game jam and shown in an early alpha state in YouTube videos, was met with excitement and attention, prompting the studio to build the game into a releasable state while still retaining various non-breaking bugs and glitches to maintain the game's entertainment value."

[+] ranger207|6 years ago|reply
You might enjoy speedrunning, especially older games like those for the Nintendo 64 or the original Playstation.

In addition, there's the Super Mario 64 0x A Button challenge, an effort to beat the game while avoiding the use of its main mechanic. It's not a speedrun, but the themes of exploiting every glitch possible is the same. It's also the source of the infamous "0.5x A Presses" video. https://www.youtube.com/watch?v=kpk2tdsPh0A

[+] swsieber|6 years ago|reply
Skate 3 has tons of glitches, or at least a few good glitches that keep on giving: https://www.youtube.com/watch?v=UaUR6u8nHoM

You can launch yourself into the air (100s of feet), run inside a dumpster to move it, drop through the floor to start falling from the sky, booby-trap a trashcan so it flies out of no-where and hit you upside the head, walk up an invisible ramp... it's fun.

[+] 91edec|6 years ago|reply
Playing any Bethesda game on release is a gold mine.
[+] scrollaway|6 years ago|reply
You'll probably enjoy The Stanley Parable then. If you haven't heard of it, I recommend you buy it and avoid actually reading about what it is. If you don't enjoy it I'll refund you myself ;)
[+] zf00002|6 years ago|reply
Check out Star Citizen, it's all terrible glitches and it's also unlikely to ever be a finished, playable game.
[+] squar1sm|6 years ago|reply
Gamedev is full of interesting problems. The industry scares me because of a few friends. Fun to hack on, on the side and I’m jealous of the story and mystery this team got to chew on for a while. I know it’s hard but still jealous of these gems.
[+] floatingatoll|6 years ago|reply
This is an excellent post-mortem, thanks for posting it.
[+] bitwize|6 years ago|reply
I imagined the MGS3 theme softly playing as the companion character climbed an invisible ladder off the map and into infinity.
[+] gambler|6 years ago|reply

[deleted]