FWIW, a nice comment[1] from forum.nasaspaceflight.com (no affiliation with NASA):
> I'm retired now but did propulsion and systems engineering on the Transfer Orbit Stage (TOS) developed by Orbital Sciences and Lockheed Martin for NASA/MSFC in the 1990's. [...] I'll make a few comments on how/where things might have gone off the rails with the RCS thruster thermal problem.
> Almost all problems occurred at interfaces between companies (prime vs. sub, customer vs. prime) or between different groups within the same company, where one group misunderstood what another group was doing, or at actual mechanical and electrical interfaces between components designed and built by different groups.
This is obviously a well-known phenomenon in software engineering and I don't think anyone here is going to be be particularly surprised that it occurs in the aerospace setting. What is a little more surprising, to me at least, is that the systems people over there don't have procedures in place to minimise risks stemming from lack of communication.
It isn't realistic for any sub-team to be fully familiar with the overall system but surely, for instance, if a team is working on component X which interfaces with components Y and Z, then it should be standard practice for the X team to spend at least some time with the Y and Z teams during development?
From one of the twitter posts cited in the forum post:
> Curious if the root is someone at Boeing accidentally not relaying vehicle updates to vendors, or if it was a conscious decision to avoid paying for change requests.
Seeing how Boeing "incidents" have piled up in recent years, and reading how most (or perhaps all) of those issues were due to "cost saving" measures, I wouldn't put it past them to have made that decision consciously, lives be damned.
I don't know if it's supposed to take a rocket scientist to figure out whether Boeing these days has been living up to its 20th century reputation for improved reliability.
From the comments it can be pretty succinct:
>Yes, I know the aircraft and space divisions are separate. Doesn't matter. Shit always runs down hill if Corporate is squatting.
I've mistaken them for the official nasa webcast more than once before realizing that the two casual dudes talking can't possibly be official commentators. Isn't this some sort of trademark infringement they're doing?
I’m shocked they’re able to keep operating like this. Can I walk around Seattle video recording bicycle thieves and call it “Seattle Police Department Video”?
I'm more upset at NASA than Boeing over this for downplaying this issue for months while doing very serious things in the background like hot fire testing. Not transparent at all. You can easily see how public perception thought everything was fine all through July here:
I'd love an investigation to see if the public perception matched NASA's perception. I would be money that it doesn't which means NASA has been hiding the truth from the public. How can anyone trust what NASA says after this?
This reads as histrionics. You want an investigation into whether the general public felt the same panic people on the project do? No thanks, I'm alright with letting them get on with it and getting the full picture later.
> I'm more upset at NASA than Boeing over this for downplaying this issue for months while doing very serious things in the background like hot fire testing. Not transparent at all.
Is "hiding the truth" only a view if the truth is worse than the public think or could you imagine arguing that NASA "hid the truth" that its safer than the public thinks?
Objectively I suspect the only hidden truths here are perceptions/knowlege that its worse than people think. Hiding you think its better is .. unlikely.
Personally I don't ascribe a moral hazard dimension here. Probably, the NASA officials who had the power to state things, were not the ones conducting testing and their PR people were put on hold. I think its a malice/incompetence thing (Hanlon's razor)
I don't understand. What would be the difference if they didn't downplay this? There'd be a massive shitstorm distracting resources at NASA and Boeing from doing their jobs and we wouldn't be any better off. How is that better for anybody?
Why is handling the issue quietly worse? Let the engineers do their fucking jobs.
> I'm more upset at NASA than Boeing
More upset than the company that couldn't build a functioning, reliable rocket? Get a grip, dude.
Curious as to moral in the Boeing division right now. If you designed/built/influenced any part of the design and watching this play out publicly. Leaving astronauts stranded and potentially with a module stuck on the space station.
Do you definitely start looking for a new job? Assume that ultimately nothing will change?
>> Curious as to moral in the Boeing division right now.
I'm even more curious about the astronauts. Are they willing to risk it? Are they even part of the discussions? Are they saying "screw that thing, get me a dragon"? I haven't heard a single word about their take on it.
What's the problem? The vehicle mostly worked. It's like your app shipped but had a spinning cursor issue and users had to manually clear cache. It's an overwhelming success by standards of software industry.
I wonder sometimes whether NASA should lean into the high risk of spaceflight instead of trying to minimize it. If they could get the public to pay attention, their budgets would go up. Highlighting the risk--without exaggerating--would be a good way to get people to care. People love (maybe even crave) drama.
Astronauts accept an amazing amount of risk, even when using proven systems like Soyuz or Dragon. ISS is one unlucky micrometeoroid strike away from total catastrophe. And yet hundreds of astronaut candidates are jostling with each other (another great drama) to be next on the rocket.
Even uncrewed missions are filled with drama. Imagine devoting 20 years of your scientific career on a probe to Mars and having it blow up on take-off or smash into the Martian surface--so close, and yet so utterly useless.
I think NASA fears that highlighting risk leads to bad press. NASA doesn't want headlines like, "NASA ignores safety concerns--story at 11". But ironically, when NASA minimizes risk, they lower the threshold for how much risk the public will accept. The more they minimize risk, the less risk the public will let them take.
I don't have any good suggestions, though. Highlighting risk inevitably invites the question of "why are we taking the risk at all?" And that's also a hard conversation.
Why is NASA covering for Boeing? Jettison that shit and let it crash into the ocean as a burnt hunk as their infinite hemming and hawing indicates is apparently overwhelmingly likely to happen.
Can you guarantee that hitting the "undock and re-enter" button right now would result in Starliner safely leaving the ISS and then clearing its orbit?
Even if that had an acceptable level of risk, that still leaves two extra crew on the ISS with no seats home in case of an emergency, and NASA's policy in recent years has been to always have emergency return capacity for every crew member onboard.
I'm not saying there isn't a path forward that involves sending Starliner back empty, there are just a lot of considerations going into that decision right now.
At some level by covering or Boeing they are covering for themselves. They were the ones putting the astronauts on it, after all.
But there is another level: there some kind of a background hate directed toward Musk and Space X. Someone in government agencies is asking themselves, how could we put some sticks in Musk's spokes? Some ask him to kidnap seals and put headphones on their heads [1] or calculate what's the chance his rockets would hit whales in the Pacific Ocean [2]. So it's not that they particularly love Boeing that much, but if Boeing's success makes Musk's company look worse, fine, then they'll support Boeing.
Imagine a scenario, for a moment, that the situation is reversed. Space X is the capsule with the issue and Boeing is the one with the cheaper and working version. There would be no hesitation to pointing fingers and accusing Space X make a large media stink about it instead of covering up.
> Whether the seals would be dismayed by the sonic booms. Now, there’ve been a lot of rockets launched out of Vandenberg and the seal population has steadily increased. So if anything, rocket booms are an aphrodisiac, based on the evidence, if you were to correlate rocket launches with seal population. Nonetheless, we were forced to kidnap a seal, strap it to a board, put headphones on the seal and play sonic boom sounds to it to see if it would be distressed. This is an actual thing that happened. This is actually real. I have pictures.
> Now, again, you look the surface, look at the Pacific and say what percentage of the Pacific consists of whale? I could give you a big picture and point out all the whales in this picture. I’m like, I don’t see any whales. It’s basically 0%, and if our rocket does hit a whale, which is extremely unlikely beyond all belief, fate had it, that’s a whale has some seriously bad luck, least lucky whale ever.
Just to make it clear, I don't like Musk, I don't have any stock in his companies, and don't buy his cars or use twitter/X. But it's still interesting to observe this effect of cover up and strange push against Musk.
They removed the autonomous flying part for this mission, so they can't jettison it without a human inside. They are supposedly working on adding that feature back in. It also cannot be attached to the canada arm so they can't even clear it away from the port its using.
The whole thing speaks to complete mismanagement on every level. That they still haven't made any kind of decision 3 months in is absolutely laughable.
Now they are saying the astronauts could be up there until MARCH. They miscalculated by EIGHT MONTHS. These people are complete clowns.
Dump that pile of junk, cancel the program, and fire all the managers involved in this cosmic fiasco.
If anyone here's familiar with how these decisions are made, I'm curious about why NASA says they need another week to choose their path forward. Given that we're already over two months into a week-long mission, what information don't they have that they would have in another week?
>Given that we're already over two months into a week-long mission, what information don't they have that they would have in another week?
As an Ars commenter observed <https://arstechnica.com/space/2024/08/nasa-acknowledges-it-c...>, it is possible that the real issue isn't whether Starliner is safe to return with humans. If that were the question two months of debate are, as you said, by itself enough to say "no". Return Wilmore and Williams on Crew Dragon. Done.
The commenter posited that the real issue is that NASA does not trust Boeing's software to undock Starliner autonomously. We know that Wilmore had to take manual control on the way up because of the thruster issues. NASA may fear that if thrusters fail again, Starliner software may again not be able to handle them, and the spacecraft might ram ISS. Thus, the agency wants a human to be able to take over if necessary. *That* is the dilemma. This is something that I and others had mentioned over the past couple of weeks, but the Ars commenter is I think the first outside NASA to put it so starkly.
Well, they said they would do more modelling. Since they've definitely already done a pile of modelling, the remaining modelling is probably running down a list of alternate assumptions and approaches in whatever modelling system they've been using. Most likely they've already gone ahead modeled out all of their most likely and high confidence assumptions and approaches. Likely the modelling results haven't quite converged, leading them to be unsure.
In parallel, my pet theory is that NASA has probably already made up its mind (astronauts are not going to return on Starliner), and have been dragging this out mostly to make it look like they aren't just going to throw their contractors under the bus (even if they deserve it). Boeing has declared cold feet over fixed cost contracts (in general, not just with NASA), and I think NASA wants to keep the rest of the contractor pool at least at ease that, okay maybe NASA might start being stingier with the money and contracts, but they aren't going just throw you under the bus when issues appear.
Well, unlike in web software, very real people could die if they screw up, and they aren't exactly pressed for time right now, so what's wrong with being careful?
I think for part of your question (why do they specifically say a week), the length isn't all that meaningful. As in, if they want to more time to make a decision, they'll just announce another week's delay.
They have weekly status update conferences, and just cancelling those conferences might be more of a PR risk than just keeping the conference and announcing that they're delaying making a decision.
It's intriguing to me that they seem to be prioritizing information collection to determine whether Starliner is viable, as opposed to definitively announce a return via SpaceX and making preparations for that.
It's like if I have a service outage, maybe I might spend a few minutes to collect debugging information, but my priority would be to bring the service back up via rollbacks or whatever to restore a previously known good state. Currently they are debugging Starliner with people stranded, but maybe they should prioritize on getting those people back home first.
Or maybe everyone involved don't consider being stranded for months in space as a bad state.
They may still be chasing down some loose ends. While additional time allows for some more models and theories, I think it becomes exponentially less likely that it will alter the safe course of action.
If the system is so complex that an extra week does yield some major new insights, that’s way too complex to use.
I believe the article contains the answer to your question. It says "engineers will attempt to model the behavior of the valve with the bulging Teflon seal over the next week and its effects on thruster performance."
As a big SpaceX fan, I appreciate the innovation and success that SpaceX has brought to space exploration. However, it's crucial that we have multiple reliable launch and crew providers to ensure the safety and sustainability of space missions. While SpaceX has been a game-changer, relying solely on one provider is risky. The ongoing issues with Boeing's Starliner highlight the importance of diversity in our space program. We need to support and develop multiple providers to maintain a robust and secure presence in space.
I'm somewhat surprised that, after the SpaceX / Boring Company "rescue submarine" offer a few years ago, Elon Musk hasn't personally suggested (over X, of course) that SpaceX send up another Dragon inside of some compressed timeframe. I'm assuming there must be some other limitations at play - maybe one can't be readied that fast, or there's some other regulatory reason?
[+] [-] mncharity|1 year ago|reply
> I'm retired now but did propulsion and systems engineering on the Transfer Orbit Stage (TOS) developed by Orbital Sciences and Lockheed Martin for NASA/MSFC in the 1990's. [...] I'll make a few comments on how/where things might have gone off the rails with the RCS thruster thermal problem.
[1] https://forum.nasaspaceflight.com/index.php?topic=60593.msg2...
[+] [-] gnfargbl|1 year ago|reply
This is obviously a well-known phenomenon in software engineering and I don't think anyone here is going to be be particularly surprised that it occurs in the aerospace setting. What is a little more surprising, to me at least, is that the systems people over there don't have procedures in place to minimise risks stemming from lack of communication.
It isn't realistic for any sub-team to be fully familiar with the overall system but surely, for instance, if a team is working on component X which interfaces with components Y and Z, then it should be standard practice for the X team to spend at least some time with the Y and Z teams during development?
[+] [-] fransje26|1 year ago|reply
> Curious if the root is someone at Boeing accidentally not relaying vehicle updates to vendors, or if it was a conscious decision to avoid paying for change requests.
Seeing how Boeing "incidents" have piled up in recent years, and reading how most (or perhaps all) of those issues were due to "cost saving" measures, I wouldn't put it past them to have made that decision consciously, lives be damned.
[+] [-] fuzzfactor|1 year ago|reply
From the comments it can be pretty succinct:
>Yes, I know the aircraft and space divisions are separate. Doesn't matter. Shit always runs down hill if Corporate is squatting.
[+] [-] moffkalast|1 year ago|reply
I've mistaken them for the official nasa webcast more than once before realizing that the two casual dudes talking can't possibly be official commentators. Isn't this some sort of trademark infringement they're doing?
[+] [-] mulmen|1 year ago|reply
I’m shocked they’re able to keep operating like this. Can I walk around Seattle video recording bicycle thieves and call it “Seattle Police Department Video”?
[+] [-] bottlepalm|1 year ago|reply
https://manifold.markets/Shihan/will-spacex-dragon-rescue-bo...
I'd love an investigation to see if the public perception matched NASA's perception. I would be money that it doesn't which means NASA has been hiding the truth from the public. How can anyone trust what NASA says after this?
[+] [-] burnished|1 year ago|reply
[+] [-] TMWNN|1 year ago|reply
As late as July 28, NASA flight director Ed Van Cise explicitly denied that the Starliner crew was stuck or stranded <https://x.com/Carbon_Flight/status/1817754775196201035>. Even if one quibbles about whether "stranded" applies in this situation (I believe that it does <https://np.reddit.com/r/space/comments/1ekicol/not_stranded_...>), "stuck" definitely does.
[+] [-] ggm|1 year ago|reply
Objectively I suspect the only hidden truths here are perceptions/knowlege that its worse than people think. Hiding you think its better is .. unlikely.
Personally I don't ascribe a moral hazard dimension here. Probably, the NASA officials who had the power to state things, were not the ones conducting testing and their PR people were put on hold. I think its a malice/incompetence thing (Hanlon's razor)
[+] [-] Sakos|1 year ago|reply
Why is handling the issue quietly worse? Let the engineers do their fucking jobs.
> I'm more upset at NASA than Boeing
More upset than the company that couldn't build a functioning, reliable rocket? Get a grip, dude.
[+] [-] 0cf8612b2e1e|1 year ago|reply
Do you definitely start looking for a new job? Assume that ultimately nothing will change?
[+] [-] phkahler|1 year ago|reply
I'm even more curious about the astronauts. Are they willing to risk it? Are they even part of the discussions? Are they saying "screw that thing, get me a dragon"? I haven't heard a single word about their take on it.
[+] [-] charlie0|1 year ago|reply
[+] [-] bamboozled|1 year ago|reply
[+] [-] katzinsky|1 year ago|reply
As long as the paychecks don't bounce they're probably more worried about their individual KPI than the actual results.
[+] [-] black_13|1 year ago|reply
[deleted]
[+] [-] numpad0|1 year ago|reply
[+] [-] GMoromisato|1 year ago|reply
Astronauts accept an amazing amount of risk, even when using proven systems like Soyuz or Dragon. ISS is one unlucky micrometeoroid strike away from total catastrophe. And yet hundreds of astronaut candidates are jostling with each other (another great drama) to be next on the rocket.
Even uncrewed missions are filled with drama. Imagine devoting 20 years of your scientific career on a probe to Mars and having it blow up on take-off or smash into the Martian surface--so close, and yet so utterly useless.
I think NASA fears that highlighting risk leads to bad press. NASA doesn't want headlines like, "NASA ignores safety concerns--story at 11". But ironically, when NASA minimizes risk, they lower the threshold for how much risk the public will accept. The more they minimize risk, the less risk the public will let them take.
I don't have any good suggestions, though. Highlighting risk inevitably invites the question of "why are we taking the risk at all?" And that's also a hard conversation.
[+] [-] humansareok1|1 year ago|reply
[+] [-] stetrain|1 year ago|reply
Even if that had an acceptable level of risk, that still leaves two extra crew on the ISS with no seats home in case of an emergency, and NASA's policy in recent years has been to always have emergency return capacity for every crew member onboard.
I'm not saying there isn't a path forward that involves sending Starliner back empty, there are just a lot of considerations going into that decision right now.
[+] [-] rdtsc|1 year ago|reply
But there is another level: there some kind of a background hate directed toward Musk and Space X. Someone in government agencies is asking themselves, how could we put some sticks in Musk's spokes? Some ask him to kidnap seals and put headphones on their heads [1] or calculate what's the chance his rockets would hit whales in the Pacific Ocean [2]. So it's not that they particularly love Boeing that much, but if Boeing's success makes Musk's company look worse, fine, then they'll support Boeing.
Imagine a scenario, for a moment, that the situation is reversed. Space X is the capsule with the issue and Boeing is the one with the cheaper and working version. There would be no hesitation to pointing fingers and accusing Space X make a large media stink about it instead of covering up.
[1] https://lexfridman.com/elon-musk-4-transcript/
> Whether the seals would be dismayed by the sonic booms. Now, there’ve been a lot of rockets launched out of Vandenberg and the seal population has steadily increased. So if anything, rocket booms are an aphrodisiac, based on the evidence, if you were to correlate rocket launches with seal population. Nonetheless, we were forced to kidnap a seal, strap it to a board, put headphones on the seal and play sonic boom sounds to it to see if it would be distressed. This is an actual thing that happened. This is actually real. I have pictures.
[2] https://lexfridman.com/elon-musk-4-transcript/
> Now, again, you look the surface, look at the Pacific and say what percentage of the Pacific consists of whale? I could give you a big picture and point out all the whales in this picture. I’m like, I don’t see any whales. It’s basically 0%, and if our rocket does hit a whale, which is extremely unlikely beyond all belief, fate had it, that’s a whale has some seriously bad luck, least lucky whale ever.
Just to make it clear, I don't like Musk, I don't have any stock in his companies, and don't buy his cars or use twitter/X. But it's still interesting to observe this effect of cover up and strange push against Musk.
[+] [-] macintux|1 year ago|reply
[+] [-] cedws|1 year ago|reply
[+] [-] plopz|1 year ago|reply
[+] [-] AmVess|1 year ago|reply
Now they are saying the astronauts could be up there until MARCH. They miscalculated by EIGHT MONTHS. These people are complete clowns.
Dump that pile of junk, cancel the program, and fire all the managers involved in this cosmic fiasco.
[+] [-] ndiddy|1 year ago|reply
[+] [-] TMWNN|1 year ago|reply
As an Ars commenter observed <https://arstechnica.com/space/2024/08/nasa-acknowledges-it-c...>, it is possible that the real issue isn't whether Starliner is safe to return with humans. If that were the question two months of debate are, as you said, by itself enough to say "no". Return Wilmore and Williams on Crew Dragon. Done.
The commenter posited that the real issue is that NASA does not trust Boeing's software to undock Starliner autonomously. We know that Wilmore had to take manual control on the way up because of the thruster issues. NASA may fear that if thrusters fail again, Starliner software may again not be able to handle them, and the spacecraft might ram ISS. Thus, the agency wants a human to be able to take over if necessary. *That* is the dilemma. This is something that I and others had mentioned over the past couple of weeks, but the Ars commenter is I think the first outside NASA to put it so starkly.
[+] [-] icegreentea2|1 year ago|reply
In parallel, my pet theory is that NASA has probably already made up its mind (astronauts are not going to return on Starliner), and have been dragging this out mostly to make it look like they aren't just going to throw their contractors under the bus (even if they deserve it). Boeing has declared cold feet over fixed cost contracts (in general, not just with NASA), and I think NASA wants to keep the rest of the contractor pool at least at ease that, okay maybe NASA might start being stingier with the money and contracts, but they aren't going just throw you under the bus when issues appear.
[+] [-] bigln|1 year ago|reply
[+] [-] dotnet00|1 year ago|reply
They have weekly status update conferences, and just cancelling those conferences might be more of a PR risk than just keeping the conference and announcing that they're delaying making a decision.
[+] [-] omoikane|1 year ago|reply
It's like if I have a service outage, maybe I might spend a few minutes to collect debugging information, but my priority would be to bring the service back up via rollbacks or whatever to restore a previously known good state. Currently they are debugging Starliner with people stranded, but maybe they should prioritize on getting those people back home first.
Or maybe everyone involved don't consider being stranded for months in space as a bad state.
[+] [-] TheCondor|1 year ago|reply
If the system is so complex that an extra week does yield some major new insights, that’s way too complex to use.
[+] [-] krisoft|1 year ago|reply
[+] [-] baggachipz|1 year ago|reply
[+] [-] alsodumb|1 year ago|reply
[+] [-] the_real_cher|1 year ago|reply
Theyre going to kill people at some point.
[+] [-] sitkack|1 year ago|reply
Use Dragon, Starliner can be a test.
[+] [-] gangorgasm|1 year ago|reply
[+] [-] mrcwinn|1 year ago|reply
[+] [-] farceSpherule|1 year ago|reply
[+] [-] andrewstuart|1 year ago|reply
[+] [-] firesteelrain|1 year ago|reply
[+] [-] m3kw9|1 year ago|reply
[+] [-] amelius|1 year ago|reply
[+] [-] xyst|1 year ago|reply
[+] [-] eagerpace|1 year ago|reply
[+] [-] double0jimb0|1 year ago|reply
They still don’t know root cause(s). That’s real bad Frank.
[+] [-] bamboozled|1 year ago|reply
[+] [-] don-code|1 year ago|reply