[dave_universetf]

BGP is the canonical way to take your entire global network offline within seconds. Glancing at BGP looking glasses, Starlink's prefixes seem to still be announced, but there could still be an accidental blackhole or routing loop within their AS, or something broken in one of their transit providers.

No idea if that's what's going on, but routing protocols are one of a few effectively global control planes that can go wrong very quickly like this.

[indigodaddy]

If it were BGP/routing, you would think we'd be able to still get a signal and the modem would think it's healthy (although maybe not if the issue prevented us from obtaining our public IP), we just wouldn't be able to route to any dst. In the current case we don't have a signal (orange light on the modem)

[ta1243]

Yes, before the drop out my traffic was coming from a downlink station in Bulgaria, on an IP on AS14593

Traceroute from "the internet" back to that IP reaches AS14593 just fine, and my endpoint doesn't get beyond the first hop of the local starlink router.

Whatever it is, it doesn't look like a peering problem

[dave_universetf]

Does the orange light specifically mean no RF link at all? Or does it include anything that prevents the modem from getting an IP address and route configuration? If the latter, BGP could still be at fault if it took out access to the control planes on the ground. But again all just guessing, from the outside all I see is the BGP routes are still being announced, and everyone seems to be seeing 100% packet loss and zero traffic.

[FollowingTheDao]

That sounds bad…

[oceanplexian]

A network of satellites gives you entirely new and exciting ways of taking your network offline, such as bricking them with a firmware update and no way to actually get up there and fix it.

[halfmatthalfcat]

Imagine being that dev who pushed the bad patch - Crowdstrike but x100

[axus]

Elon saw yesterday's article about The Promised LAN, and said "What if we connected that instead of the Internet...?" https://news.ycombinator.com/item?id=44661682#44663409

[dmonitor]

Instead, it seems he's created the LAN of the lost

[chasd00]

it's pretty amazing the amount of damage a BGP oopsie can do. Also, you can fit pretty much all the BGP admins for the entire Internet in one large room.

[hdgvhicv]

If by “room” you mean “Wembley stadium”, maybe.

[airspresso]

Messing up with BGP makes you feel alive, that’s for sure.

But hey, if you haven’t caused an incident yet, that just means you’re still in onboarding. Those SLA downtime budgets are there to be spent.

[hedora]

I rebooted my terminal and I can’t tell for sure if it sees any satellites. It looks like it doesn't.

It says it didn’t, and it says the “which way is down” thing hasn’t converged. Occasionally, the signal to noise ratio light in the app goes gray which means < 3.

It also rebooted itself.

Before the first reboot, 30% of pings went through. It’s almost like the azimuth or some other timely but cached data was corrupted.

[chrisstanchak]

It's always a bad route that was introduced during a planned upgrade

[airspresso]

I thought it was always DNS