Logging Everyone Out

[s_dev]

This seems all very reasonable. Look forward to the post mortem.

Given their size on both the web in terms of employees this unusual for Wikimedia. They typically fly under the radar. How many times has Wikipedia ever been down?

I recall AWS, Google, Microsoft having more outages -- mind they probably are considerably bigger but still they're doing something right.

[jshen]

it's not hard to have high uptime for content that is largely static. Has less to do with size than static versus dynamic, and accurate (think bank transactions) vs fuzzy (search results)

edit: fixed a typo

[dheera]

There are enough copies of Wikipedia that if it has ever been down you could just get the same content elsewhere, so people don't usually make the same fuss about it that they would about AWS/Google/Microsoft.

AWS/Google being down for even a minute or two is a big deal though.

[fireattack]

What do you mean by "Given their size on both the web in terms of employees this unusual for Wikimedia"?

(I knew this likely just a typo, but I genuinely didn't figure out what you meant.)

[mmaunder]

I'm reminded that MediaWiki runs on PHP, which isn't much loved here on HN, but does power some of the busiest sites in the world, and PHP 7 and 8 have done a great job of moving the language and performance forward.

[nurettin]

It needs to scale reads using in-memory caches. After that, all it needs to do is to be able to handle 10-20 posts a second and you're golden.

[ibejoeb]

It's been quite a while since I've looked at mediawiki. Has any of the codebase been transition to modern PHP?

[andrewla]

I remember early in the history of E*Trade, I went to my account, and it showed someone else's name and account information. Didn't even bother reporting it (I was just a kid); just logged off and on again and withdrew everything from my account and never looked back.

[harryf]

I know the full analysis isn't online but I have a problem with this part...

> This was done out of an abundance of caution, after we received one (1) user report of being logged in as someone else.

This _seems_ like a knee-jerk reaction to one data point.

There could be other causes for a user to report that, like a change to the cache key used for serving a users profile giving the _appearance_ that you're logged in as someone else, even though you're not really.

Forcing everyone to re-login could potentially make the system worse, in that you're now overloading parts of the system that has to handle those logins, plus causing all kinds of cache expiry...

I guess there's more to the story and someone who knows the system deeply knew this was the right choice but just reading the reports it seems knee-jerkish.

[jeremyjh]

It seems adequately explained in the next sentence.

> Said report coincided with the deployment of a new MediaWiki release which caused other problems around User session objects;

[usmannk]

Bugtracker: https://phabricator.wikimedia.org/T264370

[8ytecoder]

I have seen this behaviour on Facebook, ages ago. Reported it and yet nothing came out of it.

[CGamesPlay]

The "view profile as another user" feature was infamous for being... really thorough. At one point you would receive chat messages directed to the person you were viewing the profile as (I believe this was before the integration of chat and messages).

[jayar95]

That doesnt surprise me. They will let highly problematic security issues persist for weeks

[jacobwilliamroy]

My buddy told me that the U.S. marines in afghanistan once misplaced one of their fancy encrypted radios (or it might have been one of the keys to their fancy encrypted radio) so all radio communications were suspended until the next scheduled rotation.

[hexbinencoded]

Battery ran out? Left one behind? Throw them all away and visit the quartermasters' Idiocracy/MIC version of Costco to pull another 100 pack off the shelf at a cost to the taxpayers of mere millions. Oh and F-35B's and Blackhawk turbines? just chuck those like Kleenex. My mom worked in civil service for the Navy and $20k parts in the '70's went "missing" and "lost" all of the time. Then there's a USAF Major I know who became unpopular for suggesting proper maintenance rather than ruining/replacing turbines might be a better approach (likely some sort of maintenance/HQ personnel-vendor kickbacks, MIC FTW).

One of the few stories VICE didn't botch entirely: https://youtu.be/_CvWJVtEkUE

[tomc1985]

Assuming session ID's are tied to individual users and looked up via a "SELECT * from sessions where id='?'" type query, how does this even happen?

[kace91]

Badly designed caches for example. Though I have no idea what happened in this particular case.

[trynewideas]

None acknowledged publicly, but it's an open-source project with pretty open administration, so you can follow along: https://phabricator.wikimedia.org/T264370

[TheRealPomax]

Remember https://www.theregister.com/2015/12/30/steam_security_blip_e... ?

[duderific]

Not scoping a variable properly (causing a variable to leak out to global scope) could also cause this.

[Kiro]

This reminds of a thing that happened on MSN back in the day when I suddenly started to receive random users' chat messages in one of my chats. It only lasted a few minutes and it was all different kind of languages. Never heard anything about it but in retrospect it felt very serious. Imagine the same thing happening on Facebook Messenger or Whatsapp.

[donatj]

I believe the Steam Web Store had a similar issue of people being logged in as the wrong user a couple years back.

[cheeze]

This is a very common 'advanced failure scenario.' I've seen it on a handful of sites, session objects and caching are difficult and sometimes overlooked during migrations.

[gizmo385]

That was a caching issue if I remember correctly, as opposed to users be "logged in" as a different user.

[polygot]

I had a similar issue happen to myself (not on wikimedia or anything related to wikipedia.) I clicked on "login" by accident without filling in my credentials and I was logged in as either an admin user or a user called "Adam".

[andrewmcwatters]

I’ve always wondered if this class of cache issue resulted primarily from collisions of some sort. I’ve only seen it a handful of times over many years. Others here have mentioned it with services using Varnish, for instance.

[tomcatfish]

Wow, this is the first time I've seen a company pick "better safe than sorry" when it had global effects (though this isn't a large hit, it's everywhere).

I appreciate it a lot, thanks WikiMedia

[flemhans]

I remember back in the days, I disconnected my dial-up modem and re-connected with another public IP. Upon refreshing Hotmail, I was presented with another user's mailbox.

I was never able to reproduce the glitch.

[m-p-3]

So that's why my account was logged out from the mobile app. No big deal, as long as the security of the system remains intact.

[punnerud]

Love MediaWiki as a platform. I host several of them. Hope the interactive editor in PHP soon get out of beta.

[prionassembly]

Do they mean employees? My Wikipedia login lasts for 30 days, I think. I basically only login to edit articles that require so. Otherwise I just edit as anonymous.

[dionian]

What is the (he/him) after the guy's name?

[Raphmedia]

Pronouns.

That way, if someone is named "Billy" or "Alex", you won't be spending years mistakenly assuming they were a man while they could have been a woman.

This also has the beneficial side effect of covering trans individuals who use a birth name while having another gender identity.

Even more useful for usernames such as yours, "dionian". I do not know if it is your first name, last name or made-up username. So I have no clue what your gender is and I'd default to "they". If I knew, I could call you by the right pronouns.

It's just a slow shift away from assuming everyone on the internet is a man.

[Amorymeltzer]

Preferred gender pronoun:

>Preferred gender pronouns or personal gender pronouns (often abbreviated as PGP) refer to the set of third-person pronouns that an individual prefers that others use in order to identify that person's gender (or lack thereof). In English, when declaring one's preferred pronouns, a person will often state the subject and object pronouns along with the possessive adjectives—for example, "she, her, hers", "he, him, his", or "they, them, theirs"—although sometimes, only the subject and object pronouns are stated ("he, him", "she, her", "they, them").

https://en.wikipedia.org/wiki/Preferred_gender_pronoun

[function_seven]

When people list their preferred pronouns, why list them all?

Here we have "he/him". Sometimes I see "he/him/his" or "she/her/hers", "they/them/theirs", etc.

I doubt people are mixing and matching among the different types. In other words, I've never seen someone prefer "he/them/hers".

Seems like we can standardize this to just one type (e.g. the subject pronoun). A simple "he", "she", or "they" will do the trick.

Yes, I bikeshedded it. Sue me.

[simcop2387]

This person has decided to provide the pronouns he would like to be used when referred to

[DoreenMichele]

I have begun putting she/her in some of my profiles (twitter, reddit) because I still get mistaken for a man at times. To be fair, my middle name -- Michele -- can be a male name in some places. It's the Italian version of Michael, IIRC.

It is my personal policy to not correct people who misgender me in most cases, especially if that is the only thing I would be saying (I will sometimes clarify if it is part of a larger comment, but I try to be gentle about it). I would rather put that info somewhere and let them have the chance to learn of it without me having to correct someone.

Different people have different reasons for noting their pronouns. Some do it because it is trendy. Some do it because they have genuinely been misidentified in online spaces. Some do it to show themselves as allies to certain groups.

Without him saying why he did it, no one here can genuinely tell you why he chose to do so.

[dEnigma]

His preferred pronouns, I would assume.

[DanBC]

Chris is a name used for men and women. He's letting you know he identifies as male.

[narenkeshav]

I've been seeing this trend quite often these days.