top | item 43934313

The dark side of account bans

127 points| ponco | 9 months ago |madelinemiller.dev

113 comments

order

chrisldgk|9 months ago

I have a similar, though less horrible story. We recently launched an app where we wanted to have a Facebook sign-in button since our target audience would be people that use Facebook a lot. Being Gen Z and not having a Facebook account at all (anymore) I made one for Meta developers, set everything up for development and tested stuff for a few weeks. When we finally did want to go live, we went through the Meta for Business verification process, which succeeded, but a minute later my account got banned. I got to appeal (sending a video of your face looking multiple directions), but for some reason that wasn’t enough for them. I got fully banned with the only option to appeal going to court. So now apparently I have a Facebook developer account and a verified business, both of which I cannot access at all anymore.

We do not currently have a Facebook sign-in button in our production app.

hofrogs|9 months ago

You now have a banned account, and Meta has a video of your face looking multiple directions and data about your business and development activity. A fair and common trade from the big tech point of view.

distalx|9 months ago

Yeah, I had a similar frustrating time. Not a personal ban, but trying to help a friend verify their business for the WhatsApp API.

We followed the process twice, really carefully. Both times Facebook failed to verify the business. No clear reason why.

We posted everything on their developer forum hoping for help, but got nothing back. Just silence!! Many people were facing the same issue like us.

Ended up having to give up and tell my friend I couldn't help. Honestly, the fear of getting banned mysel if I kept pushing stopped me from trying it one more time.

these platforms are difficult to deal with when things go wrong.

yellow_lead|9 months ago

I've reported a lot of prostitution ads on Instagram, but they always responded to the report with "we found nothing wrong". After reporting these ads, Meta decided I liked them and started showing me more and more, but I've given up on reporting them, since no action is taken.

Meta seem to not actually check reports. My theory is that they just rely on some percentage of reports maybe relating to followers, reports, and time. If it exceeds a threshold, there's an action. Otherwise, no action.

dsign|9 months ago

I believe that part of the problem is how trigger happy the platforms are to reports of CSAM and other "black issues". They don't do any research, they just ban and call it a day. Your government could go to them and say, "hey, you should do due research before banning a user,", and the platform will turn back and say "you passed a law last month where I would be banned from operating in the country if I allowed any photo of somebody under 18 wearing less than a full-body skimo wolly suit, but I'm not allowed to ask for an ID to know the user's age. What am I supposed to do? Deploy a trigger happy AI? I just did!"

So, in a sense, we live in the best world possible.

May I suggest that you just block and mute the prostitutes, if they offend your sensibilities? That's what I do. Because in the next iteration of censorship, we might lose the social networks altogether.

lynx97|9 months ago

I suspect not reacting to that kind of problem is intentional. Just like YouTube didn't do anything against the pirated music uploads in the 2010 years, simply because being able to play your favourite pop song via YouTube was attractive to early users. Same with audio books these days. Some pirated accounts are being used to upload complete Audible books. They dont even bother to remove the "This isAudible" intro, which apparently could easily be used to autodetect such uploads. As said, I am guessing porn ads do provide Meta some form of income, and thats why they don't do anything about it. Big tech is shady like f**.

immibis|9 months ago

My Occam's razor is they do whatever makes them the most money. Are the prostitutes paying lots for ads? Leave them up. Are you complaining a bunch and not paying for ads? Ban you. Are people complaining about you a bunch (including 1000 identical complaints from sockpuppets) and you're not paying for ads? Ban you.

voidUpdate|9 months ago

Same reason twitch don't ban big accounts that break ToS, only the small ones. The big accounts make them money so they don't want to ban them. I recommend watching an Amouranth stream and keeping the ToS open in another window, and counting the violations.

graemep|9 months ago

The only time I reported anything to Meta, I reported a blatantly racist comment to FB ("do not interbreed with them because they are evil") and the assessment was that it did not breach their community standards.

db48x|9 months ago

> She recommended a few cafes, and I went to look at their menus. To my surprise, I couldn’t access their menus. They were only on Instagram, and it’d force a sign in before showing me the content.

Wow. There are over 30 restaurants within a mile of where I live, and not one of them uses Instagram to host anything as far as I know. They always have a real web page, even if it’s cobbled together on Wix or some other horrible thing, and the menu is invariably a PDF (because they had to send something to the printer, and they don’t know how to put something nicer on the web). Sometimes you have to allow third–party JS to get their webpage to render so that you can get the PDF though.

Well, I guess the national chains don’t post a PDF these days. They give you a real menu and even offer online ordering and delivery. Most people probably don’t need to look at a McDonald’s menu to know what they sell though.

reginald78|9 months ago

The other day I went to view my kid's elementary school class page and was hit with an instagram login prompt.

AStonesThrow|9 months ago

What I've found very useful is Google Maps for ordering online. I typically use DoorDash, which can access and aggregate almost any restaurant I'd care to patronize, but if the restaurant isn't listed, Google Maps can accurately direct me not only to the restaurant's main website, but also to their menu, and a complete list of online ordering services.

I've been able to reliably target a restaurant 20 minutes away by bus, select my order, pay for it online, and have it ready for pickup by the time I disembark. It's a pretty sweet arrangement.

If your chosen restaurant directs you to some dumb QR code and Instagram, try just looking them up on Google Maps. You'd be surprised what resources are available there, including plenty of reviews and customer-submitted photos of the very dishes you're looking to order!

OsrsNeedsf2P|9 months ago

I doubt they'll stay with custom websites for long, given 1. The new generation grew up with Social Media as a native service and 2. How much easier it is to run a Business account vs your own website

benhurmarcel|9 months ago

I’ve been to some restaurants that have a QR code to their own website but I get blocked by their Cloudflare settings for some reason. So even a proper website doesn’t avoid issues.

me4502|9 months ago

Honestly hearing how different this is for Americans makes me wonder if it’s a self-fulfilling prophecy type situation. Everyone in Australia is already using Meta services, so businesses just go where the customers are and keep putting everything on Meta services. Less about Instagram being a good solution, and more just being good marketing when it’s where your demographic is.

dangus|9 months ago

This unfortunately is why anonymity is so important on the Internet. I would not want to be a part of many of these communities with an account that is connected to my real life accounts in any way. Different email, different username, no discernible connection to my real identity.

But also, shame on Meta and most other gigantic companies for having no due process for anything like this. I have no idea how they just go along with it and decide in favor of the reporter with zero evidence. Sure, maybe bans need to be proactive but there should at least be an appeals process.

There should probably be more laws that cover permanent bans for business platforms that have a marketshare above a certain size. You shouldn’t be able to be permanently banned from services like this that dominate their respective markets without a robust process. There should probably be cases where even the violation of certain aspects of ToS can’t get you banned permanently, where law overrides ToS. Imagine if you were banned from getting phone service from AT&T in the 1980s and how devastating that could be.

jjav|9 months ago

While I sbsolutely agree there needs to be laws preventing account bans for any website larger than some threshold, this is also a reminder why it is so incredibly vital to support open standards that are not owned by anyone. Like email, IRC.

Every walled garden with a single corporate owner will 100% guaranteed abuse that ownership power.

The only way forward is open protocols that interoperate without anyone having a controlling interest.

chrismorgan|9 months ago

> This unfortunately is why anonymity is so important on the Internet.

I’m not convinced by your argument. I acknowledge that maintaining an isolated identity for the Discord/whatever accounts would have limited the scope of the damage, but it’s often not very practical for a variety of reasons, and perhaps more importantly, the anonymity is part of the problem too: the trouble-maker has anonymity, and uses it for bad.

Under the Law of Moses, a discovered false witness in court was to receive the punishment he tried to cause, explicitly as an enduring deterrent to dishonesty. Deuteronomy 19:16–21:

> If an unrighteous witness rises up against any man to testify against him of wrongdoing, then both the men, between whom the controversy is, shall stand before the LORD, before the priests and the judges who shall be in those days; and the judges shall make diligent inquisition; and behold, if the witness is a false witness, and has testified falsely against his brother, then you shall do to him as he had thought to do to his brother. So you shall remove the evil from amongst you. Those who remain shall hear, and fear, and will never again commit any such evil amongst you. Your eyes shall not pity: life for life, eye for eye, tooth for tooth, hand for hand, foot for foot.

I presume Meta knows who made the report. I wonder if they ever make diligent inquisition, and ban the troll. I doubt it, but suppose, just suppose they did apply a standard like this.

Imagine if the account-reporting procedure stated a clear policy: “if we find your report vexatious, we will ban your account instead”; or “if we find your report vexatious, we will tell your target your account details so they can pursue legal action if they desire”. (This is, of course, overly simplified, and would probably deter legitimate reports too.)

Such harassment and threats as these frequently break laws, but anonymity is one factor that makes it much harder to pursue in a legal system. (Though it’s hardly the only thing; jurisdiction problems are a rather big deal with online stuff, and legal systems are often not tuned for pettier squabbles.)

Returning to the original case: if there were no anonymity at all, and the guy had to threaten via his real identity, I doubt it would have happened, and remedy might have been easier if it did.

I’m not against anonymity, I just feel the total picture is more nuanced than you’re presenting it as. Anonymity has both advantages and disadvantages.

antonvs|9 months ago

> In Australia, a vast majority of people purely use Facebook Messenger as the way to keep in touch with people.

My condolences.

philistine|9 months ago

Canada's the same.

maxehmookau|9 months ago

This author sticking around is far more than that community deserves. If I ever received that sort of message from a user of anything I'd built, I'd walk away forever.

fmajid|9 months ago

And she is aa woman, which means she probably gets 100x the abuse a man would from entitled gamergate incels.

immibis|9 months ago

What if that was actually what the troll wanted?

j1elo|9 months ago

> I couldn’t access their menus. They were only on Instagram, and it’d force a sign in before showing me the content. > so many restaurants and cafes here purely use Instagram (or sometimes Facebook) to host their menus.

That's idiotic. Really, reaaaally idiotic. Like, "talk to the manager and tell them they are doing a very stupid thing" levels of idiotic. </Rant>

This "we're too modern and digital to get out heads out of our asses" attitude towards adopting "cool technology" with complete disregard for an analog Plan B gets to my nerves.

Here, since Covid, lots of places stopped offering physical menus, instead they put a QR code to some webpage or gigantic PDF file that contains the menu (and can be updated every day to push prices up if needed, heh).

I hate that services assume you must have a hundreds-dollar device on your pocket at all times to even be able to access the basic service they provide. I just power off my phone and tell them "look I ran out of battery, what should we do?". They usually do have a physical menu, thankfully.

But other things aren't so nice. My employer offers a restaurant card, and because now we live in a "digital era", they don't issue physical cards any more, only digital ones to be used with Google Wallet on Android. Turns out I'd like to install GrapheneOS on my Pixel, and you got it: Wallet doesn't work on GrapheneOS for payments! I'd want my physical restaurant card, please.

molochai|9 months ago

Went to a Starbucks the other week, had a gift card to blow... looked pretty dim and potentially like they were closing shop but I could still see people. No hours posted on the door, "scan this QR code to get the hours." I opted to tug on the door handle to figure out that they were, in fact, closed.

I get why that is useful for them (looks cool, no having to reprint the physical hours stickers, less ugly than a paper version I guess) but the assumption that it is easier for me to pull out my phone and scan the code versus just... trying the door... whew.

Reminds me of working in restaurants ages ago. Credit card swiper down? Cool, pull out the "knuckle buster" for your "analog Plan B" -- all's well, though the manager's probably pissed at staying late to manually enter the transactions. We probably could not even do that anymore because many cards have gone to non-imprintable numbers.

kassner|9 months ago

I really don’t understand why some societies rely so much on Meta services, specially considering they are SO MUCH WORSE to administer, specially for a non-tech user, compared to creating a website in Wix (or even a free Wordpress.com account), or maybe updating your Google Maps business with the right information.

Maybe that shows a lack of critical thinking in the general population? I.e.: people must be using FB/IG because it’s the only thing they know, and never questioned themselves if there are other avenues?

ajross|9 months ago

> This "we're too modern and digital to get out heads out of our asses" attitude towards adopting "cool technology" with complete disregard for an analog Plan B gets to my nerves.

All restaurants have an "analog plan B", it's called walking in and asking for a menu. These are tiny businesses run by people who love food, not tech. It really is asking too much to expect them to work at and understand the ideas behind digital freedom, at least if you want them to serve you good food.

Notably Google Maps curates a photo set of the physical menu of basically every restaurant in the urban USA, if not everywhere. I find a lot of people don't actually know this. I never bother looking at restaurant sites anymore except for places where I know they're likely to have changed the menu (and even then it's a crapshoot, per your original point).

kranke155|9 months ago

This is literally the new normal in a lot of places in Europe. I have no idea where you are that you think you could complain about this, they would just eye roll at you.

stevage|9 months ago

I also lost my Facebook account a few years ago, because someone tried to hack it I think.

Similarly there were no open avenues of appeal.

By a big fluke I actually did some work for FB and in the process they unlocked it.

But yeah, monopolies controlling access to public services with no oversight sucks.

wil421|9 months ago

My buddy forgot his old Facebook but some scammer found his old password on one of those pwned websites. A year ago his old account started selling random things like a huge swing set in California. I always wondered what they were up to with his account of Facebook marketplace.

dredmorbius|9 months ago

Sue in small claims court. This exists in Australia:

<https://www.fcfcoa.gov.au/gfl/fairwork-small-claims>

The law is what provides justice, the mechanism is the court system. There are many who try to dissuade you from this option because it is effective. Use it.

Write your consumer protection entity. In much of the US that is your state attorney general's office. I believe you can start with the ACCC (Australian Competition and Consumer Commission), here:

<https://www.accc.gov.au/consumers/problem-with-a-product-or-...>

Write your government representatives.

- City council members.

- County supervisors / local councils.

- State representatives / MRA.

- Federal representatives / MPs.

Call their office(s).

Write their office(s). Email and postal mail.

Follow up with them. If you don't get a sufficient response, let the office know.

Australian governmental organisation: <https://peo.gov.au/understand-our-parliament/how-parliament-...>

Contact NGOs advancing personal digital rights in your country. For Australia this seems to be Digital Rights Watch (<https://digitalrightswatch.org.au/>) and EFF Australia (<https://efa.org.au/>). These and similar organisations elsewhere should already be very familiar with account-lock-out and similar problems, and can likely direct you as to how to register your claim most effectively.

Tweeting, blogging, and throwing up your hands is what Meta would most like you to do.

Do more and cause them pain.

Organise.

fweimer|9 months ago

> I did end up speaking to a few engineers at Meta through connections, either directly or through others, many of which recommended suing Meta to unlock my accounts. This was extra complicated due to me living in Australia, so it’s not something I tried.

Why is this complicated? Do the Meta ToS include a choice-of-venue clause for consumer accounts?

decimalenough|9 months ago

Suing people across jurisdictions is a nightmare. As a trivial example, anybody can theoretically sue any Singaporean entity in Singapore's Small Claims Tribunal, but if you don't have Singaporean ID, you need to show up in person in Singapore to register for the CJTS Pass system. Then you need to send registered mail from Singapore to the person you're using, apply for a special exception to be able to appear over Zoom instead of in person, provide proof that your home jurisdiction permits you to testify over video in Singapore, etc etc. And this is just for small claims!

Rebuff5007|9 months ago

There is so much that is sad and infuriating about this.

How can a stranger have so much hate towards someone who's content they want to consume? Why are meta engineers suggesting suing their own company instead of pointing to someone to walk through a basic appeals process? The obvious, why do so many things in the world not work without a smartphone and a google/facebook account with internet access? Why isn't anyone doing anything about any of this? PMs at facebook? Congressmen/women?

The system is not working.

nikanj|9 months ago

Accountability sinks are a huge issue with large corporations. The people you can reach are not allowed to act due to policies and processes, and the people setting the policies are not reachable to a regular customer - nor would they change the policies just to accommodate one out of their millions of users

https://aworkinglibrary.com/writing/accountability-sinks

thrance|9 months ago

I really wish Europe would consider a continent-wide ban on Meta and X. We gain nothing positive from these companies, they only serve to destabilize our democracies, disinfom citizens and ostracize people.

reconnecting|9 months ago

I'm just curious how many votes your comment will get.

n4r9|9 months ago

Sounds like an awful experience. It's easy to create fake FB or Insta accounts to get around the restaurant menus problem, but getting WhatsApp banned must be a nightmare. I didn't even know that could happen!

reginald78|9 months ago

Is it easy? Last time I tried to create a throwaway account it demanded a video of my head. This was after automatically banning on creation my account.

cube00|9 months ago

Good luck getting past the sign up flow to even get an account if your fingerprinted device is on their ban list.

ValdikSS|9 months ago

I don't know about the country you live in, but the usual 100% bullet-proof method to make your message seen by the living human being is to send a paper snail mail.

You won't necessary receive the answers for your questions, but you'll get back your account.

If the simple message did not help, you file a pre-trial claim and send it to them.

subscribed|9 months ago

The country she lives in is plastered all over the article you didn't read :)

nickdothutton|9 months ago

Users reporting users is a terrible way to manage account misuse/bad behaviour. Your average user on your average platform is a low form of life incase you hadn’t noticed.

Rastonbury|9 months ago

Worse is if your competitors report you and get you banned while having profitable ads, there goes a large marketing channel and likely a chunk of revenue, think shopify stores, restauranteurs. Reddit is full of these stories

grishka|9 months ago

It's an okay way as long as the ultimate decisions are made by real humans.

hnthrowaway4166|9 months ago

LinkedIn is especially egregrious in the account restriction aspect, especially with how essential it is to many people's lives.

Recently, I found that I couldn't log in to my LinkedIn account. They made the issue seem like an error, so I spent a lot of time trying to log in on different device/browser combinations. Eventually, I folded and tried to submit a support ticket, just to be met with a login page.

After some searching around, I found direct links to the support forms, but none of them worked — they all displayed the same error when I tried to submit a ticket. The only other option was to contact them on Twitter. Tried to create an account just to be met with a generic "something went wrong" error, after wasting my time with what was easily the most awful CAPTCHA I've ever seen.

Tried creating a new LinkedIn account — no good, they ask for phone verification and won't let me reuse my phone number.

For some time, I thought my only option was going to be legal action, until a lightbulb lit up in my head. I went back to the support form, and instead of providing the email I used for my LinkedIn account, I used another email. The ticket was submitted successfully.

At that point, it was clear that my account had some shadow ban in place. My only guess is that I tripped some psychotic automated system. How is it in any way acceptable that a company can essentially, and completely arbitrarily, blacklist someone from the job market like this? No notification or warning, nothing — they literally try to gaslight you with fake errors.

I'm hoping that my support ticket will have some effect (not sure it will, given what I've experienced so far), or else I'll once again be left with legal action as my only recourse. So far, nothing.

Marsymars|9 months ago

I also recently tripped something at LinkedIn and now it asks for me to scan a QR code with a mobile device in order to upload ID to prove the identity of my two decade-old account that has a secure password + 2FA.

Given that I don’t seriously use LinkedIn (I just use it to scope out other people), I’ve just permanently stopped using it at all.

teddyh|9 months ago

> This experience was eye opening; it shocked me at how fundamental online platforms owned by companies have actually become to modern life.

Oh no! If only there was some way you could have known!

<https://xkcd.com/743/>

antonvs|9 months ago

I had a reddit account permanently banned for saying something to the effect that a certain out of control billionaire could be dealt with by launching him into space in an electric vehicle that one of his companies manufactures.

Apparently "reddit legal" thought that was a credible threat. I find that strangely encouraging.

verzali|9 months ago

Reddit once banned my account for "report abuse". The content I had reported was then removed by them for being against their rules. No amount of appeals could get a response from them.

lynx97|9 months ago

Reads like a good decision on the side of Reddit. If you can't see how this type of statement is not OK, even after the fact, shows banning you was the right decision.

immibis|9 months ago

Reddit's been way out of control on this and I'm surprised people still use it at this point. Though the vast majority of Reddit content is posted by bots to manipulate users - so maybe they don't. Reddit is a Dead Internet existing here and now in front of our eyes.

In addition to platform-wide bans most popular subreddits also use subreddit bans to create false impressions of consensus.

Last scandal I heard was they banned people who upvoted memes about Mario's green brother.

dredds|9 months ago

We're at the point that MAMAA/FAANG (might as well call them MAGA now) can dictate 0% tax in the UK instead of a measly 2% because those platforms were co-opted by foreign countries to promote Brexit, leading to economic servitude. Starlink is now being foisted upon economically susceptible countries leading to a potential monopoly of connectivity, giving ultimate power to broker even more concessions down the line, not to mention increasing political thought-control.

Welcome to 2025's take on 1984.

louthy|9 months ago

> can dictate 0% tax in the UK instead of a measly 2%

This hasn't happened (yet, at least).