Option d: nobody outside of the Twitter TnS teams (including Paul Graham) has any idea of what Twitter is detecting as spam and what not, and thus can't actually judge how good a job is being done.
First, we don't see the spam that was outright blocked, so we have no idea of what the false negative rate is. Second, I bet that this is not a binary block/allow decision, but there are all kinds of ways of reducing the engagement that probable spam gets without outright blocking. The latter is operationally preferable since it reduces the cost of false positives and since it makes the iteration loop for the spammers a lot slower.
(But also, I can't remember when I last saw spam in my Twitter feed.)
Crazy how we can have such different experiences. I follow less than 100 accounts and easily 50% of the responses are always spam/scams. It makes Twitter almost unusable for me.
The most ironic part is that Twitter will show the spam/scams, but hide other replies, from real people, that Twitter thinks are mean.
Option E: Nobody can even agree on what Twitter spam is.
Is it bots? I follow a few bot accounts and get value out of them. Is it blatantly promotional content? This actually seems to be one of the intended uses of Twitter. Is it low-quality content? I feel like lots of people earnestly tweet out low-quality stuff, and who gets to judge what's low-quality anyway?
…which both grow with account size (& other forms of prominence, & certain topic-areas), so your anecdotal testimony that it's not in your feed doesn't do much to qualify or refute the magnitudes of others' problems.
Twitter's biggest internal spam problems are:
• Twitter's ads are repetitive, poorly-targeted crap
• Twitter does not provide reliable ways to disable their unwanted inserts – quickly ignoring any number of 'see less often' choices, and doing things like randomly reverting people from their chosen 'latest' to Twitter's algorithmic 'home' feed.
As fair definitions of 'spam' or even more generally 'harassment' include "continued unwanted interactions against expressed preferences", this means Twitter Inc is the biggest spammer/harasser on its own platform.
Here's one way they do it that isn't documented or widely known: an account can get "searchbanned". While your account is searchbanned, your new tweets can't show up in other people's search results unless they follow you. There is no indication when you've been seachbanned or when the ban is lifted, and no documentation on its existence or how to get un-searchbanned. We know because we gather community input on a specific hashtag, and have gotten complaints that specific people's contributions weren't included, because they didn't show up in the search.
Interestingly, we had one person's account whose search results still showed searchbanned tweets. They would not show for that person if they logged out. We also could not find out why that person's account in particular could see them.
The spam Twitter lets through is ridiculous. Is it hard to catch all offenders? Yes. Is it obvious Twitter is no where close to that? Also yes. We should have higher standards for our social networks.
I see lots of spammy content from startup founders, but there is clearly (or maybe bots are better than I think) a human behind it as it appropriately replies to stuff and appropriately piggy backs off trends.
I’ve known a few startups that built their initial user bases utterly deluging Reddit and Twitter in manual spam.
> has any idea of what Twitter is detecting as spam and what not, and thus can't actually judge how good a job is being done.
That's wrong. Results can be seen even if the process is not transparent. If every bridge the engineer builds collapses, I can judge him without looking into all the drawings. If every business the entrepreneur runs collapses I can make conclusions from that, even without seeing the books. If there's a lot of garbage on Twitter, I can judge it's doing a bad job even if I don't get to see its algorithms.
The experience is based on who you follow. Twitter builds a list of topics based on your activity and it never gets edited, so even if you unfollow someone, that list still influences everything you see.
Algorithm development has been poor and really frustrating to users because if they even come across someone who followed the Kardashians (for example), they literally get spammed with that news for the life of their accounts, and Twitter's "mutewords" functionality has also not worked for many years, I can't tell if that is intentionally so or not.
I get ~2 cryptocurrency spam messages per day. It's not the end of the world, but it's annoying.
Twitter could solve my spam problem tomorrow with a setting that says "don't show any posts with links if they're from accounts none of your friends or friends of friends follow".
That's a bold assumption giving Twitters track record of terribleness. I don't understand what the line of thought here is, anyway - do you want to convince people that see tons of spam on Twitter every day that they are imagining things? It's not real?
The spam doesn’t show up in your feed, it comes in the replies. If you’re a large account like pg you’re probably getting a ton of spam replies to every tweet.
Twitter's a company with such a strange relationship with the users of its own site.
I modded /r/Twitter on reddit for a year but burned myself out (I chose to de-moderate FYI) simply because Twitter doesn't care about its community or even recognizing the existence of the community that developed around trying to provide the support the company won't provide on its own. Perhaps my take is cynical, but I really do like the concepts of a social media service like Twitter. The execution of it, however?
Perhaps this is an outsider's perspective but it seems people who work for Twitter would rather pat themselves on the back rather than make improvements.
My complaint of the week - you can't say the words "hacked" and "account" without having scambots asking you to get in touch with "their friend" who will help you restore access to your account. or something. It's just a fucking scam.
Also, just look at threads in reddit flagged with the Bug Report, Complaints, or even Question flair. The users are just bewildered and the experience is 100% user-hostile:
> Twitter doesn't care about its community or even recognizing the existence of the community that developed around trying to provide the support the company won't provide on its own.
It's fascinating that such a community would even need to develop, but maybe it's because of the difference in mediums?
I'm working on an alternative and would love to chat if there's a way to connect!
..I think they intend to "fix it" by making people's interactions on Twitter more narrow.
They don't truly like the fact that Twitter is a bunch of popular people shouting into the void, they want to bring Twitter 'closer' to something like Facebook (but not exactly) where you are interacting with a closed loop of friends in order to engage more every-day (read: less popular) people.
Anyone who has created an account on Twitter in recent years knows that new accounts are indiscriminately opted into new mandatory 'intelligent' social features like the purple star recommendations[0], while anyone with an old account will never see such a thing.
They're keeping their old user base, doing nothing to improve spam for them, while driving new users to a more closed-loop friend system where spam doesn't matter.
> Anyone who has created an account on Twitter in recent years knows that new accounts are indiscriminately opted into new mandatory 'intelligent' social features like the purple star recommendations[0], while anyone with an old account will never see such a thing.
Hmm I have an old account (Sep 2007) that sees purple star recommendations…
What I don't understand is that Twitter makes it so difficult for new humans to sign up for an account, how are the robots getting around it?
I wanted to create a second Twitter account, but almost immediately after signing up it was blocked for "suspicious activity". I hadn't even posted yet. The only way to resolve it was to add a phone number to the account, but I only have one and it was already assigned to my first Twitter account.
So I was stuck.
How on earth do the spam robots do it? There are so many.
Someone I know well was hired at Twitter more than a decade ago as a senior software engineer. He had experience with natural language processing, and was given a project to identify spam and bot accounts.
He worked on this for a while, all the data he needed was made available and he analyzed every account on Twitter. His analysis said one third of all accounts were bots.
He presented these results to management, who said the number must not be that high, and discussed what it would mean for their MAUs or whatever metrics if these accounts were removed.
None of the identified accounts were deleted. Instead, the project to identify them was canceled, and the engineer quit.
You can block words, such as NFT or crypto, by going to your user settings. I would recommend everybody to use this feature a lot to avoid stress and content that makes the angry, it’s quite helpful.
Anytime I see the comments on posts about social media moderation, I laugh my ass off.
People calling it a “hard” problem simply parrot the narrative they see other engineers who have tackled it in leadership positions adopt. The reality is it’s a “Dirty” problem. The equivalent of a sewer cleaner in the old days that is a thankless, low pay, low career prospect, riddled-with-politics , nightmare.
Look up the salary bands for roles in spam fighting. The top 5% probably make bank and even that is not wildly large amounts relative to the valley (~$650k TC which is director level)
The LARGE pools of spending go toward maintaining an army of contractors that get passed down a barrage of things that some teams flag through automation. Same at twitter as it is as Facebook and Google. You need only hit up LinkedIn for the right search keywords to find the contractor hotspots.
The point being, Paul is right. If these platforms wanted to solve the problem, the smartest hires would go there motivated by a culture enabling high impact and good compensation.
The hell hole that is moderation operations would make even those guys with the Six-Sigma-black-belt-world-championship-something ops degrees shed tears for how soul destroying the environment is to work in.
Source: Have friends in these roles. Seen it play out a bit more than a decade at this stage.
What is universally or objectively "offensive"? Is a young attractive woman who is only following me to try to get me to follow her OnlyFans, "offensive"? Is a die-hard materialist atheist, or a strident born-again Christian "protected by the vaccine of God", "offensive"? Is plain nudity "offensive"? Violent photos or movies? Vanilla sex? Hardcore sex? BDSM sex? Bad words? (Bad word filters are easily defeated with creative misspellings or Unicode.)
Is being mean "offensive"? How would you detect that well? The more intelligent the meanness, the harder it would probably be to detect...
Are certain thoughts or concepts offensive? Are they ALWAYS offensive (across both time and location on the Earth), or only for the time being, or only for a place?
Is the word "Jews" offensive? Always, or depending on context or who is saying it?
Is it possible to be satirically offensive in a way that an algorithm would have difficulty detecting? What if I quoted something offensive to argue against it? (I literally got a tempban for this once.)
Perhaps they are "incredibly bad" at it because it is an algorithmically impossible problem that is deeply tied into the subjective sensibilities and tastes of a perceiving consciousness at one point in time (or set of consciousnesses, all of which perhaps only accidentally happen to coincide)
in the thread, someone links PG's filtering method, which claims 99.5% effectiveness? that would still be getting me 20+ spam emails a day at that rate
further, if you are trying to block, lets say, crypto scams, on a platform which allows strong positive discussion about crypto, which allows people to talk about stuff they are selling for crypto, etc etc etc, you easily start losing points to differentiate
the defender needs to classify every message on the site, in a way that allows detecting spam well after classification, while maintaining over a 99.8% rate these days, while aiming for a 0% false positive rate
the attacker just needs to type random messages at their keyboard and use reused passwords / buy client id/secrets from shitty devs, to get access to verified accounts
When you provide a service to more than a handful of people, there is no winning in any of the decisions you make. There will always be groups of people who will be angry at your actions/inactions.
I've recently seen complaints that gmail has gotten worse at detecting spam, and I've personally seen YouTube's comments are filling with spam. The spammers have probably just gotten better at it.
No tech company needs to care at all about fraud as long as valuations and funding are so closely tied to monthly users, revenue, and other metrics that can be used to launder bad usage as growth.
I've seen some spam on Twitter, but it's is nothing compared to what I see in Instagram comments. Go to any soccer/football post and it is full of bots posing as attractive women.
This is what happens when you outsource moderation to algos and temp workers. Try spamming HN or Reddit with crypto giveaway scams (or any other scam) and see how long you last. (Hint: not long). Algos help , but invariably smart spammers will evade them, hence the needs for humans. Twitter does not lose much business to spam. All they need is to keep most of it under check.
I have a four letter dormant Twitter account.
I get about four mentions/replies per day that link to some nondescript crypto airdrop since several weeks. I manually mark each and every such tweet as spam and block the user, which needs 5 clicks or so. Nothing seems to help.
You know at some point Twitter will just arbitrarily snatch your user name up and assign your account a randomly generated sequence. It's been done before without a given explanation. Just keep that in the back of your mind.
I followed a lot of academics that turned into spambots around 2020. I'm not sure if they were hacked or just transformed like locust, but they started exclusively spamming covid and anti-trump articles from US media.
[+] [-] jsnell|4 years ago|reply
First, we don't see the spam that was outright blocked, so we have no idea of what the false negative rate is. Second, I bet that this is not a binary block/allow decision, but there are all kinds of ways of reducing the engagement that probable spam gets without outright blocking. The latter is operationally preferable since it reduces the cost of false positives and since it makes the iteration loop for the spammers a lot slower.
(But also, I can't remember when I last saw spam in my Twitter feed.)
[+] [-] ryanSrich|4 years ago|reply
The most ironic part is that Twitter will show the spam/scams, but hide other replies, from real people, that Twitter thinks are mean.
[+] [-] jonas21|4 years ago|reply
Is it bots? I follow a few bot accounts and get value out of them. Is it blatantly promotional content? This actually seems to be one of the intended uses of Twitter. Is it low-quality content? I feel like lots of people earnestly tweet out low-quality stuff, and who gets to judge what's low-quality anyway?
[+] [-] gojomo|4 years ago|reply
• abuse of @-replies
• unsolicitied DMs
…which both grow with account size (& other forms of prominence, & certain topic-areas), so your anecdotal testimony that it's not in your feed doesn't do much to qualify or refute the magnitudes of others' problems.
Twitter's biggest internal spam problems are:
• Twitter's ads are repetitive, poorly-targeted crap
• Twitter does not provide reliable ways to disable their unwanted inserts – quickly ignoring any number of 'see less often' choices, and doing things like randomly reverting people from their chosen 'latest' to Twitter's algorithmic 'home' feed.
As fair definitions of 'spam' or even more generally 'harassment' include "continued unwanted interactions against expressed preferences", this means Twitter Inc is the biggest spammer/harasser on its own platform.
[+] [-] stochaztic|4 years ago|reply
Interestingly, we had one person's account whose search results still showed searchbanned tweets. They would not show for that person if they logged out. We also could not find out why that person's account in particular could see them.
[+] [-] 3pt14159|4 years ago|reply
The spam Twitter lets through is ridiculous. Is it hard to catch all offenders? Yes. Is it obvious Twitter is no where close to that? Also yes. We should have higher standards for our social networks.
[+] [-] jonathan-adly|4 years ago|reply
Edit: WARNING this will immediately fill your feed with spam. Don't click anything that you get.
[+] [-] MattGaiser|4 years ago|reply
I’ve known a few startups that built their initial user bases utterly deluging Reddit and Twitter in manual spam.
[+] [-] smsm42|4 years ago|reply
That's wrong. Results can be seen even if the process is not transparent. If every bridge the engineer builds collapses, I can judge him without looking into all the drawings. If every business the entrepreneur runs collapses I can make conclusions from that, even without seeing the books. If there's a lot of garbage on Twitter, I can judge it's doing a bad job even if I don't get to see its algorithms.
[+] [-] winternett|4 years ago|reply
Algorithm development has been poor and really frustrating to users because if they even come across someone who followed the Kardashians (for example), they literally get spammed with that news for the life of their accounts, and Twitter's "mutewords" functionality has also not worked for many years, I can't tell if that is intentionally so or not.
[+] [-] pmarreck|4 years ago|reply
Yep. Perfect example of the Survivorship Fallacy/Bias https://en.wikipedia.org/wiki/Survivorship_bias
[+] [-] tptacek|4 years ago|reply
Twitter could solve my spam problem tomorrow with a setting that says "don't show any posts with links if they're from accounts none of your friends or friends of friends follow".
[+] [-] stefan_|4 years ago|reply
Like, no, I see it every day. Lots of it.
[+] [-] imgabe|4 years ago|reply
[+] [-] Graffur|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] fleddr|4 years ago|reply
Take an account that attracts bots, Elon Musk is one of many examples. What do we see...
a) Hundreds of replies in the very first few seconds. It's humanly impossible to post this fast.
b) Replies share the exact same profile photo, typically of a well known original account.
c) User names are all a slight misspelling of a well known original account.
d) What they post is repetitive, same spam links and texts.
Every single pattern above, as well as combinations of them, are absolutely trivial to detect. But absolutely nothing is done about it.
[+] [-] riffic|4 years ago|reply
I modded /r/Twitter on reddit for a year but burned myself out (I chose to de-moderate FYI) simply because Twitter doesn't care about its community or even recognizing the existence of the community that developed around trying to provide the support the company won't provide on its own. Perhaps my take is cynical, but I really do like the concepts of a social media service like Twitter. The execution of it, however?
Perhaps this is an outsider's perspective but it seems people who work for Twitter would rather pat themselves on the back rather than make improvements.
My complaint of the week - you can't say the words "hacked" and "account" without having scambots asking you to get in touch with "their friend" who will help you restore access to your account. or something. It's just a fucking scam.
Also, just look at threads in reddit flagged with the Bug Report, Complaints, or even Question flair. The users are just bewildered and the experience is 100% user-hostile:
https://www.reddit.com/r/Twitter/search?q=flair%3ABug%2BRepo...
https://www.reddit.com/r/Twitter/search?q=flair%3ACOMPLAINTS...
https://www.reddit.com/r/Twitter/search?q=flair%3AQuestion&r...
[+] [-] wizzwizz4|4 years ago|reply
Have you heard of Mastodon? https://joinmastodon.org. If so, how do you think it compares?
[+] [-] holler|4 years ago|reply
It's fascinating that such a community would even need to develop, but maybe it's because of the difference in mediums?
I'm working on an alternative and would love to chat if there's a way to connect!
[+] [-] slimsag|4 years ago|reply
They don't truly like the fact that Twitter is a bunch of popular people shouting into the void, they want to bring Twitter 'closer' to something like Facebook (but not exactly) where you are interacting with a closed loop of friends in order to engage more every-day (read: less popular) people.
Anyone who has created an account on Twitter in recent years knows that new accounts are indiscriminately opted into new mandatory 'intelligent' social features like the purple star recommendations[0], while anyone with an old account will never see such a thing.
They're keeping their old user base, doing nothing to improve spam for them, while driving new users to a more closed-loop friend system where spam doesn't matter.
[0] https://twitter.com/mattthr/status/1009095580109426688?lang=...
[+] [-] tekacs|4 years ago|reply
Hmm I have an old account (Sep 2007) that sees purple star recommendations…
[+] [-] LeoPanthera|4 years ago|reply
I wanted to create a second Twitter account, but almost immediately after signing up it was blocked for "suspicious activity". I hadn't even posted yet. The only way to resolve it was to add a phone number to the account, but I only have one and it was already assigned to my first Twitter account.
So I was stuck.
How on earth do the spam robots do it? There are so many.
[+] [-] president|4 years ago|reply
[+] [-] paulpauper|4 years ago|reply
[+] [-] jzwinck|4 years ago|reply
He worked on this for a while, all the data he needed was made available and he analyzed every account on Twitter. His analysis said one third of all accounts were bots.
He presented these results to management, who said the number must not be that high, and discussed what it would mean for their MAUs or whatever metrics if these accounts were removed.
None of the identified accounts were deleted. Instead, the project to identify them was canceled, and the engineer quit.
More accounts means more money. Follow the money.
[+] [-] upbeat_general|4 years ago|reply
[+] [-] NelsonMinar|4 years ago|reply
[+] [-] dgellow|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] hamiltonians|4 years ago|reply
Crypto giveaway livestream scammers stand to make from $300 million to -$1 billion/year
https://scaminvestigations.substack.com/p/youtube-crypto-giv...
https://twitter.com/saylor/status/1487141374386450440
The spam and scams are so persistent because they make so much money, the scammers invest considerable time evading the algos, staying one step ahead.
[+] [-] nostromo|4 years ago|reply
https://www.infoworld.com/article/2674702/techology-business...
[+] [-] marban|4 years ago|reply
[+] [-] DisjointedHunt|4 years ago|reply
People calling it a “hard” problem simply parrot the narrative they see other engineers who have tackled it in leadership positions adopt. The reality is it’s a “Dirty” problem. The equivalent of a sewer cleaner in the old days that is a thankless, low pay, low career prospect, riddled-with-politics , nightmare.
Look up the salary bands for roles in spam fighting. The top 5% probably make bank and even that is not wildly large amounts relative to the valley (~$650k TC which is director level)
The LARGE pools of spending go toward maintaining an army of contractors that get passed down a barrage of things that some teams flag through automation. Same at twitter as it is as Facebook and Google. You need only hit up LinkedIn for the right search keywords to find the contractor hotspots.
The point being, Paul is right. If these platforms wanted to solve the problem, the smartest hires would go there motivated by a culture enabling high impact and good compensation.
The hell hole that is moderation operations would make even those guys with the Six-Sigma-black-belt-world-championship-something ops degrees shed tears for how soul destroying the environment is to work in.
Source: Have friends in these roles. Seen it play out a bit more than a decade at this stage.
[+] [-] riffic|4 years ago|reply
Content moderation at scale is impossible to do well. There's never going to be perfection here. It's over.
Your points are also 100% valid and on-the-nose.
https://www.techdirt.com/articles/20191111/23032743367/masni...
[+] [-] ALittleLight|4 years ago|reply
[+] [-] pmarreck|4 years ago|reply
Is being mean "offensive"? How would you detect that well? The more intelligent the meanness, the harder it would probably be to detect...
Are certain thoughts or concepts offensive? Are they ALWAYS offensive (across both time and location on the Earth), or only for the time being, or only for a place?
Is the word "Jews" offensive? Always, or depending on context or who is saying it?
Is it possible to be satirically offensive in a way that an algorithm would have difficulty detecting? What if I quoted something offensive to argue against it? (I literally got a tempban for this once.)
Perhaps they are "incredibly bad" at it because it is an algorithmically impossible problem that is deeply tied into the subjective sensibilities and tastes of a perceiving consciousness at one point in time (or set of consciousnesses, all of which perhaps only accidentally happen to coincide)
[+] [-] asojfdowgh|4 years ago|reply
further, if you are trying to block, lets say, crypto scams, on a platform which allows strong positive discussion about crypto, which allows people to talk about stuff they are selling for crypto, etc etc etc, you easily start losing points to differentiate
the defender needs to classify every message on the site, in a way that allows detecting spam well after classification, while maintaining over a 99.8% rate these days, while aiming for a 0% false positive rate
the attacker just needs to type random messages at their keyboard and use reused passwords / buy client id/secrets from shitty devs, to get access to verified accounts
[+] [-] pchristensen|4 years ago|reply
[+] [-] 2OEH8eoCRo0|4 years ago|reply
[+] [-] ziml77|4 years ago|reply
[+] [-] nitwit005|4 years ago|reply
[+] [-] Sebguer|4 years ago|reply
[+] [-] partiallypro|4 years ago|reply
[+] [-] paulpauper|4 years ago|reply
[+] [-] habi|4 years ago|reply
[+] [-] riffic|4 years ago|reply
You know at some point Twitter will just arbitrarily snatch your user name up and assign your account a randomly generated sequence. It's been done before without a given explanation. Just keep that in the back of your mind.
[+] [-] iqanq|4 years ago|reply
How much are you getting paid for moderating Twitter?
[+] [-] MarkMc|4 years ago|reply
I think it's just more exciting for people at Twitter to work on NFTs than to do boring work like spam control or filtering ads by user language.
[+] [-] nathias|4 years ago|reply