top | item 39888383

A deep dive into email deliverability in 2024

350 points| xoneill | 2 years ago |xomedia.io | reply

227 comments

order
[+] xyst|2 years ago|reply
Given how much weight “Gmail”, “Outlook”, and “Yahoo” email providers pull, I have always wondered about a different type of attack on business entities: “targeted failed deliverability”

Basically in this attack, a victim (particularly a business or mailing list or NGO) is sending out bulk emails to which the attacker owns. Even sourcing this out to shady off shore click farms would work too.

Attacker then marks the victim’s emails as spam in Gmail/Yahoo/Outlook. The “AI spam filters” pick up on this new “spam activity” and will then mark future emails as spam or even delete them before reaching real customers.

After a year, company bleeds money on a quarterly basis. Ad departments wonder why there is decreased engagement through email. Technical departments are bamboozled.

Maybe a big company will be able to weather the storm or just ditch email altogether. But small companies would definitely take a hit. Even smaller NGO or political mailing lists would lose donations (assuming email was a significant source of new donations).

Probably a very low vector of attack tbh, but something that has lingered in my mind.

[+] Fatnino|2 years ago|reply
I know of victims who had their legit email template lifted by actual slammers. The spammers would embed the legit template invisibly in their emails and then only have a few short lines visible with the actual scam. The idea being that filters would see the majority of the email is legit looking and let it in. Eventually users would flag enough of these as spam and the template itself would trigger the blocks. Then the spammers would move on to the next victim who's email template still gets through filters.

Meanwhile the first victim is left to pick up the mess where none of their email gets through anywhere.

[+] kirse|2 years ago|reply
Yea I've thought about this but not from the "attack on entities" angle but moreso a consumer-rights / boycott angle. I've had a negative enough experience with a large "maximizing shareholder value" company that I went back through my email history and marked every single one of their comms as spam.

Might be a drop in the bucket, but it doesn't take many votes to make a difference in the spam world.

I'm sure this will evolve soon enough and email delivery might increasingly become pay-to-play with all sort of backroom agreements, if it isn't already.

[+] elorant|2 years ago|reply
That’s why most e-commerce sites send their marketing stuff from a different domain. If it gets flagged they can still send transactional e-mails from their main domain. Assuming that both mail servers are on different ips.
[+] nemomarx|2 years ago|reply
A webforum I know has a rule against marking email notifications they send at spam (You can opt out of receiving them through the site, they just don't want you doing it on the email client end) to avoid this happening to them. For a small org, it's kind of a real risk?

I'm not sure how larger orgs mitigate it.

[+] jeromegv|2 years ago|reply
Isn't that what DMARC policy would prevent? If the emails being sent by the attacker are failing SPF/DKIM, then we can configure the DMARC policy so that Gmail never delivers those fake emails in the first place. So that attack would not be happening.
[+] semanticist|2 years ago|reply
I don't think this would work in practice. My employer sends daily deal emails without using a third-party service like SendGrid or SES and what we do is pay a company like Validity who interface with the big email providers for us. They have honeypot emails that get and validate our emails, they get feedback from the providers on how much they like/dislike us, and we get reports on this.

So an attack like this would be very obvious very quickly, even leaving aside that we'd notice a huge spike in email sign-ups and probably kill their accounts (especially since they're not going to be buying anything from those sock puppet accounts!).

[+] dns_snek|2 years ago|reply
Do political mailing lists get through in the first place? I don't live in the US, but someone must've sold one of my throwaway gmail addresses thinking I did.

Every couple of months I end up checking its spam folder and it's just a daily barrage of spam from both DNC and RNC, 1-2 emails per day like clockwork. None of them ever got through to the inbox though.

[+] Neil44|2 years ago|reply
You don't typically use your main email system for bulk sending, you use a third party for that who is used to taking that heat.
[+] pompino|2 years ago|reply
This is Google's business model, they throw completely legitimate emails your business sends into spam/marketing, so you're forced to pay them for gmail ads.
[+] BeFlatXIII|2 years ago|reply
> political mailing lists would lose donations

Now you have me rooting for the bad guys.

[+] bongodongobob|2 years ago|reply
That's just email working as intended. You pay a third party a small monthly fee to handle email blasts via a relay.
[+] ttul|2 years ago|reply
This change was necessary and long overdue. Requiring domain owners who send significant volumes of email to properly sign their messages allows receivers to more clearly delineate good from bad based on domain reputation rather than IP address reputation.

As more domains send email through shared IP space on transactional and marketing services, having the ability to attach reputation reliably to the sender domain is incredibly helpful in reducing abuse.

[+] adrian_b|2 years ago|reply
The condition about "significant volumes" is not true.

Google states that the new requirements are mandatory only when you send at least 5000 messages per day.

This is a lie. I send at most a few messages per day and usually less than one per day was towards a gmail account and I had implemented a part of the requirements, but not all of them.

Nevertheless, Google has started to reject my messages, so I was forced to waste time with the implementation of all requirements, even if they are somewhat redundant.

[+] cj|2 years ago|reply
As a sys admin, what do you do when you see 5% of your email hitting spam because the recipient’s Office365 mail server is misconfigured?

Agreed it’s a net positive, but it kills me when the reason emails land in spam is misconfiguration at the recipient’s end. (Like forwarding emails which breaks SPF)

[+] ozim|2 years ago|reply
We also slowly but surely are moving towards IPv6 which will make reputation based on IP somewhat useless when I can have as many new IP addresses as I want. They would have to make all newly seen IPv6 addresses not trusted by default when bad actor could send each email from different IPv6 address.
[+] xoneill|2 years ago|reply
Agreed, this is the upside for sure. Despite frustrations, I'm hoping this cleans things up.
[+] inetknght|2 years ago|reply
Spam is indistinguishable from malicious content.

You did not sign up for the "newsletter". Your email address was harvested and given to malicious actors hell-bent on screwing you. Clicking on anything will take you to a website where your best interest is not at all what the company is going to do with your information. At best you might just remove one source of junk in your inbox. At worst, you end up clicking on something that turns out to install malware on your machine.

So what should you do?

1. Don't click on unsubscribe links.

2. Click the spam report button

3. Stop using big email services that ignore spam reports. Gmail panders to other big businesses by letting them spam you without giving you the option to blacklist the entire domain yourself. Malicious content will continue to enter your inbox until you move to an email provider that takes your privacy and security seriously.

[+] DeanGadberry|2 years ago|reply
Fair assessment. Which email service provider do you use?
[+] r1ch|2 years ago|reply
I'm surprised how many big companies fail the one-click unsubscribe test. Whether it's Cloudflare or Akamai blocking the connection, pages that take 5+ seconds to load, pages that require you to sign in or input your email address again... don't be surprised when customers reach for the Report Spam button instead.
[+] nottorp|2 years ago|reply
I don't unsubscribe from emails I haven't opted in to. So report spam it is.
[+] Ayesh|2 years ago|reply
I'm using NextDNS with AdBlock list, which is effectively a Pi-hole on the cloud.

The most annoying this is when email senders use click tracking on domains that are blocked by those AdBlock lists. I keep a separate browser instance to copy-paste those links into, but then I have to login again.

I prefer sending unsubscribe emails instead of clicking links. Gmail can automate it.

[+] LVB|2 years ago|reply
I’ll unsubscribe, but now call BS on the “this may take 14 days to take effect…” nonsense in 2024. If I’m getting more emails in a couple of days, they’re getting marked as spam. (Looking at you, TripAdvisor. If you can figure out how to build AI-generated itineraries, you can figure out how to not email them.)
[+] foreigner|2 years ago|reply
Agreed, I think of it was a simple UI competition. Customers will do whichever is must convenient: unsubscribe or report spam.
[+] hedgehog|2 years ago|reply
One thing the April changes break is forwarding between e-mail services. If you currently forward from say an old university address at [email protected] to a personal GMail account at [email protected] that will no longer work. This must be relatively uncommon if the major providers are charging ahead with these changes but it's pretty annoying for the people affected.
[+] acidburnNSA|2 years ago|reply
I'm surprised anyone's been getting through at all without perfectly configured SPD, DKIM, and DMARC. I've had a well configured self-hosted personal email server for years and still struggle to get through sometimes, though it does seem to be getting better.
[+] cj|2 years ago|reply
The thing that kills me about DMARC is how often is fails with Microsoft specifically. And also with any use case involving the recipient forwarding mail (which breaks SPF alignment)

I want to follow best practices it recently changed p=quarantine to p=none after fear that legitimate emails aren’t passing DMARC despite properly configured DKIM and SPF.

Hell, I would love p=reject but not until recipients fix their incoming mail servers to handle edge cases like email forwarding breaking DMARC

[+] superkuh|2 years ago|reply
The worst is when they accept the mail but silently tag it spam and put it some place the intended receipient will never see it. Google's gmail is the worst about this. Corporate email isn't email anymore. It's a walled garden / silo like Facebook.
[+] 77pt77|2 years ago|reply
My personal VM has just been placed in some RBL because the entire /24 address space was blacklisted.

Someone (allegedly) sent SPAM and now my machine that sends maybe 3 emails a week is blacklisted

[+] andimm|2 years ago|reply
I think you mixed envelope (RFC5321) and headers (RFC5322) in your text.

The domain name in the From: field in the email envelop header is inspected and aligned with other domains authenticated by either SPF or DKIM:

The envelope does not have any header, the headers are in the content/body of the email. Also your screenshot of the "Here’s an example email envelop from an organization that passes all of the email security guidelines:" are the mail headers and not the envelope information.

Great presentation on this topic from dmarc.org

https://dmarc.org/presentations/Email-Authentication-Basics-...

[+] gtech1|2 years ago|reply
Roughly 50% of my daily Spam comes from @gmail & @hotmail/@outlook accounts.

What exactly are they doing about that ?

[+] jjav|2 years ago|reply
> What exactly are they doing about that ?

Nothing at all, because they are too big to care.

Aggressive decentralization is the only way to save the Internet. Host your own email, get everyone you know to host their own email.

[+] yobbo|2 years ago|reply
Yep. It seems to be from stolen accounts, maybe gathered from leaked account/password lists.

The result is that outgoing hotmail/outlook smtp servers are added to blacklists until they start content filtering their own users.

[+] jesterson|2 years ago|reply
They may not - but you fail to properly set up DMARC.
[+] nanidin|2 years ago|reply
I ran my own mail server for more than a decade. Same IP the entire time, never sent spam (for personal use only.) Finally threw in the hat last year and moved to a paid service - it was a pain to tell every person I sent mail to to check their spam box and mark me as not spam or add me to contacts. Beyond that, gmail smtp servers kept getting onto spam blocklists, so I wasn't receiving mail from gmail at times.
[+] willyt|2 years ago|reply
Speaking as someone in an industry that receives a lot of unwanted and seemingly un-unsubscribable marketing emails. I have never ever bought anything from a company that has sent me an email cold. I have my inbox set to show the first two lines and I delete them without opening them pretty much all the time. The only thing marketing emails do is annoy me.
[+] deadbunny|2 years ago|reply
So what any competent sysop has been doing for years?
[+] EGreg|2 years ago|reply
1. GMail will block your email if you don’t allow one-click unsubscribe. But this is very insecure since anyone can unsubscribe you if you forward your email

Easy Unsubscribe: Implement easy unsubscribe options (One-click Unsubscribe). Gmail users have tools to report spam, unsubscribe from unwanted emails and control their inbox experience. If it is too difficult to unsubscribe from your emails, customers will be more likely to flag your email as spam. Additional links provided in the ‘References’ section at the end of this article.

2. At the same time, Apple’s ITP will start removing all the information from the URL and only leave the domain, if it classifies your site as a “bounce tracker”. This means you won’t even know who to unsubscribe on one click! So all your emails will be blocked.

https://getcake.com/apples-intelligent-tracking-prevention-2...

[+] snowwrestler|2 years ago|reply
Gmail does not require one-click unsubscribe, what they actually require is that you include the “List-Unsubscribe” header in bulk emails, with a functioning mailto or http target.

If I forward your newsletter, that’s not a bulk email and it won’t include that header.

This is an important distinction that seems to get glossed over in a lot of the coverage and guides about the recent Gmail and Yahoo changes.

[+] gruez|2 years ago|reply
>2. At the same time, Apple’s ITP will start removing all the information from the URL and only leave the domain, if it classifies your site as a “bounce tracker”. This means you won’t even know who to unsubscribe on one click!

>https://getcake.com/apples-intelligent-tracking-prevention-2...

Your source doesn't actually say that "ITP will start removing all the information from the URL", only that it will "limit it the same way as third-party cookies" and will be "purging website data in such instances".

[+] Avamander|2 years ago|reply
You can do one-click unsubscribe with headers as well, those aren't usually forwarded by MUAs.
[+] louis-lau|2 years ago|reply
Why would apple identify the domain used for unsubscribes as in use solely for being a bounce tracker?
[+] Animats|2 years ago|reply
Then there's the other side - receivability. IDrive is supposed to send me an email each day reporting backup status as seen by the backup servers. Those messages have been flaky since mid-February. Logs indicate the backups run; it's just the completion emails that fail.

Their support people blame me, although they admit others have the same problem. They're not using a mail delivery service - the emails come directly from an IDrive server.

They're sending to my web site, which forwards to my personal address. There's no filtering at the first stage, and a division into Accept/Greymail/Junk at the next stage. Neither Google nor Yahoo is involved at any point.

[+] keybored|2 years ago|reply
I’ve gotten some emails from Gmail about delaying my emails to Gmail users because I apparently send too many emails. I use git-send-email(1) which might send a cover letter plus X patches right after each other. These Gmail users are then in the CC. So I’m not a mailing list. The email list is the To recipient.

I’ve been wondering if this was the cause. I don’t send out 5000 emails (I’m not 10X). But there’s this part:

> While these guidelines primarily affect bulk senders, senders with less volume per day can also be affected if they are not adhering to these guidelines.

I haven’t looked into it yet but I guess I should.

I use my own domain and I’m hosted by a not-Gmail provider.

[+] paulnpace|2 years ago|reply
I wonder how this ends up impacting government agencies and especially courts and law firms. My experience has been all three struggle with these things.