I guess my question is can you please fix your braindead blacklisting?
Several times per year—I can practically guarantee it’ll happen sometime in December, and indeed had to deal with this just five days ago—I end up with a bunch of users whose email notifications stop working because Microsoft have started blocking the entire netrange where my server lives. I don’t have control over other Linode customers, guys! I even wrote extra code to stop sending mail to addresses that start bouncing specifically to avoid blacklisting, so after MS finally processes a blacklist mitigation request, someone also has to go in and re-enable those accounts.
SPF, DKIM, DMARC are all configured; I’ve sent from the same IP address for about a decade; I’ve not once received an email abuse report; mail volume is low (most days, volume does not reach the minimum threshold for SNDS to report data[0]). I’ve never had any other mail provider blacklist my server. SNDS always says everything is OK as I am S3150s. What is even the purpose of SNDS at this point when it lies about what is going on?
[0] P.S. The janky SNDS calendar widget resets the month to the current month every time you click on a date, even if the date being viewed is in a previous month. I don’t have any hope that anyone will ever touch SNDS code again since it was clearly designed in the early 2000s and the copyright on the site is now ten years old, but this is a pretty silly bug.
My guess is that the effectiveness issue isn’t actually due to SNDS and is probably related to sender reputation having famously high false positive rates. I read a paper a while back which introduced a different algorithm with tighter bounds on regret, I didn’t really understand it tbh, but I can implement it behind a flight and run a data study to see if it works better. The problem is that most graph based stuff doesn’t scale super well because of something-something complexity classes. I think the lady who architected it 5 years ago didn’t do a great job and there’s a bunch of arbitrary config stuff which was put as a placeholder and then became enshrined in stone… but the guy maintaining it rn is really smart so I’ll have him review my half-assed PR when he’s back next week (and idk how long it’ll take to finish the other half of it, shit never ships around here).
About the calendar widget thing… man am I glad I our team doesn’t own that. No one ever touches legacy stuff cause they’re afraid it’ll break or no one will update but the trick is to file it as an accessibility bug since that gets someone to actually prioritize it since it shows up in reports that the execs read. But dude good luck getting that off the backlog, the one engineer we have who is good at UX stuff (i.e, can code with both quality and velocity instead of just one) has her hands full as is.
Here is the issue that most ESPs are facing.. Every 5-6 months something is being enabled or not from Outlook's side which affects either IPs or the domain name of the sender and messages land in Junk folder or in quarantine zone.
Now, I do know that the IPs might be affected by complaints or spamtraps, or maybe the client sent something suspicious, but trust me most ESPs don't allow those messages to be sent. Also, when the IPs appear GREEN in SNDS, and SPF/ DKIM and DMARC are a part of DNS authentication and headers appear like this: CAT:HSPM;SFS:(13230031)(4636009)(451199024)(7596003)(356005)(7636003)(86362001)(450100002)(8676002)(1096003)(14286002)(34206002)(5660300002)(336012)(26005)(42186006)(9686003)(33656002)(83380400001)(7846003)(33964004)(564344004);DIR:INB;
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
You are expecting that quarantine zone is the last place to find a legit message.
For obvious reasons I won't share more details, but I bet that from time to time someone is messing with spam filters that can easily result in false positive and angry senders.
In any case, especially when we raised tickets to Outlook, at least please inform your team not to reply like robots. If they will share with us the exact reason why a message landed in junk folder that would really help us. If it is the content, we will change it. If it is related with the sender, we will block the sender. If those are complaints, we will block senders and check their subscription sources, but at least we need something especially when SNDS shows Green IP, 0 spamtraps, 0 complaints.
Thank you for reading this.
Yo I’m not even gonna apologize about this, it would be so wack if we didn’t do that:
a) if a mail sever looks like it’s gonna send spam, then you gotta block it. I personally have philosophical hang ups about this, like it’d be wrong to sentence someone to prison for crimes they didn’t commit just because a system added up some points and made a prediction with high confidence, but in real life, you absolutely need to be proactive.
b) there is literally no way to do this that wont immediately get abused. Trust me we’ve tried. We make it nearly impossible to get unlocked on purpose because if it was easy, then it’d be like 1 innocent person using it and 99 attackers due to the adversarial incentive structures.
Now ofc there’s more nuance here, we really do want to get it wrong less often, and you do pay us so it’s not fair to blame it all on the bad guys, so I’m grateful for the feedback but I think you should give me even more detailed feedback since there’s not much I can do except give a vague high level explanation unless you help me by being specific.
Similar question as my sibling comments. I have rented a server with a static IP address for over ten years now. Nobody else has used this IP during this time. Yet, every few months I have to beg Microsoft to unblock the IP. In the beginning I could do this on my own, but something changed a few years ago and now I have to beg my ISP (netcup) instead to contact Microsoft on behalf of me to temporarily whitelist the domain. Then wait another 2-3 months and do the same dance again.
Why? Why can Microsoft not learn that an IP has been healthy and spam-free for 10+ years and only bother me when there is actual spam is being sent?
Aww man, not joking this actually breaks my heart, something about the way you wrote it makes it sink in how much we’ve failed you. I’m angry at how much of your time we’ve wasted and this experience is completely unacceptable.
…I think this is just a systemic issue beyond my ability to comprehend, let alone solve, and— I hope I’m wrong about this but honestly when I look ahead it seems the future is only going to get worse for people like you. Which I wish I could phrase in a way that was more kind and respectful, it’s not what anyone wants, these unthinking scars inflicted on email as a medium.
But what I can do is make sure that it’s not worse for you, specifically. If I was perfect I’d attack this rot at its core, but I’m not, so I’ll just solve the problem in front of me even though I know it doesn’t scale and hope God forgives me. Get in touch with me directly and I’ll figure out how to make sure you don’t have to jump through those hurdles again.
Why doesn't whitelisting an address ensure one receives messages from it, the address has never sent spam, sends at most a couple of emails a day. But I couldn't receive emails from it, there was no notification or information despite the address being on my whitelist?
Huh? This shouldn’t be possible in principle? Don’t quote me on that though, I wish I’d paid more attention to my notes but they’re a mess and haven’t kept up with newer changes, if they were accurate at all in the first place. I’d submit an escalation so support can look into it.
Eg. Known bad domains, known bad IP addresses, incorrectly setup DKIM / SPF, no reverse DNS, non-matching reverse DNS, and that's before even looking at content to determine whether spam.
For privacy and compliance reasons (read: “oh boy wouldn’t wanna get sued, eh?” reasons) we actually don’t snoop into the message body much. Hooray, good job on not doing the maximally big brother thing for once, MS!
My hot take is that this prolly won’t last because every org descends to doing a creepy level of data collection eventually so I have a textbook on privacy preserving ML downloaded for when we join the “surveillance but we found a way to make it technically legal” squad. We haven’t done that yet though.
What's the best way to quickly get MS to trust a server/domain?
Does MS ignore IP reputation in cases where the domain has a good reputation?
How would you go about getting a new domain and an IP address from a public cloud provider working consistently?
I've had issues with outlook when it comes to new domains and IPs, but after some time it works. I do however usually have more email than a personal server so what's the best way - if such a thing exists - for a personal server that has much lower volume of mail to be trusted?
Hmm, oh wow, occasionally I’m reminded that if I flipped sides to run phishing campaigns I’d be totally unstoppable.
There isn’t a quick way, by design. You need to wait a minimum period and meet some predicates, and the organized scammers already know what the period is via empirical testing but I’m not comfortable disclosing details of those predicates for disorganized scammers to use. More so because I’d definitely get into trouble for it than due to any belief in security via obscurity. Cushy job makes you risk averse.
Since I can’t share any of the tricks, some general advice— the main thing that matters is a long track record of good behavior. You can end up in a vicious cycle where you fight the system when it punishes you and then it doubles down on the beatings— this is bizarre and kafkaesque and happens all the time. What you want is for there to be two-way communication, if it’s unbalanced with traffic being broadcast but no one engaging with it, that’s going to be cracked down on sooner than if recipients reply.
I don’t. I have slept in the daytime ever since covid and actually got a move to the east coast approved as a health accommodation after I started routinely missing important afternoon meetings due to my incurable insomnia (mornings are easy when you stay up all night). I still struggle with it, especially since it’s not a consistent offset to my circadian rhythm. There’s data I’ve collected but it’s hard to fit a simple function to it— it’s not like I’m on a 26 hour schedule either. This isn’t due to trauma or addiction, my brain is just an outlier in many dimensions and this is one of them.
My penis enlargement pill newsletter isn't showing up in my customers' inboxes. I could have been a penis-enlargement millionare if it wasn't for your stupid spam filter. What to do?
csnover|2 years ago
Several times per year—I can practically guarantee it’ll happen sometime in December, and indeed had to deal with this just five days ago—I end up with a bunch of users whose email notifications stop working because Microsoft have started blocking the entire netrange where my server lives. I don’t have control over other Linode customers, guys! I even wrote extra code to stop sending mail to addresses that start bouncing specifically to avoid blacklisting, so after MS finally processes a blacklist mitigation request, someone also has to go in and re-enable those accounts.
SPF, DKIM, DMARC are all configured; I’ve sent from the same IP address for about a decade; I’ve not once received an email abuse report; mail volume is low (most days, volume does not reach the minimum threshold for SNDS to report data[0]). I’ve never had any other mail provider blacklist my server. SNDS always says everything is OK as I am S3150s. What is even the purpose of SNDS at this point when it lies about what is going on?
[0] P.S. The janky SNDS calendar widget resets the month to the current month every time you click on a date, even if the date being viewed is in a previous month. I don’t have any hope that anyone will ever touch SNDS code again since it was clearly designed in the early 2000s and the copyright on the site is now ten years old, but this is a pretty silly bug.
sheikheddy|2 years ago
About the calendar widget thing… man am I glad I our team doesn’t own that. No one ever touches legacy stuff cause they’re afraid it’ll break or no one will update but the trick is to file it as an accessibility bug since that gets someone to actually prioritize it since it shows up in reports that the execs read. But dude good luck getting that off the backlog, the one engineer we have who is good at UX stuff (i.e, can code with both quality and velocity instead of just one) has her hands full as is.
vel0city|2 years ago
You do have control over being a Linode customer though. If Linode isn't doing enough to prevent abuse, they deserve to be blocked.
Sarp402024|2 years ago
Dunedan|2 years ago
sheikheddy|2 years ago
a) if a mail sever looks like it’s gonna send spam, then you gotta block it. I personally have philosophical hang ups about this, like it’d be wrong to sentence someone to prison for crimes they didn’t commit just because a system added up some points and made a prediction with high confidence, but in real life, you absolutely need to be proactive. b) there is literally no way to do this that wont immediately get abused. Trust me we’ve tried. We make it nearly impossible to get unlocked on purpose because if it was easy, then it’d be like 1 innocent person using it and 99 attackers due to the adversarial incentive structures.
Now ofc there’s more nuance here, we really do want to get it wrong less often, and you do pay us so it’s not fair to blame it all on the bad guys, so I’m grateful for the feedback but I think you should give me even more detailed feedback since there’s not much I can do except give a vague high level explanation unless you help me by being specific.
TonyTrapp|2 years ago
Why? Why can Microsoft not learn that an IP has been healthy and spam-free for 10+ years and only bother me when there is actual spam is being sent?
sheikheddy|2 years ago
…I think this is just a systemic issue beyond my ability to comprehend, let alone solve, and— I hope I’m wrong about this but honestly when I look ahead it seems the future is only going to get worse for people like you. Which I wish I could phrase in a way that was more kind and respectful, it’s not what anyone wants, these unthinking scars inflicted on email as a medium.
But what I can do is make sure that it’s not worse for you, specifically. If I was perfect I’d attack this rot at its core, but I’m not, so I’ll just solve the problem in front of me even though I know it doesn’t scale and hope God forgives me. Get in touch with me directly and I’ll figure out how to make sure you don’t have to jump through those hurdles again.
unknown|2 years ago
[deleted]
currysausage|2 years ago
Avamander|2 years ago
pbhjpbhj|2 years ago
What's the rationale there?
sheikheddy|2 years ago
BLKNSLVR|2 years ago
Eg. Known bad domains, known bad IP addresses, incorrectly setup DKIM / SPF, no reverse DNS, non-matching reverse DNS, and that's before even looking at content to determine whether spam.
sheikheddy|2 years ago
My hot take is that this prolly won’t last because every org descends to doing a creepy level of data collection eventually so I have a textbook on privacy preserving ML downloaded for when we join the “surveillance but we found a way to make it technically legal” squad. We haven’t done that yet though.
What do you mean by tiers, exactly?
NorwegianDude|2 years ago
Does MS ignore IP reputation in cases where the domain has a good reputation?
How would you go about getting a new domain and an IP address from a public cloud provider working consistently?
I've had issues with outlook when it comes to new domains and IPs, but after some time it works. I do however usually have more email than a personal server so what's the best way - if such a thing exists - for a personal server that has much lower volume of mail to be trusted?
sheikheddy|2 years ago
There isn’t a quick way, by design. You need to wait a minimum period and meet some predicates, and the organized scammers already know what the period is via empirical testing but I’m not comfortable disclosing details of those predicates for disorganized scammers to use. More so because I’d definitely get into trouble for it than due to any belief in security via obscurity. Cushy job makes you risk averse.
Since I can’t share any of the tricks, some general advice— the main thing that matters is a long track record of good behavior. You can end up in a vicious cycle where you fight the system when it punishes you and then it doubles down on the beatings— this is bizarre and kafkaesque and happens all the time. What you want is for there to be two-way communication, if it’s unbalanced with traffic being broadcast but no one engaging with it, that’s going to be cracked down on sooner than if recipients reply.
Biganon|2 years ago
sheikheddy|2 years ago
trympet|2 years ago