top | item 45473730

Self-hosting email like it's 1984

269 points| xmx98 | 5 months ago |maxadamski.com | reply

182 comments

order
[+] sam_lowry_|5 months ago|reply
(had to dug my comment from under a flagged parent)

I self-hosted for well over 20 years, I did not throw the towel and I do not plan to. Self-hosting is a sign of pride. Neither my government nor my Prime Minister nor even my Ministry of Interior or Foreign Ministry can host their own email.

Last time I checked, only State Security self-hosted.

I was probably lucky, but I rarely had delivery problems. The last one was a couple years ago with Microsoft swallowing my emails and it was due to the combination of a fairly old exim and a TLS certificate verification quirk at *.protection.outlook.com. I found a fix in the form of a configuration option somewhere on SO.

In all fairness, there is very little maintenance involved, and whenever I have to do maintenance work, I take the opportunity to learn something new. Like this year, I decided to finally replace my aging Debian jessie setup by Arch Linux, and I rewrote all cron jobs as systemd timers.

I must admit that when I send a really important email, I check the mail server log if it went off without errors, but this does not bother me as checking logs manually once in a while is a good thing anyway.

Lastly, a piece of advice: treat self-hosting like a hobby and learn to enjoy it.

Oh and the very last thing: the person who designed Exim configuration for Debian deserves a special place in hell for all the hours wasted. If you set up Exim on Debian, just figure out how to use the upstream exim config and adapt it to your needs.

[+] Xenoamorphous|5 months ago|reply
> I decided to finally replace my aging Debian jessie setup by Arch Linux, and I rewrote all cron jobs as systemd timers.

Man, I wish I had 1% of the motivation I had 20 years ago to do something like this, before all the full time job, wife and child.

[+] hmng|5 months ago|reply
My first email usage was at University, pre-WWW. After that I briefly used some ISP email service, but that was on a time of very limited storage and POP only accounts, so I started hosting my own email even before having an always-on internet connection, using a relay and dynamic DNS to receive email when online. Now a days, I use a small VPS to route and receive email, but final destination and storage is on my home server. Over the years, I had, like others here, to ask Outlook and other providers to unblock my IP or domain, but it has been rare.

I really don’t want to live in a world where only two or three companies run email for the entire world, and this is my little act of resistance.

[+] isodev|5 months ago|reply
It’s amazing how today we have social networks bending backwards to be able to call themselves “open” and “decentralised” when we already have all the tools we need to be truly independent.

I think when we’re building something with “good UX” the major point of “does this remove agency from users” is somehow missing from the picture. When everything runs on some kind of system, it’s not extraordinary to expect people to know how it works and maybe be able to do it themselves.

Otherwise, fast forward a decade of simplifications, and we can’t even install an app without someone on the other side of the world approving the “transaction”.

[+] mey|5 months ago|reply
> treat self-hosting like a hobby and learn to enjoy it.

This is why I have stepped away from a lot of my self hosting. I have turned my attention/time elsewhere. Apparently though the time/money balance is shifting a bit again, so it may be worth it to go back.

My biggest hesitance to self hosting email specifically is dealing with spam. What does that look like these days and do you have any pointers to share?

[+] elgaard|5 months ago|reply
I have been self-hosting for about 25 years. I remember the protection.outlook.com issue. Once there was an issue with a bank that tried to do encryption, but used an expired certificate. But once I told them what the problem was, and that it was a problem for paying customers, they actually fixed it.

Being able to check the server log can be very useful. E.g. to tell someone that their mail was delivered to a served using their domain name, with that IP-address at that time.

[+] commandersaki|5 months ago|reply
Email for me is a critical service, and the reasons I stopped self hosting after about 15 years is:

1. Because I couldn't ensure consistent backup and restore with regular monitoring,

2. no disaster recovery plan and in doing so it'd be more expensive than going through another email provider,

3. not always on top of security (my friend that I colo'd with also ran an email server and his system was struck with ransomware (with no backup [except a copy of email via thick client] or DR); I seemed to get away unscathed because I was using FreeBSD which generally less of a target).

I agree that it is little maintenance, but once you're off the happy path, it can be a huge pain in the arse and devastating.

[+] Krei-se|5 months ago|reply
Configure the dmarc reports, they tell you a lot and automatically why someone swallowed your mail.
[+] jesterson|5 months ago|reply
Same here. Dont wanna piss on your party but I don't see any particular pride. Prime minister or any minister to that matter are pretty pathetic positions in my books, but that's totally different conversation.

No delivery problems if you set up everything correctly. It's not luck, just the same reason why well maintained car runs smoother than something that's seen last maintenance 100,000 miles ago.

[+] stebalien|5 months ago|reply
I used to do this. What finally killed it wasn't reputation, it was the fact that I needed 100% uptime or risk losing messages, getting my address blacklisted, etc. Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.

Worse, GitHub (back in 2016 and 2018) would mark a recipient as "unavailable" after a single bounce, refusing to send any more notifications to that address. They since improved the situation and their support was actually very helpful and responsive here, but it's pretty clear that modern SMTP senders have an expectation that recipients will be "always online" that didn't exist when the protocol was invented.

[+] dijit|5 months ago|reply
I have a feature (called greylisting) whereby my server intentionally rejects the first mail it receives from a domain.

I have never had anyone claim that their mail has not been delivered to me, and I get a lot of mail.

Retry is built in to the spec, and if you’re really worried you can put a second “receive” SMTP server on the internet with a lower priority, and have it backhaul with LMTP.

———

Email was designed in a time where hosts were not perpetually connected to each other.

[+] logifail|5 months ago|reply
> it was the fact that I needed 100% uptime or risk losing messages

Q: If your server(s) is/are offline for a few hours, why would you "lose messages"?

I've just checked my own email server -> "up 219 days"

Honestly, compared with the stuff we do all day, this is not hard...

[+] Krei-se|5 months ago|reply
This is fearmongering. My mails always got resent after some hours or a day. It's absolutely NOT possible to tell if the problem is on your side, senders side or somewhere in between why a mail is not delivered once and no standard server config would simply toss it.

Host your own mail. I get 99% deliverability with 0 repuation since i do dkim and spf correct.

Don't be distracted by the "complexity" - if you config right it's totally doable.

Gives you actual private caldav too btw

[+] BikiniPrince|5 months ago|reply
I know right. It’s like, “what did they do to my boy?” as to huddle over the bullet ridden corpse of your son.
[+] abdullahkhalids|5 months ago|reply
Here is my advice to anyone wanting to test out self-hosting email. Start by using your self-hosted email to sign-up for accounts. You don't have to use the email address for your personal correspondence

Use Mail-in-a-box to get started [1]. You can literally set it up in a couple of hours by following the instructions and everything should just work.

After a few years, you can think about switching your personal correspondence to your new email.

[1] https://mailinabox.email./

[+] watermelon0|5 months ago|reply
I can recommend Stalwart [1] which is a complete mail service contained in a single binary, that doesn't really have any external dependencies, and is really easy to install and update.

I've looked (and tried) a few other projects in the past, but Stalwart was the easiest to setup, and I haven't had any issues with it so far.

[1] https://github.com/stalwartlabs/stalwart

[+] bonzog|5 months ago|reply
I've been running MIAB for a few years now with generally good success as an outgoing sender using a rented cloud machine and a "clean" reputation IP. I've had to email the Microsoft postmaster on one occasion when my emails weren't reaching Outlook users, but they were surprisingly helpful and it's been working fine for years now. It's a good learning exercise in setting up stuff like DKIM/SPF/DMARC.

That said - receiving account sign-up emails is the absolute biggest pain in the backside with Mailinabox! The greylisting anti-spam feature relies on bouncing unknown senders and waiting for a retry. The trouble is, many legit sites just don't bother retrying. So email verification for new accounts and 2FA-type stuff often takes ages to come through, if at all. MIAB stubbornly has no easy, mail user-facing way to temporarily disable spam filtering and it's a real PITA at times.

[+] boplicity|5 months ago|reply
Modern email providers, especially ones offered by ISPs often have the same problems that people criticize self-hosted providers for. Even Google has problems. For example, I regularly order via companies that use Shopify. Now, all of the shopify emails are going straight to spam in Gmail, despite constantly marking them as not spam. (These even pass dmarc/spf/dkim etc, so who knows what's going on here.)

Email delivery and receiving is not hard, but it's inevitably going to be imperfect, no matter the provider you use. There are so many bad actors out there, it's surprising that it works as well as it does.

[+] johnea|5 months ago|reply
Actually, full strength virtual (multi-domain) email hosting is also quite doable.

This is a great guide that's been used and updated for many years:

https://www.purplehat.org/?page_id=1450

Once hosting email for yourself, you may want to add new project-specific domains, or host email for friends and family. The database user accounts actually make it easier to add and remove users after the system is up and running.

This Purplehat guide provides a step by step procedure that's allowed many people and orgs to bring self-hosted email online...

[+] drnick1|5 months ago|reply
I think the following is a better guide for someone looking for a complete setup that includes an IMAP server and that can be used with regular email clients like Thunderbird:

https://workaround.org/ispmail-bookworm/

I set up my own server more or less following the above guide, but eschewed the database in favor of plain text files. I wanted to keep things simple since I am the only user, but the above guide should scale to big enterprise setups.

[+] Krei-se|5 months ago|reply
I have a writeup in german about self-hosting current and with debian trixie on https://krei.se/Doc

If you do it yourself and do it correct it's a pleasure. I have automatic updates with automatic reboot, tailored systemd to make sure all is well and status reports per mail - total bliss, easy 2-3 years, with trixie now even 5 until you have to touch it again.

It's mature software.

Host yourself! The peace of mind and control is totally worth it.

[+] man8alexd|5 months ago|reply
Where is UUCP? Why are addresses not bang paths? Where is sendmail.cf?
[+] munchlax|5 months ago|reply
Right. You better not self-host like it's 1984 because that would also mean you're an open relay. And vulnerable for pretty much anything you can think of.
[+] hmng|5 months ago|reply
Those wore the days :-) I remember playing on a University lab with half a dozen Unix workstations, sending an email with the path of server1!server2!server3 etc and hearing the email flowing from server to server by the noise of the disks!
[+] reaperducer|5 months ago|reply
Why are addresses not bang paths?

That's what I thought of when I saw the title, too.

Where are my ...killer!jolet! people at?

[+] ajross|5 months ago|reply
Ditto. I was sorely disappointed to click through "1984" to find a subheading on "setting up postfix".
[+] spkm|5 months ago|reply
I've been selfhosting for like dunno 10-15 years. Cheap kimsufi box, opensmtp, dovecot, later then rspamd, done. Never really had a problem. At one point telekom.de blocked my mailserver. I contacted them via [email protected] (or something) explaining that while kimsufix boxes are notorious for shady stuff, this is actually a legit mailserver and they whitelisted me shortly after (yeah I was surprised too how smooth that went). So, yeah, can't confirm all the troubles everyone seems to get on about. However I do own the kimsufi box (and the corresponding IP) for a long time now, so maybe I'm just lucky.
[+] dugite-code|5 months ago|reply
I've been self-hosting my email for over 10 years now (I'm going to link a bunch of my old comments on old email HN threads). I have fallen back to using Amazon's SES to send because all of Digital Ocean's IP blocks suddenly got marked as bad and I don't have enough volume to improve a new IP reputation - https://news.ycombinator.com/item?id=39891262, https://news.ycombinator.com/item?id=38471262

I use Gmail as a free spam harvester to train my own spam filter - https://news.ycombinator.com/item?id=38843288.

But as others here have suggested greylisting is extremely helpful in this space as legitimate servers should always retry. Well only my power company is the exception and they will fall back to sending paper bills, but even Gmail falls foul for them. It's also one big reason I'm not worried about up to a week downtime. But I have two email servers, a receiving and a storage server, the receiving is cattle and I car re-deploy in minutes if needed. - https://news.ycombinator.com/item?id=38512732

On greylisting I would say using https://github.com/stevejenkins/postwhite (even if it's very old and not actively maintained) has proven very important for the annoying 2FA emails, I strongly contend that email isn't suitable for this use case but that's another conversation)

[+] defanor|5 months ago|reply
I missed an incoming message (fortunately an unimportant one) from Amazon SES recently, since its 54.240.27.30 address was listed by bl.spamcop.net: Amazon kept trying different addresses while running into greylisting, until it tried that address and was rejected. Possibly it is less of an issue when sending between large providers (e.g., Amazon to Gmail), but apparently still not a perfect solution to ensure message delivery.
[+] billfor|5 months ago|reply
Assuming this is not hosted on your home system, since ISPs may block the ports and also most of the dynamic ips allocated are blacklisted, the issue with postfix is that its difficult to have a single set and forget config if you intend to use it on multiple internal machines, like for getting your root email on each system to one mailbox. Ideally you want a single main.cf for all your internal machines and for the outgoing/incoming mailhost to be determined solely by your mx or internal dns alias, but this is next to impossible with a single postfix config without getting mail loops on the system that is the mailhost. Exim and sendmail at least separate out the submit config from the rest of the configuration. Also you would be insane to try this without fail2ban or something similar. Postfix does a reasonable job of handling attackers but it does so quietly -- so you may not see the activity.
[+] 627467|5 months ago|reply
Say I want to test the waters for selfhosting email, and I already have my how domains setup with SaaS like Google workspace and equivalent. Is there a way to setup mx records so that both google and my own server gets email for a while? This would be useful to test the waters over a few months before fully migrating
[+] dizhn|5 months ago|reply
Not with MX but, look at google's split domain documentation. You can either have them handle the domain and forward you a copy, or you can have your own domain be the primary and forward to google. I have been using the latter for a few years now since not all of the users in the domain are using Google Workspace. They have a special address for forwarding to so you don't get into a loop. It has been working flawlessly for us.
[+] man8alexd|5 months ago|reply
You can set up a lower-priority MX to point to Google, so if your server fails, then email is delivered to Google. But if your server is misconfigured and returns permanent 5xx errors for legitimate emails, then it won't work, and the emails won't be delivered to Google.
[+] habibur|5 months ago|reply
Configure google to forward mails to your self hosted server.

When replying reply from your self hosted server.

That way you can gradually shift over.

I had been self hosting like this for years.

[+] nzeid|5 months ago|reply
No easy answer here. Individual MTAs or a cluster of them typically live under one unique domain. In your scenario, you'd have to point your existing records (or just MX) to your self-hosted instance, and have your self-hosted instance relay/autoforward to Gmail under a different domain. This might entail simply setting your Gmail back to @gmail.com.
[+] hmng|5 months ago|reply
Not really, SMTP relays will only send messages once, to one server.

But it’s not receiving that is the problem, that is generally fine, if ports are open at ISP / network level. It is the sending that is often tricky. Sending email on the other hand can be done from multiple servers (if SPF correctly configured) And nothing prevents you from sending email directly from your own relay. You could try that, and reception would not be affected.

[+] lutusp|5 months ago|reply
> "If something isn't working for you, please double-check your DNS records, and triple-check that TLS certificates are readable by the Postfix user, and that DKIM keys are readable by the OpenDKIM user. Postfix and OpenDKIM logs will also be useful. The OpenDKIM config file is especially unforgiving of typos, so watch out for small mistakes!"

I tried this over a period of years, aggressively changing my email server configuration as new challenges appeared, before realizing the basic problems were (a) a server's configuration is a moving target that requires constant revision, and (b) if your ISP has ever hosted a spammer, even briefly and inadvertently, then its entire address block may be universally blacklisted and you have to change ISPs, possibly several times.

So ... I gave up. If I had nothing better to do, if I just wanted to play email server whack-a-mole, that would be different, but I have a life apart from pleading with giant email recipients to trust my little server.

It's not as though Google, Microsoft, et al. have an incentive to trust small email servers -- quite the opposite. They can -- and do -- make the argument that they shouldn't trust anything but another big player like themselves.

[+] defanor|5 months ago|reply
FWIW, some of the things I configure differently:

- More of anti-UCE, with postscreen (greylisting, DNSBL and DNSWL checks), policyd-spf, body_checks, check_sender_access, check_client_access, postscreen_access_list.

- Setting "home_mailbox = Maildir/", to keep mail in user directories and in the Maildir format (which seems to be less prone to corruption than mbox is, and well-supported by MUAs).

- Leaving TLS defaults, except for the paths. I used to set mandatory TLS, but then ran into some servers not using it, and figured that I do not trust the involved servers more than channels between them anyway (especially the servers that do not support TLS). Being overly strict with allowed protocol versions (or even ciphers) also reduces compatibility, while for encryption it is better to rely on OpenPGP.

- I do set Dovecot (for both IMAP and SMTP submission); the recent configuration change did not seem like a big deal to me, and it was documented, so I found it easy to update. It is nice to be able to use email from a server (and that ability does not go away with Dovecot), but a local MUA also has its advantages.

- Registered at dnswl.org, to improve deliverability in some cases.

[+] pjmlp|5 months ago|reply
More like 1994 thereabouts, in 1984 most of us would be very lucky to have a dial up connection to the local BBS, under local phone call price rates.
[+] donio|5 months ago|reply
Not even that, Postfix didn't exist in 1994. This is a 2025 mail server setup and about as vanilla as it gets.
[+] ralferoo|5 months ago|reply
I first started maintaining a nail server in 1997. Most of the stuff in the article is newer than that.

For 1984, I'd have expected UUCP and bang paths to peer mail hosts. Instead the article starts off by setting up DKIM, from over 2 decades later!

[+] xvilka|5 months ago|reply
There's a way better solution for self hosted email these days - Stalwart[1]. Supports all necessary protocols and extensions, including modern JMAP. And, of course, it's memory safe, unlike Postfix and friends.

[1] https://github.com/stalwartlabs/stalwart

[+] crossroadsguy|5 months ago|reply
Where do people self-host these emails? When email self-hosting is talked about, my thoughts wander to Fastmail, Migadu, etcetera (I use one like these), but I quickly realise that's not it. On those lines, I do not believe these mail self-hosting folks are talking about some VPS, or server from some provider, or even AWS, et al., either — not self-hosting enough. It must be a computer/server always running at their home/basement/or so (with whatever power/Internet backup setup they have—or maybe not, as they might find it acceptable if something was missed/dropped). So is it that? And if that's what it is, then what is that mail self-hosting home setup of yours? What all have you got there? Just curious, I doubt I can go through that, as my patience gives in even trying to set up a VPS for a seedbox when it is time for the first maintenance/tweak.
[+] Gud|5 months ago|reply
I have a VPS at Hetzner. I pay 4 euro per month. It's inside a FreeBSD jail for separation.

Keeping FreeBSD up to date is extremely simple. run pkg update && pkg upgrade. It rarely breaks. Can't remember it ever breaking.

The main reason I prefer FreeBSD over Linux distros are the far superior package managers(pkg for binaries and ports for source code).

I also host my own web server using nginx, and sometimes other stuff. All in separate jails.

Back when I was a kid, I used to have my own servers.

[+] man8alexd|5 months ago|reply
The problem with self-hosting is finding an IP with an clean reputation and not on any block lists, with good neighbours (people sometimes block /24), with an open outgoing port 25. Then you'll need to slowly warm up this IP for weeks or months.
[+] talkingtab|5 months ago|reply
I personally believe it is worth exploring the idea of a different email realm for communities. The concept is pretty simple. Don't accept email from gmail, microsoft, hotmail or any other non-community member. Community members don't spam, don't send email in bulk and have reputation.

It is funded by pay-per-transgression. If you are a community member and someone receives unwanted email your reputation suffers. If you are gmail, et al you have to pay for each email sent & received.

Someone once wrote (let me know if you know the source) that users are not the customer, because they don't pay. It is advertisers who are the real email customers. This has resulted in a business model where users are prey animals. This is upside down and probably cannot be fixed without a hard fork.

I don't mean this is a good idea, or implementation. But I think it is a good direction.

[+] kinotoko|5 months ago|reply
For anyone interested in getting a mail server, I can really recommend Michael W. Lucas' Run Your Own Mail Server