While I don't necessarily trust an external company with all my emails, I also don't trust myself to maintain the myriad daemons involved in this setup without doing something subtly wrong that results in my server not sending/receiving all the mail it should -- or, worse, being used for spam.
What would be useful is a pre-assembled virtual machine image or other form of appliance that allows you to deploy and test a mail server within about an hour or so, without having to duct-tape any of this together yourself.
I've been hosting my own mail since 1996. It's actually one of the easier services to self-host:
1) SMTP was developed for unreliable environments. If you have problems with uptime, your incoming email will bounce around for 5 days before it gets dropped. So assuming you can get your SMTP server running one day out of five, you shouldn't be in danger of losing anything.
2) Contemporary daemons like postfix and dovecot have sane defaults, so even a naive default install should be mostly secure. They're also extremely low velocity, so once you set it up there's not a lot of ongoing maintenance.
E-mail and the extensions and extras that make it go and make it nice /are/ duct-tape. I wouldn't advocate for /anyone/ running their own mailserver unless they're totally aware of what they're doing and on top of spam issues.
When considering if running your own mailserver is practical for you, consider the total cost of ownership; you'll most likely be paying for a dedicated server (or vps, VM, or whatever the cool kids are using these days) which will require staying on top of all the "normal" things; monitoring, backups (and you'll probably want to test them), updates, what blacklists to use, ensuring that /your/ server doesn't end up blacklisted, etc. Unless you've been a sysadmin before and know what this all entails, I wouldn't really recommend it. If you're not part of some bigger org (a hosting co or bigger company), it'll also be harder to get your one-off no-name server removed from blacklists, since this looks just like something Joe Linkfarm would do. I could show you this one really simple trick that a mother found to whiten your server reputation, but...
As far as creating a VM image that sets all this up for you, it's kinda the point that it doesn't exist; mailservers don't work well when run by people who don't really care about them. It's quite far away from a "set and forget" deal. This is why many hackers / sysadmins still use GMail; it's not the best at everything, but it's OK enough at most things that it's the least hassle to use.
That all said, I do agree that it'd be great to have a service which offers both more control for power users (I'd love a mailfilter-style config w/regex support for GMail), better privacy (easy PGP integration, etc), tag support, threading, and normal IMAP support, but I'm not quite holding my breath. This seems like a decent problem for some start-up to solve :) It's not a "sexy" problem, but it's a real one.
> What would be useful is a pre-assembled virtual machine image or other form of appliance that allows you to deploy and test a mail server within about an hour or so, without having to duct-tape any of this together yourself.
There at least a couple Linode stack scripts that will do this, but I haven't given any of them a try since last time I checked they were mostly/all Ubuntu based. Dovecot + postfix isn't all that difficult to set up, and there are literally tons of guides all over the place (Arch wiki, Linode, Dovecot's site, etc.) for dovecot, postfix, courier, and anything else you might want. Perhaps the most dumbfounding thing for a beginner is the certificate step...
To play the devil's advocate, what exactly is the practical use of all this if most of your family and friends are on Gmail (and couldn't be arsed to figure out pgp)? From what I can see, your emails will now be sent in the clear over the internet, instead of staying within google's servers. Either way, the government's going to get your data, but at least you're protected against... /more/ unscrupulous people snooping on your stuff?
That's a great point. Reminds of the time I taught my friend to use PGP and sent him an encrypted email. Every single time, he would reply in plain-text, thus exposing my older conversation. When asked why, he told me it's too much of a pain to do it. So my being careful about my privacy doesn't help if other people don't play along.
The shuttering of Reader and Gmail's new and awful Compose have convinced me - more than years of Stallman rants and NSA snooping - that relying on closed-source SaaS is a terrible idea. It is a question of if, not when, your SaaS provider does something you intensely dislike, and which you have no power to change.
With SaaS, you don't even have the power to say - Damn your improved version, blast your hip new design, I will stick with the old one, thankyouverymuch. You are always on Version Now.
The more intertwined your relationship and dependence on integration with related products, the messier the ensuing divorce.
This, to my mind, represents a far more inevitable reason to figure a way out of GMail.
I'm not sure why you can't do those things on FastMail.
(disclaimer: I work for FastMail)
Sure we have folders rather than tags, which means you can't add multiple of them to the same message. Probably the biggest lack is that you can't manage IMAP flags via the web interface. Otherwise, our search is now very powerful (since about March this year) and allows you to build filters that show messages from multiple folders in a single view.
OP here. Fastmail had 4 out of my required 5 features. I used it for a while. I'm still a paying customer (I paid for a bunch of years). Tags were a deal breaker. It's just not sufficient for how I want to organize things.
Just another happy FastMail user here. I don't need tags/labels, so you have everything I need.
I have an Enhanced account that is linked to my domain, and an Ad Free account where important mail gets forwarded to. The latter is accessible from my phone, but the former isn't, so if anyone steals my phone they can only see the last few messages I exchanged, and I can just disable an alternative login (similar to revoking an API key) to lock my phone out permanently. I also set up my default personality in the Ad Free account to send all mail through my Enhanced account, so every mail I send from my phone is automatically saved in my Enhanced account and even has the correct DKIM signature.
One question, though, and I'm sure a lot of people are curious about it. You are an Australian subsidiary of a Norwegian company, but your servers are in the US. What happens when an American three-letter agency wants to see the contents of your servers?
I signed up for Fastmail enhanced, really like it so far, only things I don't like are the read only LDAP contacts and the security settings. Specifically on the security settings, I want to see support for a limited alternative login which can access IMAP desktop/mobile clients (such as K9) but not have full unrestricted web access. Tags, I can without those if they can't be implemented well. Thanks for a great product.
Tags would be useful, but the UI aesthetics could be better too: now it's too plain and minimalistic. Look at Yandex Mail for example: it's just less boring. I guess you could attract much greater audience by adding theme support and proper marketing in light of this outrage.
That said, I like fastmail very much: keyboard navigation, search and overall speed are great!
Setting up a server in any hosting environment at this point comes with the assumption that its contents can be read at any time by the operators and whoever they let in without you ever knowing about it.
Organization and categorization are the sticking point features, given what I've seen of most open source webmail applications. But worth looking around. If you have a basic mail server image, you can keep trying out applications on top of it to see what works for you.
Going beyond that to something with a whole lot more encryption and less of an ability for hosting providers to read your data would really require a product dedicated to that end: that is hard to get right.
> Setting up a server in any hosting environment at this point comes with the assumption that its contents can be read at any time by the operators and whoever they let in without you ever knowing about it.
How exactly is that any different than it was 6 months ago?
> Setting up a server in any hosting environment at this point comes with the assumption that its contents can be read at any time by the operators and whoever they let in without you ever knowing about it.
Indeed. If you want to be secure, you need to keep your email server at home, or in some other location you trust, and make sure all communication on the net is encrypted. That way, if an adversary wants your secrets, they will have to burgle your house.
> That's still a lot better than Gmail.
True, because it takes effort for GCHQ/NSA/whoever to look at your server, which they probably won't do.
> Going beyond that to something with a whole lot more encryption and less of an ability for hosting providers to read your data would really require a product dedicated to that end: that is hard to get right
Assuming you don't have any sort of console access configured and its a physical server how could they do that without letting you know via at least a reboot?
When I saw the link's title, I immediately thought that it would be another webmail client hosted by someone, especially given the domain name. Because almost everytime I see a "hacker's X" or "X for hacker" title on HN I'm just afflicted by the content, so I got used to it.
But here, what a pleasant surprise. The post is actually describing a real hacker's replacement for Gmail (which coincidentally is almost my setting, except I use mu [1] instead of notmuch). I'll keep it as a reference to send people asking for alternative email hosting.
I did this for a long time, but it's really annoying:
1. If your provider goes down, you lose mail.
2. If you are conversing with people who are using an insecure mailer, such as gmail, Yahoo, etc (which is probably > 99.9% of all e-mail users), your e-mail is still accessible to the NSA, or to some Fortune 100 advertising company.
3. It's only a matter of time before the "big dogs" in email abuse the position and decide who is and isn't allowed to send/receive email outside of their little oligarchy, either on their own or at the behest of governments.
Like so much else that has been corrupted, we need to scratch the current architecture as too insecure, and build something truly secure for the future. This isn't in the interests of the Googles of the world, and it's actively in the worst interests of the NSA/FBI/CIA, so it's probably the right thing to do.
1. It's likely he's storing emails on the VPS. This puts us back at square one. A third party has a copy of your emails. And we know email does not garner the same privacy protections as postal mail.
2. You need a domain name. That system (DNS), as it is currently implemented (i.e., everyone setting their root zone to servers they do not control), is highly centralized -- few people maintain their own root zone, despite being easy to do. Domain names are susceptible to false allegations copyright and trademark infringement by private parties, not to mention easy censorship by the US gov't. When you lose your domain you lose email. (Though you shouldn't have to: email works fine with IP addresses in brackets.)
So what's the solution:
1. Get a reachable IP (e.g., through ISP) or get a VPS. But if you get a VPS only use it to pierce NAT (how is left as exercise for reader - hint: supernode), not run a mail server. Don't store sensitive data like email on a VPS, or route sensitive data through it.
2. Use IP addresses not domain names. Alternatively, set up your own DNS that is available as a peer-to-peer service, or have your email contacts use a DNS server and root zone you collectively maintain: free domain names that you control. No one can censor your DNS (phonebook), except you.
On a side note, they seem to have just hit v4 two days ago.
Second side note, if you decide to use k9, be sure to turn off the signature under composition settings for each account you add.. it's turned on by default.
>handing an advertising company most of my personal and professional correspondance seems like a bad idea
That's your main complaint? Google is an advertising company. People buying ads on Adsense don't have access to your personal information. This is simply not true.
Many of us did similar in the 90s. I might go this route again but would use Postfix and Dovecot. I'd do this for my wife and kids as well - but if I get hit by a bus, email eventually not working is not something I should burden my wife with.
Tip: don't use dovecot-lda. Let postfix handle it. Dovecot LDA is chroot hell.
I worry about the same things which is why I'm actually migrating my email to outlook.con slowly (in spite of the NSA etc). She knows my passwords already.
I always thought a big part of the reason people used gmail was for the snazzy web-based UI that was one of the first popular AJAX-based web applications.
I eagerly read the article to see what alternative to this feature the author was suggesting, so I was surprised to see he's reading the emails with a standalone client...in fact, it's an emacs plugin!
Normal people definitely don't want to manage a mail server though. Life is too short to waste figuring out why you're banned on Spamhaus for the 93th time.
GMail sucks, but a home-made contraption is not the alternative.
To be fair to the author, I don't think an article that starts with "A Hacker's..." and/or involves Emacs has a pretension to being a solution for normal people.
I had a similar setup a couple of years ago. The main problem I had was the maintenance required. If you have any machine publicly accessible you have to be on top of security updates and proper system hardening. I gave up after my exim4 Debian system got 0-day rooted.
If doing it again I would avoid a Debian based distro. I'd probably use openbsd. And the less ports open the better.
Nice stringing together of unixy tools to get this working. I had not heard of notmuch and its related ecosystem (afew, alot, etc.), so that's a useful discovery on my part.
I ran a set up similar to this for many years. It's not that hard, for those with a little unix experience. As moxie mentions, email is very forgiving—you have to break it badly and leave it broken for a long time before you start to lose messages.
What eventually drove me to GMail was spam. I tried a bunch of different filters, and never found one with good-enough accuracy. Finally I decided that the independence and privacy wasn't worth the time I spent fiddling with filters and dealing with misclassified messages. As far as I can tell, Gmail is 100% accurate. Problem solved.
It would be incredibly useful if there was a mail service that received email over SMTP, encrypted it straight away with a public key, then just dumped the encrypted email into a general-purpose online storage solution (e.g. an S3 bucket).
That would IMO provide a good base for encrypted client-side apps to build on top of. Open source would better be able address the problem of writing a client once the money needed for hosting and storage is taken out of the equation.
I think more of us need to run mail servers. For ourselves, for our families, and possibly for others who are willing to pay. Email is far too centralized now, at a handful of companies, in a handful of data centers. So in that regard, running a mail server on a VPS at one of the popular providers is kind of missing the point.
My local cable ISP doesn't allow incoming or outgoing connections to port 25, nor incoming connections to port 80. So at least for now, I can't run a mail server in my home. I've thought about switching to DSL, but then I would take a major hit in speed, in both directions.
Luckily, I have another option. There's a hosting provider where I live (Wichita, Kansas) that offers KVM-based virtual machine hosting. So I'll get a VM there, and if the service is any good, I'll move there from Linode. The pricing isn't competitive with Linode, let alone DigitalOcean, and I doubt that the connectivity is as good, since the server will be in a building here in Wichita rather than a real data center. But I'm willing to try it, in order to support a local business and fight the centralization of the Internet.
[+] [-] LeafStorm|12 years ago|reply
What would be useful is a pre-assembled virtual machine image or other form of appliance that allows you to deploy and test a mail server within about an hour or so, without having to duct-tape any of this together yourself.
[+] [-] moxie|12 years ago|reply
1) SMTP was developed for unreliable environments. If you have problems with uptime, your incoming email will bounce around for 5 days before it gets dropped. So assuming you can get your SMTP server running one day out of five, you shouldn't be in danger of losing anything.
2) Contemporary daemons like postfix and dovecot have sane defaults, so even a naive default install should be mostly secure. They're also extremely low velocity, so once you set it up there's not a lot of ongoing maintenance.
[+] [-] seanp2k2|12 years ago|reply
When considering if running your own mailserver is practical for you, consider the total cost of ownership; you'll most likely be paying for a dedicated server (or vps, VM, or whatever the cool kids are using these days) which will require staying on top of all the "normal" things; monitoring, backups (and you'll probably want to test them), updates, what blacklists to use, ensuring that /your/ server doesn't end up blacklisted, etc. Unless you've been a sysadmin before and know what this all entails, I wouldn't really recommend it. If you're not part of some bigger org (a hosting co or bigger company), it'll also be harder to get your one-off no-name server removed from blacklists, since this looks just like something Joe Linkfarm would do. I could show you this one really simple trick that a mother found to whiten your server reputation, but...
As far as creating a VM image that sets all this up for you, it's kinda the point that it doesn't exist; mailservers don't work well when run by people who don't really care about them. It's quite far away from a "set and forget" deal. This is why many hackers / sysadmins still use GMail; it's not the best at everything, but it's OK enough at most things that it's the least hassle to use.
That all said, I do agree that it'd be great to have a service which offers both more control for power users (I'd love a mailfilter-style config w/regex support for GMail), better privacy (easy PGP integration, etc), tag support, threading, and normal IMAP support, but I'm not quite holding my breath. This seems like a decent problem for some start-up to solve :) It's not a "sexy" problem, but it's a real one.
[+] [-] sciurus|12 years ago|reply
http://www.kolab.org/ http://www.sogo.nu/english.html http://www.zarafa.com/ http://www.open-xchange.com/home.html https://www.zimbra.com/
[+] [-] ics|12 years ago|reply
There at least a couple Linode stack scripts that will do this, but I haven't given any of them a try since last time I checked they were mostly/all Ubuntu based. Dovecot + postfix isn't all that difficult to set up, and there are literally tons of guides all over the place (Arch wiki, Linode, Dovecot's site, etc.) for dovecot, postfix, courier, and anything else you might want. Perhaps the most dumbfounding thing for a beginner is the certificate step...
[+] [-] captn3m0|12 years ago|reply
[1]: http://bitnami.com/stack/roundcube
[+] [-] easy_rider|12 years ago|reply
[+] [-] chj|12 years ago|reply
[+] [-] 13throwaway|12 years ago|reply
[+] [-] magic_haze|12 years ago|reply
[+] [-] thex86|12 years ago|reply
[+] [-] tubelite|12 years ago|reply
With SaaS, you don't even have the power to say - Damn your improved version, blast your hip new design, I will stick with the old one, thankyouverymuch. You are always on Version Now.
The more intertwined your relationship and dependence on integration with related products, the messier the ensuing divorce.
This, to my mind, represents a far more inevitable reason to figure a way out of GMail.
[+] [-] joe_bleau|12 years ago|reply
Trusted TLS connection established to gmail-smtp-in.l.google.com[173.194.79.27]:25: TLSv1 with cipher RC4-SHA
Anonymous TLS connection established from mail-yh0-f48.google.com[209.85.213.48]: TLSv1 with cipher RC4-SHA
[+] [-] sliverstorm|12 years ago|reply
[+] [-] brongondwana|12 years ago|reply
(disclaimer: I work for FastMail)
Sure we have folders rather than tags, which means you can't add multiple of them to the same message. Probably the biggest lack is that you can't manage IMAP flags via the web interface. Otherwise, our search is now very powerful (since about March this year) and allows you to build filters that show messages from multiple folders in a single view.
[+] [-] dbpatterson|12 years ago|reply
[+] [-] kijin|12 years ago|reply
I have an Enhanced account that is linked to my domain, and an Ad Free account where important mail gets forwarded to. The latter is accessible from my phone, but the former isn't, so if anyone steals my phone they can only see the last few messages I exchanged, and I can just disable an alternative login (similar to revoking an API key) to lock my phone out permanently. I also set up my default personality in the Ad Free account to send all mail through my Enhanced account, so every mail I send from my phone is automatically saved in my Enhanced account and even has the correct DKIM signature.
One question, though, and I'm sure a lot of people are curious about it. You are an Australian subsidiary of a Norwegian company, but your servers are in the US. What happens when an American three-letter agency wants to see the contents of your servers?
[+] [-] yfefyf|12 years ago|reply
It would be better if you have a more desirable .com domain name. I will probably buy an Enhanced account to use my own domains after free trial.
[+] [-] da_n|12 years ago|reply
[+] [-] mrgreenfur|12 years ago|reply
[+] [-] kroger|12 years ago|reply
The only problem I have with FastMail is that I've been receiving an increased amount of spam in my inbox.
[+] [-] ansgri|12 years ago|reply
That said, I like fastmail very much: keyboard navigation, search and overall speed are great!
[+] [-] brendoncrawford|12 years ago|reply
[+] [-] josephb|12 years ago|reply
I just wish it had an address book and calendar that I could sync.
Those are the things I miss the most from Gmail.
[+] [-] exratione|12 years ago|reply
That's still a lot better than Gmail.
Setting up your own mail server is not a terrible woe-inducing undertaking if you have a working recipe to follow and are comfortable with the Unix command line (e.g. http://www.exratione.com/2012/05/a-mailserver-on-ubuntu-1204...).
Organization and categorization are the sticking point features, given what I've seen of most open source webmail applications. But worth looking around. If you have a basic mail server image, you can keep trying out applications on top of it to see what works for you.
Going beyond that to something with a whole lot more encryption and less of an ability for hosting providers to read your data would really require a product dedicated to that end: that is hard to get right.
[+] [-] mbell|12 years ago|reply
How exactly is that any different than it was 6 months ago?
[+] [-] cabalamat|12 years ago|reply
Indeed. If you want to be secure, you need to keep your email server at home, or in some other location you trust, and make sure all communication on the net is encrypted. That way, if an adversary wants your secrets, they will have to burgle your house.
> That's still a lot better than Gmail.
True, because it takes effort for GCHQ/NSA/whoever to look at your server, which they probably won't do.
> Going beyond that to something with a whole lot more encryption and less of an ability for hosting providers to read your data would really require a product dedicated to that end: that is hard to get right
That's what I'm working on.
[+] [-] oijaf888|12 years ago|reply
[+] [-] p4bl0|12 years ago|reply
But here, what a pleasant surprise. The post is actually describing a real hacker's replacement for Gmail (which coincidentally is almost my setting, except I use mu [1] instead of notmuch). I'll keep it as a reference to send people asking for alternative email hosting.
[1] http://www.djcbsoftware.nl/code/mu/
[+] [-] just2n|12 years ago|reply
1. If your provider goes down, you lose mail.
2. If you are conversing with people who are using an insecure mailer, such as gmail, Yahoo, etc (which is probably > 99.9% of all e-mail users), your e-mail is still accessible to the NSA, or to some Fortune 100 advertising company.
3. It's only a matter of time before the "big dogs" in email abuse the position and decide who is and isn't allowed to send/receive email outside of their little oligarchy, either on their own or at the behest of governments.
Like so much else that has been corrupted, we need to scratch the current architecture as too insecure, and build something truly secure for the future. This isn't in the interests of the Googles of the world, and it's actively in the worst interests of the NSA/FBI/CIA, so it's probably the right thing to do.
[+] [-] _oxford|12 years ago|reply
1. It's likely he's storing emails on the VPS. This puts us back at square one. A third party has a copy of your emails. And we know email does not garner the same privacy protections as postal mail.
2. You need a domain name. That system (DNS), as it is currently implemented (i.e., everyone setting their root zone to servers they do not control), is highly centralized -- few people maintain their own root zone, despite being easy to do. Domain names are susceptible to false allegations copyright and trademark infringement by private parties, not to mention easy censorship by the US gov't. When you lose your domain you lose email. (Though you shouldn't have to: email works fine with IP addresses in brackets.)
So what's the solution:
1. Get a reachable IP (e.g., through ISP) or get a VPS. But if you get a VPS only use it to pierce NAT (how is left as exercise for reader - hint: supernode), not run a mail server. Don't store sensitive data like email on a VPS, or route sensitive data through it.
2. Use IP addresses not domain names. Alternatively, set up your own DNS that is available as a peer-to-peer service, or have your email contacts use a DNS server and root zone you collectively maintain: free domain names that you control. No one can censor your DNS (phonebook), except you.
[+] [-] kefs|12 years ago|reply
https://code.google.com/p/k9mail/downloads/list
On a side note, they seem to have just hit v4 two days ago.
Second side note, if you decide to use k9, be sure to turn off the signature under composition settings for each account you add.. it's turned on by default.
[+] [-] t0|12 years ago|reply
That's your main complaint? Google is an advertising company. People buying ads on Adsense don't have access to your personal information. This is simply not true.
[+] [-] ishbits|12 years ago|reply
[+] [-] pyre|12 years ago|reply
[+] [-] kryten|12 years ago|reply
I worry about the same things which is why I'm actually migrating my email to outlook.con slowly (in spite of the NSA etc). She knows my passwords already.
[+] [-] csense|12 years ago|reply
I eagerly read the article to see what alternative to this feature the author was suggesting, so I was surprised to see he's reading the emails with a standalone client...in fact, it's an emacs plugin!
[+] [-] hcarvalhoalves|12 years ago|reply
GMail sucks, but a home-made contraption is not the alternative.
[+] [-] mjn|12 years ago|reply
[+] [-] siddboots|12 years ago|reply
Ugh!
All you are saying is that if you aren't looking for a home made contraption, then a home made contraption isn't what you are looking for.
[+] [-] werid|12 years ago|reply
[+] [-] Aldipower|12 years ago|reply
[+] [-] alemhnan|12 years ago|reply
Or like Heroku: you create your "managed" mail server with a click.
[+] [-] nvarsj|12 years ago|reply
If doing it again I would avoid a Debian based distro. I'd probably use openbsd. And the less ports open the better.
[+] [-] mjn|12 years ago|reply
[+] [-] cdjk|12 years ago|reply
You probably should add SPF records too, if you don't want your outgoing mail marked as spam.
[+] [-] cwp|12 years ago|reply
What eventually drove me to GMail was spam. I tried a bunch of different filters, and never found one with good-enough accuracy. Finally I decided that the independence and privacy wasn't worth the time I spent fiddling with filters and dealing with misclassified messages. As far as I can tell, Gmail is 100% accurate. Problem solved.
[+] [-] richdougherty|12 years ago|reply
That would IMO provide a good base for encrypted client-side apps to build on top of. Open source would better be able address the problem of writing a client once the money needed for hosting and storage is taken out of the equation.
[+] [-] mwcampbell|12 years ago|reply
My local cable ISP doesn't allow incoming or outgoing connections to port 25, nor incoming connections to port 80. So at least for now, I can't run a mail server in my home. I've thought about switching to DSL, but then I would take a major hit in speed, in both directions.
Luckily, I have another option. There's a hosting provider where I live (Wichita, Kansas) that offers KVM-based virtual machine hosting. So I'll get a VM there, and if the service is any good, I'll move there from Linode. The pricing isn't competitive with Linode, let alone DigitalOcean, and I doubt that the connectivity is as good, since the server will be in a building here in Wichita rather than a real data center. But I'm willing to try it, in order to support a local business and fight the centralization of the Internet.
[+] [-] voidlogic|12 years ago|reply
https://s3.amazonaws.com/uploads.blog.zimbra.com/wp-content/...