top | item 18100807

Solving My Email Problem

179 points| BeetleB | 7 years ago |blog.nawaz.org | reply

125 comments

order
[+] nickjj|7 years ago|reply
My inbox is at 0 and other than gmail's auto-filtering tabs I don't have too much extra filtering going on (most of it is related to Google Alerts).

For the last 20 years my rule has been "If I check my email and inbox > 1 then deal with the email right now". I usually check my email every few hours.

I'm not Mr. popular, but I'm also not anonymous. I do personal support for 30,000+ people who have taken one of my online courses so I get a fair amount of email and other things. I tend to write massive responses from scratch for a lot of them as well.

The key is not to let it build up. Then you'll have 99 problems but email won't be one of them.

Also I'm not sure if this is related but I'm one of those people who think it's unthinkably rude to ignore someone. I would never in a million years read an email and not respond to it (unless it clearly wasn't meant to be responded to). Likewise, if I see a > 1 inbox I feel compelled to handle it ASAP because I don't want to keep that person waiting.

(Of course there's exceptions, like I'm not responding to pure spam, but I do get about 50 emails a week where people ask to post a guest post on my blog and I always manually respond nicely to them with a "no thanks but good luck" type of message along with some context around their niche).

[+] bo1024|7 years ago|reply
I can't really relate to this -- my relationship with email is pretty different. Lots of my emails represent an ongoing todo item that requires some hours of work, or just some time to reflect, so I can't deal with it immediately.

I could turn each one into a TODO item tracked somewhere else and archive the email, then later search for the email and reply or take action. But it makes more sense to me to categorize the emails and use an organized inbox itself as my todo list.

[+] glitchc|7 years ago|reply
This is a wonderful, but unfortunately very naive, approach to email.

That you firmly believe this approach just means you haven't worked in a govt. dept. or any kind of position where your response represents the response of the organization (or a sub-set thereof). In such circumstances, emails that are open-ended (responses risk being misunderstood), ambiguously phrased (responses will be twisted to server the author's needs) or controversial (any response paints the organization in a negative light) are often best ignored. If your question is ambiguous, a busy boss or colleague does not owe you an answer or vice versa.

Sometimes people respond out of anger or frustration. You could respond in kind, and escalate the battle, or simply wait and let the individual calm down. Not responding is a very useful technique when conflicts arise between team members or in the management chain.

I would rather ignore an angry abusive email from an upset colleague who is having a rough time in life, than respond in kind and escalate it into an HR battle. Of course if the behaviour continues, there would be no choice, but most people are rather contrite and apologetic after a brief outburst and it's much better for everyone to just pretend that it never happened. People make mistakes after all.

[+] ktosobcy|7 years ago|reply
Am I so weird that I don't have problem with my e-mail box? I don't receive spam (gmail, but recently all mail providers got really great at it and even if I do receive some spam is quite sporadic). I don't receive newsletters because (shock!) I don't sign-up for them (and even if I do once every blue moon I immediately click 'unsubscribe').

Yes, there is some small problem of having older messages "waiting for the right time" but this is not the problem of the e-mail itself but rather mental block to dedicate 1-3h to write response (you would spend same amount of time writing regular mail).

[+] manigandham|7 years ago|reply
No, this is just good email discipline.

Almost everyone I know who gets way too many emails and complains about 100s or 1000s of unreads has put themselves in that situation by signing up for everything. Spam is already handled well and there's just not that many sources of email if you take away newsletters and notifications.

[+] oooglaaa|7 years ago|reply
You’re not the only one! I unsubscribed from mostly everything and now my inbox gets anywhere from 1 to 2 new spam messages per week. Still annoying when it does happen, but a big improvement from before when I was getting 40+ emails per day, in which most were newsletters I never opened or just spam
[+] swaggyBoatswain|7 years ago|reply
Im in the same boat, never had a problem managing emails. I get dumped on marketing emails but I make it a commitment to unsubscribe and it doesnt take terribly long. Most marketing spam I get is legit marketing with legit unsubscribe services. For work I just use thunderbirds bayensian junk filter, and spamassassin on mailhost and it works great. Filter out emails for important clients and for automatic system updates from vendors

Gmail does a lot of filtering for me though already. I write a few filter rules if I intentionally want spammy emails, e.g. watching github changes, status on shipments, etc.

My rule of thumb for managing email is thusly

- if the email is read, its done

- if theres 100 spam mail, and the top link is marked read, all ones beneath it are too spammy to delete or remove out of sheer laziness

-if the email is starred, I need to refer back to it, but its not actionable (e.g. welcome email for hosting sites)

- if the email is marked important, defer it for later

I spend very little time on email per day following these rules. I have many filter rules set up so nothing but important emails get to me in designated sections

My rule of thumb for email is thusly

- only look at email 3 or 4 times at most. Once in morning, afternoon, evening.

- send emails asap if it can be done in 5 minutes or tell receipient about a delayed response later for longer actions.

-modify subject titles to reflect status on email thread

- DRY, dont repeat yourself. Send a link instead. Use autoresponders if your on vacation etc

- every month or two, go through all your spam, unsubscribe to everything

- use autotext expanders to write email signatutes if its necessary

-use ctrl + enter to send

- limit yourself to zero to two mouse left clicks per email. If you are doing anymore, you are doing it wrong. E.g. dont organize emails its a waste of time learn to use search and make rich subject email titles.

[+] jeppebemad|7 years ago|reply
Im 100% on the same boat. I’m not too strict in regards to handing out my email, but my inbox (Gmail) is completely kept free of spam. Is it really just gmail keeping me safe?
[+] cm2187|7 years ago|reply
My solution is to have my own domain, and a little tool to create and delete email aliases. When a company asks for my email, I just give them a new unique alias. I receive spam on this alias, 1) I know who leaked, 2) I delete the alias, stopping the spam. It is transparent to my correspondents (I can reply from the alias), cuts the spam pretty much to zero, and is low maintenance. It also has a small security benefit: even if I were to share a password between websites (which I rarely do), the login (email, or login part of the email) would be different/non guessable.
[+] 333c|7 years ago|reply
I do something similar, except rather than creating aliases, I just have a catchall that sends everything to my inbox. Then, if one address starts receiving spam, I create a rule to move all emails sent to that address to spam.

As far as responding from aliases, I haven't figured out a way to do that in my mail client (Mail.app), though I am able to accomplish it in my provider's web interface.

[+] Pxtl|7 years ago|reply
Gmail has this capability built in because you can just add +whatever to the end of the name and it's an alias to your email account, and you can block the [email protected] it as needed. Only problem is many sites mistakenly think + is an invalid char.

No overhead of creating addresses, you have an infinite address space for your account to play with freely.

More mail providers should offer such a workflow.

[+] checkyoursudo|7 years ago|reply
I do the same thing. I also have multiple domains. I will create email-forwards to my main email addresses to use mostly for website logins and such. I don't think that level of paranoia is really necessary, but I do it anyway.

For my alias management, I wrote a script to add and delete aliases and reload the MTA. Makes it super easy to do. Long time ago I would have to go through the process of creating a fake hotmail/gmail/yahoo/etc email address if I wanted to do something similar. This way is much nicer.

[+] msantos|7 years ago|reply
I do the same thing with Gandi.net . Their email aliases solution let's you achieve the same result, while doing away with the hassle of running your own email stack.
[+] BeetleB|7 years ago|reply
This was my first idea, scaled up a bit. I wanted a unique email address per person - not just for when I sign up. This way if someone harvested my email address through someone else, I could just delete the alias and give that person a new alias.

Way too much maintenance. This solution is easier.

[+] 0xBAADA555|7 years ago|reply
What little tool might that be? I'm looking to do this myself with a domain.
[+] anotherevan|7 years ago|reply
spamgourmet.com is not a bad solution similar to this.

The email aliases will forward a set number of emails, after which they go to /dev/null. You can white-list senders who you want to continue seeing emails from.

[+] nicwolff|7 years ago|reply
I did this with procmail 20 years ago: http://angel.net/~nic/spam-x/procmailrc.txt – and then stopped because lots of spam comes with random people's real e-mail addresses as the From, and my challenge autoreplies were quite rightly being reported as spam by their innocent recipients. Now I just use FastMail and let their excellent Bayesian filtering do its job. (Oh, and I filter all mail with "unsubscribe" in the body into a "Bulk Mail" folder".)
[+] JimDabell|7 years ago|reply
Between then and now, that problem was solved – you send autoreplies to the Return-Path header, which is protected by SPF.
[+] Operyl|7 years ago|reply
Mmm. I have transactional emails in my boxes that still have unsubscribe in the body.
[+] dredmorbius|7 years ago|reply
SMTP-time rejection, with a mechanism for specifying workaround ... doesn't exist, but if it did could address this failure mode of challenge-response.
[+] altharaz|7 years ago|reply
In France we have a software vendor called MailInBlack.

Their solution is exactly the one proposed in this article, where the sender has to solve a “challenge” to get in your mailbox.

If I think this approach is really effective, it also creates a huge pain for a lot of tools relying on emails such as WebEx invites or when you want to contact someone for sales.

As a result I think that this approach might be better if it was for instance triggered only on emails with an “Unsubscribe” link, or on emails with specific keywords.

[+] getpost|7 years ago|reply
This is super-annoying for legitimate, infrequent correspondents. Somebody asks me to email them something, and I need to take the time to notice and read the automated response, click to access an unknown website, and then try to solve the captcha, which sometimes requires multiple attempts. You asked for my recipe, don't hassle me when I'm trying to help you.
[+] BeetleB|7 years ago|reply
>If I think this approach is really effective, it also creates a huge pain for a lot of tools relying on emails such as WebEx invites or when you want to contact someone for sales.

I haven't used WebEx, but what is the problem exactly? Are you concerned you won't see the invite or that the sender will get an annoying automatic response?

Incidentally, this is all just a Python script. If you can construct a reasonable enough pattern for the email address, you can always have custom rules for them - it's fairly trivial. Just like I have a list where emails go straight to quarantine without producing an annoying email.

>or when you want to contact someone for sales.

My original design was to automatically whitelist anyone I send email to - in the end I didn't go that far, but it will likely alleviate this problem. My solution is mostly for my own personal email, though. If you plan to conduct a lot of business where you expect/want random people you don't know to email you, then this scheme won't work well.

>As a result I think that this approach might be better if it was for instance triggered only on emails with an “Unsubscribe” link, or on emails with specific keywords.

I just did a query. Fully one third of the quarantined emails do not have the word "unsubscribe".

[+] self|7 years ago|reply
> The whole thing probably took a few hours to write.

Did you know about TMDA before you wrote it?

https://en.wikipedia.org/wiki/Tagged_Message_Delivery_Agent

[+] BeetleB|7 years ago|reply
I think someone (you?) told me about this, but I couldn't remember the name and it's hard to construct a search query to find it.

Also, almost all their links (Documentation, etc) are not working - I think they're having server issues.

[+] arkaine|7 years ago|reply
Mailinblack does this for more than ten years now, mostly in France. This technique is called a "challenge-response" filtering. It does work pretty well with other antispam techniques. Their solution send a daily digest to the user (x per day) of quarantined emails, allowing them to liberate legitimate emails of lazy senders who do not respond to the challenge-response email. They also use outgoing emails from users to populate their personal whitelist.

Their site is not shinny but the product does the job. https://www.mailinblack.com/en/

[+] altharaz|7 years ago|reply
From my experience, it seems that the daily digest is not enough :). And they still seem to blacklist WebEx invitations, which is really weird as it is definitely a "standard" in web-conferences.
[+] ttul|7 years ago|reply
This is know as Challenge-Response and ... it doesn’t work. What happens when an automated system sends you a message? How do you handle the backscatter problem (ie when spammers spoof the sender address causing your helpful challenge messages to be sent to someone random who then complains..).

Laudable effort, but it’s not a solution.

[+] BeetleB|7 years ago|reply
Hard to say something is not a solution when it's been working for over a year.

>What happens when an automated system sends you a message?

I'm not sure I see the problem. I don't want automated emails in my Inbox. Period.

For the few ones that you'd think I want (e.g. overdue notices), they are in my "other" list - those that don't go into either my inbox or my quarantine - so they don't get an autoresponse email. But frankly, there are very few automated emails that I want.

As an example, I've set up Zillow alerts that tell me if a house in my neighborhood is on sale, or has been sold. In one sense, I do want these emails, but I simply don't want them in my inbox. They remain quarantined, and I'll see them whenever I check my quarantine pile. Most of the sites I intentionally signed up for because they occasionally will send me something useful - they all still go to my quarantine pile. I don't need them in my inbox intermingling with more important emails.

>How do you handle the backscatter problem (ie when spammers spoof the sender address causing your helpful challenge messages to be sent to someone random who then complains..).

If the spoofed email is in my whitelist, they don't get an email. If not, what exactly is the problem? That some random stranger got an email from me? That's just spam for them - I doubt they'll even notice. At best, they may realize someone is using their email address as a From in a spam email.

At least in the 1+ year of running this, none of these has been problems.

[+] nulbyte|7 years ago|reply
This is interesting. I was initially tempted to replicate something similar to try it, but then took a moment to think about what my real gripes with email actually are. Then I went to work on solving them, one by one.

First, I have a tendency to keep emails in my inbox as well, thinking I will do something with them at some point. I can't say I've ever waited a year to respond; however, I have probably waited several months before deleting (archiving, since I use Gmail) an email. This often comes after realizing that I am not, despite my earlier assessment, going to do anything about this particular email. Second, the endless notifications are abhorrent.

Today, I finally got around to adding some little used account information into my KeePass database and archived two emails. Then I realized I wasn't going to do anything with the other 20 emails in my inbox, so I archived them, too. Then I turned off notifications on my phone. I turned off Outlook's toaster at work years ago after I was frustrated with the amount of useless junk my colleagues or other work groups keep sending me. Unable to remove myself, I just decided to deal with the junk and all the other emails I get on my time, not anyone else's. I took that practice to my personal accounts, after deciding that if I won't waste my time even when I'm getting paid, I should definitely not waste my time when I'm not.

Now if I could only talk my employer into letting me disable my email account when I go on vacation, a la Daimler: http://time.com/3116424/daimler-vacation-email-out-of-office....

[+] hzhou321|7 years ago|reply
I use my own simple email scripts with simple rules and it also works! Over the time, my system has become even simpler. I used to have Procmail as mda, now simply a 5 line Perl script. I used to have spamassissin as pre-filter, now I skipped it altogether. I used to have a database for whitelists, blacklists, and filter lists. Now I simply have hand-coded rules (several) and hand edited whitelist/inbox lists. There are three areas for my emails: inbox (as well as a dozen categorized ones, important boxes also sends notification to my phone), unsorted (handful every day), and spam box (hundreds every day). I do check my unsorted box about daily; I only grep my spam box if I am expecting something and something didn't come.

Every time I got a spam leaked (into unsorted) and I am annoyed or I want certain mail go to certain inbox, I simply edit my scripts (which I know exactly where and how). It is easier than -- e.g. -- reading procmail's man page.

Specific, simple, personalized approach is surprisingly effective than any generalized approach. The latter is a hard problem, but the former often isn't. The former I can choose my own compromises.

[+] masukomi|7 years ago|reply
E-mail is currently useless for me. people have been talking about going through and unsubscribing from things.

NOT counting the spam folder i have 52,541 unread emails.

I am not going through all of that to click a bajillion unsubscribes considering i never actually chose to receive email from 99% of the people / companies sending me email.

I'd forgotten about the whitelist strategy. I'd written it off as obnoxious to the sender, but as i am reminded of this years later i'm thinking it's more obnoxious for me to never respond to people, partially because i miss their emails, and partially i don't bother looking most of the time because there's too damn much to deal with.

[+] CogitoCogito|7 years ago|reply
Mark all old mail read and start over today? Just unsubscribe going forward.
[+] xte|7 years ago|reply
My personal mail solution is composed of:

- a good taxonomy on a maildir in my home and on my IMAP (thanks mbsync+cron+a small script to watch inbox with IMAP Idle)

- a good automatic filtering/refiling (imapfilter, for now)

- a good spam blocker (spamassassin)

- a good MUA which support both physical taxonomy and a search-based one

The workflow it's roughly simple: I have some saved search in notmuch-emacs mostly 'unread', 'important', 'inbox', 'live', 'todo'. When I have little or no time I look only at unread+important, when I have time I go through the rest of unread. Live stuff are thing in progress that I have to watch but no action or todo, todo are...todo's...

[+] akho|7 years ago|reply
What I’d love to see is an email-like system where each message is accompanied by a small sum of money (1¢ or so). If the message is not useful to the recipient, they are free to take the 1¢, otherwise, if the message is useful, they wouldn’t (and it will return to the sender either when the email is replied to, or, say, in a week). This would very quickly kill all spam-type business models, and would be unlikely to cost anything for legitimate users.
[+] JeanMarcS|7 years ago|reply
I’ve made something like that 15 years ago.

It didn’t worked. People receiving the quarantine mails often misunderstood it or dismissed it without reading it.

We tried with CAPTCHA, without, nothing worked.

Our users had to spend a lot of time checking for false positives.

In the end we gave up.

Oh, and if you want to do something like that, add a reasonable delay for your response as some Spam list check for automatic responses and consider them as SPAM and will ironically block you for that ! Been there...

[+] teddyh|7 years ago|reply
> 2. Spamassassin. This worked for many years, but then a few years ago, the accuracy went down like crazy.

I switched to Rspamd and haven’t looked back. It’s much better.

[+] megous|7 years ago|reply
Bogofilter also works quite well if you have enough training material.
[+] danaliv|7 years ago|reply
I just want to be able to classify email by “sent by a human” and “not sent by a human.” My inbox isn’t a (bad) todo list, it’s a (bad) notification center.
[+] fdavison|7 years ago|reply
I filter anything with the word "unsubscribe" into a separate folder, then add a few filters to handle the exceptions.

What is generally left in my inbox is stuff directly mailed to me that deserves attention. 90% of the stuff in the unsubscribe folder get deleted without looking beyond the subject line.

[+] napsterbr|7 years ago|reply
Ah, like many people, I'm also one of those who let thousands of messages build up and respond them maybe a year after they were sent.

But my reason for such slow response, I believe, is a little bit different than most. I have social anxiety, and reading email is extremely painful to me. Not replying, but reading. I almost never open my inbox. The last email I sent was six months ago.

Sigh. Anyone with similar issues?

[+] ryuushen|7 years ago|reply
Ahhh, don't I ever. Too scared to have an engage with most online communities, let alone e-mails directed at me. I probably won't read any replies but I wanted to enjoy this brief connection with someone with a similar experience.
[+] kardos|7 years ago|reply
So your system automatically sends back messages to the non-whitelisted addresses. Those messages will eventually be classified as spam by big operators such as gmail. How do you prevent that from reducing your "email reputation" and ending up on spam email blacklists?
[+] BeetleB|7 years ago|reply
To be frank, this is a concern to me, and I think I'm already on some spam lists. Recently two folks using Gmail told me my (regular, written by me) email showed up in their Junk folder. I checked others who use Gmail and it showed up fine for them.

I may end up using a separate domain for those emails. I'm pretty sure that's a weak solution.

[+] lgeorget|7 years ago|reply
On my own domain where I've set up a mail server, I've never received that much spam. I have the impression that people are more civilized now than in the late 2000s with respect to emails. In my case at least, unsubscribe links work reasonably well.
[+] cm2187|7 years ago|reply
The risk is some dodgy website (small shop, forum, etc) that has been pwned and which leaks your detail to spammers, not so much the company you give the email to.
[+] antirez|7 years ago|reply
Not sure how spam is the problem. Mostly solved by good filters. The problem is the amount of legitimate personal emails that one gets, creating the TODO list problem Paul Graham was talking about in his quote at the top of the blog post here.
[+] spac|7 years ago|reply
This is exactly the problem: Some emails you can’t respond because they require taking actions, so they stay in the inbox to remind you. And then it’s the broken window principle... one cool thing I recently found is that on Trello you can have board specific addresses that create a new card when an email is sent to them. This allows me to move the TODO problem outside the inbox.