top | item 32537565

Why don’t we do email verification in reverse?

298 points| splash123 | 3 years ago |blog.yossarian.net | reply

321 comments

order
[+] simonw|3 years ago|reply
A general rule of thumb I've built up over the years: resist the temptation to innovate around login!

Look at the most commonly used flows that are not obviously terrible and try to implement as close a match to them as possible.

When I've tried to innovate around login in the past I've found that any clever ideas I come up with inevitably run into road blocks pretty quickly.

Here's one example: why have a separate login form from a signup form? They both involve asking the user for an email and password, why not have one form that can do either depending on if the user's account already exists?

I quickly learned why: if you don't understand the user's intent when they submitted that form, you can't show them error messages that match their mental model as to what is going on. This makes for an incredibly confusing experience the moment you step off the happy path.

----

That's not to say it's not interesting and valuable to think through new login mechanisms. They're very interesting design challenges! But its good to be prepared to find out that vanishingly few innovative ideas will work out as genuine improvements.

[+] PaulHoule|3 years ago|reply
That’s what I used to believe. Circa 2000 the advice was ‘do what Yahoo! does’ and that was good, but when I revisited that in 2006 I found that Yahoo’s signup and login process was deformed by the need to discourage users from responding to phishing attempts —- a bank needs to be concerned about this but it will hurt the login for other sites that have higher iq users and aren’t big enough that spamming everybody is a good tactic.

I worked on one site (a chat service for Brazil) that had a 20% success rate for the login flow at first that we got up to the high 80s by tweaking this and that. Later I worked on one aimed at scientists that was in the low 80s before optimization.

Some tweaks from a long time ago are:

1. Put something like ‘we sent a message to <b>[email protected]</b>‘ on the post-registration form where they are likely to see it if they make a mistake. Today forms are likely to ask for your email twice to catch this, annoying as it is.

2. Put a form field for the registration code on that form, we found that oriented people towards looking for a registration code. We still sent a link to click on but we found that users preferred to type the code into the form which was fine with us because they succeeeded.

3. Always store the current URL that the person clicked ‘register’ on and store it in the database so they end up where they started after the process is complete.

[+] melvinmelih|3 years ago|reply
When I started on my first project as a junior engineer I thought I was being clever to build an app without any passwords: every time you wanted to login you received a new email with a link to log you in. Technically, this worked great but after a while I received many complaints from frustrated users who kept looking for the “signup form”…
[+] ozim|3 years ago|reply
Super typical developer thinking :) two pieces of code look the same but are not doing the same things.

I usually see it when people implement add/edit for some entity, that should be separate forms but are crammed into one to be "DRY".

[+] nsgi|3 years ago|reply
The standard login/sign up form is broken, though. People will just use the same password across websites or write it down. You can't win
[+] conductr|3 years ago|reply
I once built exactly what the author proposed and it worked but confused users a lot because they’d have never experienced it before.

There was a lot more leak in the signup funnel as people would just leave once asked to do something besides giving their (usually auto filled) info.

So I tend to believe trying to generally match the flow they are accustomed to is the best way, even if you think it’s not the best way.

[+] marcosdumay|3 years ago|reply
> When I've tried to innovate around login in the past I've found that any clever ideas I come up with inevitably run into road blocks pretty quickly.

Either that or you are throwing roadblocks in your user's way. Even if there is no reason for the most common UX to be the way they are, it is what users are used to, and know how to deal with.

[+] LinAGKar|3 years ago|reply
Sounds like a lot of people would accidentally create new accounts.
[+] HWR_14|3 years ago|reply
For your example, this seems bizarre. Every "sign up" process I can think of ends with you being signed into to your new account. Every "log in" process includes a button or link that says "are you a new user, sign up here". You chose a very streamlined thing to worry about.
[+] neetcode22|3 years ago|reply
100% agree with you. I've gone through the entire journey of experimenting + building my own authentication to using a reliable open source solution.

Been using this passwordless login recipe by supertokens for my recent projects: https://supertokens.com/docs/passwordless/introduction

Re-inventing the wheel calls for a lot more than usually anticipated. Best to keep it simple - and if you do end up innovating might as well open source it or contribute to an already popular OS solution.

[+] kqr|3 years ago|reply
At the same time, the whole idea of a shared secret instead of asymmetric crypto is source to so many leaks that I wish we would pressure services to not use password-based authentication as their default.
[+] djbusby|3 years ago|reply
Haha, the mistake I've helped implement thrice. Two forms, always.
[+] samus|3 years ago|reply
Indeed; as soon as the article stated that SPF and DMARC are required, it stopped sounding like a good idea.
[+] neximo64|3 years ago|reply
Some reasons:

- Not everyone will have set email up on the phone.

- If a laptop is used it adds more complexity if the email is not set up on the device, (or e.g if Gmail is used via a web browser)

- Not everyone will use the Mail app as default and will need to pick the right app

- There may be multiple emails and additional complexity to pick the right 'from' if there are multiple emails being used.

- It is a test of the SMTP server, not the ability to receive email.

- The email needs to match exactly what is being used to sign up when sending the email. With an email inbox [email protected] can be used as an example.

[+] simonw|3 years ago|reply
Right: this would fail for me on my laptop, because I use gmail for email and don't have anything setup such that a mailto: link would compose a new email from my gmail account.
[+] 9935c101ab17a66|3 years ago|reply
Well said. Particularly the point around people not having email set up on an “OS” level. There are a lot of people who already struggle to effectively use modern computers and smartphones, and you’ll run into all kinds of weird edge cases. For example, I previously helped a friend who had just been using the gmail website on their smartphone for years because they had forgot their password but Safaris keychain had saved it so they could auto fill the password.

I’ve encountered countless other scenarios where a user is using software in “non-ideal” way because it works for them or they don’t know any better.

[+] jcynix|3 years ago|reply
> Not everyone will have set email up on the phone.

Exactly. And not everyone accepts HTML E-Mail, i.e. I really hate these "endless" state carrying URLs, which wrap around in my 99 chars wide terminal windows. A short SHA-xxx key should be sufficient, "dear" web form designers.

And last but not least I use greylisting, which still is a useful tool to suppress spam, but delays confirmation emails.

[+] raverbashing|3 years ago|reply
Good summary

Using mailto: is just asking for trouble at this day and age.

[+] radu_floricica|3 years ago|reply
I... don't particularly get it. He mentions spoofing, he writes a page about how spoofing works... but says a lot less about how it actually impacts his solution or how to fix it.

Besides the fact that implementing a new security scheme means you have to think through every possible path and can be sure you're still missing a few, there are two major issues:

- not everybody has SPF or DKIM, and definitely not everybody has DKIM.

- both of those authenticate the domain, not the username. Within the local org network I can probably spoof email usernames without much effort.

Plus... I don't think mailto links really work universally. I remember the last time I clicked one it opened an unconfigured Outlook, even though I use gmail.

[+] woodruffw|3 years ago|reply
Author here. The risk with spoofing is that someone might register an email address that they can't actually send mail from.

You're right that SPF and DKIM are not universal (besides not being strictly for user authentication); this scheme would require a domain to have both in order to be secure and would require some kind of policy attestation around email local parts, which would exclude some email providers and some users. That's why it's "a random idea I had on the subway," and not something I'm doing in production :-)

[+] cwilkes|3 years ago|reply
Yeah I don’t get it. Is this the reasoning?

Email is slow to get to me

I use neomutt (there are dozens of us, dozens!) and so links sometimes are wonky.

So let’s use SPF and DMARC, something totally outside of a regular email user’s control, to do this task. And hope that it works.

[+] infensus|3 years ago|reply
When opening for the first time, Gmail asks to be registered as the mailto handler, but I guess people just hit "no" not knowing what's that about.

And there's probably a lot of web-based clients that didn't care to implement this, so yeah, if you want to use mailto links, you better have a big and patient customer support team at hand

[+] ShowalkKama|3 years ago|reply
> - not everybody has SPF or DKIM, and definitely not everybody has DKIM.

A correctly configured mailserver should reject your email (or mark it as spam) if you don't use both DKIM and SPF. It's safe to assume 99.999% of users use both.

[+] perlgeek|3 years ago|reply
Some potential problems:

* not everybody has an email client set up that works with mailto: links

* sending from an address and receiving emails for an address are very different, and they don't map 1:1

* If you rely on users sending mail to you, the user doesn't know if / when their email is received and processed. They send the mail, try to login, it doesn't work. They have to wait until it's processed, and then forget about your service. OTOH if they receive a mail from you, the mail in their inbox reminds them that they signed up, and as soon they receive it they can use their account.

* I doubt that DMARC/DKIM/SPF are actually deployed as widely as OP thinks, and excluding users based on some (to them) obscure/unknown feature of their email provider is bound to generate frustration. If they self-host their email, your support might end up debugging / having to help them with their email setup.

* There are some subtleties you have to get right, like verify the MAIL FROM, not the From: header, etc. Email is actually a mine field when it comes to security (it's not specified what should happen if a header is sent twice, so some attackers try to fool virus scanners by finding combinations that the scanner interprets one way, but the recipient interprets another way, etc.) Worse, if a verification fails, it's your responsibility to debug it, and you might have to explain the subtleties to a non-technical user.

[+] indymike|3 years ago|reply
> not everybody has an email client set up that works with mailto: links

Mostly bad chrome configs where you need to browser plugin to lauch chrome.

>I doubt that DMARC/DKIM/SPF are actually deployed as widely as OP thinks, and excluding users based on some (to them) obscure/unknown feature of their email provider is bound to generate frustration.

We're at about 30% with DMARC and SPF being deployed in about 65% of email servers. That said, a lot of the really big ones (google, MS, etc...) are in the 30% so are a lot of control panel hosts. Where the slow uptake is, as always is corporate IT departments. Doing the whole DMARC/DKIM/SPF dance isn't hard to do and really puts the clamps on spoofing.

[+] wootest|3 years ago|reply
Ignoring every other benefit and concern, verification in the way proposed is a bad idea because part of the verification process in most cases is verifying that the service can send messages to you that actually get delivered. If you do this and then send the first "log in link" email which gets held up as spam or putatively malicious since some server has the temerity to not be located inside the US, doesn't have a DKIM signature, etc, you haven't really verified everything that you need to know. Of course, those things can change at the drop of a hat anyway, but I'd rather have verified that 1 time than 0 times.

If all you do is use it for login and will never need to send a message, then fair enough, the email is just essentially a random string you can prove ownership of, and your ability to send messages that will be delivered to the corresponding mailbox is incidental. But that's not a general enough conclusion to hold for why "we" (all cases) should do it that way.

[+] throwoutway|3 years ago|reply
In most cases, I don’t want the service to send me emails (spam)
[+] gingerlime|3 years ago|reply
Besides the security and usability problems others mentioned, I also highly doubt it will improve confirmation conversions.

One thing that helped us improve confirmations -- we A/B tested it and confirmation rates increased ~8%: send a 4-digit confirmation code rather than just a link.

It's easier and more familiar on mobile, especially if you see the code on the push notification, so don't even need to open the email. I suspect it's less likely to land as a promotion email in Gmail, because of the code/semantics. It's quite a fun experience as well in my opinion. It seems like more work, but entering 4 digits on the same open tab is easier and nicer experience than clicking on an email link.

There are some security considerations. A short code can be brute-forced, so you need to rate-limit the number of guesses.

[+] ozim|3 years ago|reply
Not having a link in email is important. I believe most of the times email from my systems go to spam because there is a link in it.

I don't have any hard data but I can imagine that if email contains link it will be checked by more rules in heuristic checks. If there are no links - most likely it is not spam.

[+] t0mas88|3 years ago|reply
I've seen similar numbers on an ecommerce site. The theory there was the people may be on a device where their email isn't logged in. So doing that and then clicking your link is a lot of work and could run into even more hurdles like Gmail 2FA popping up because they don't use that device frequently.

While the 4 digit code is easy enough to read on their phone in a notification and then type into the other device.

[+] simlan|3 years ago|reply
Agreed. Usability on mobile is best with the short codes. Even 2 FA is fairly efficient that way as long as the messages are designed to show the code in the preview.
[+] hollerith|3 years ago|reply
>send a 4-digit confirmation code rather than just a link.

The wording is unclear, so I will ask for clarification: Does the improved email contain both a 4-digit code _and_ a link (URL)?

[+] IncRnd|3 years ago|reply
This is not well reasoned. Sending emails can be easily spoofed, because sending doesn't fully check the identity of the sender as being in control of the email account. There is discussion of various technologies like SPF or DKIM, but those are not universally applied. When they are applied, there isn't universal quality in their application.

The crux of verification using an email account is that the person _controls_ the email account. That means the person can receive an email at that account. If the person cannot receive an email at a specific account, that account cannot be used as their identity at the web server.

[+] mgarciaisaia|3 years ago|reply
I'd argue the crux is about proving that Me The Service can send You The User whatever info is needed (password reset, notification of planned downtimes, pricing changes, warnings about service abuse) to this address - and you'll get it.

I don't care about you "controlling" an email address - you can easily get fake ones for free. It's about both parties having agreed _at least once_ on a way for the service to communicate important stuff to you.

[+] woodruffw|3 years ago|reply
> This is not well reasoned. Sending emails can be easily spoofed, because sending doesn't fully check the identity of the sender as being in control of the email account. There is discussion of various technologies like SPF or DKIM, but those are not universally applied. When they are applied, there isn't universal quality in their application.

I bring up both of these at the end of the post, under "potential limitations." I'm very aware that SPF/DKIM don't do identity authentication, and that they're not universally applied!

This post is best read as a random thought, not an exhortation or formal argument. It says as much in the first sentence.

[+] stormbrew|3 years ago|reply
I would hate this, and probably instantly bail out of any flow that tried to force it on me.

Being able to receive mail on an address should not necessarily imply a capability to send from it. This scheme rules out many actual or potential schemes for email anonymity, especially in a world where very little email is still transactional relatively speaking.

I do not give (almost) any website an address I send normal email from, nor do I want to. And my default configured client for mailto:, if I even have one on the device I'm using, is not going to be set to the single use mail I want to sign up with.

[+] cuteboy19|3 years ago|reply
$Work does not allow outgoing (out of domain) emails but *Requires* signup to a bunch of websites for stuff like provident fund management and stock compensation. This kind of workflow is not possible for those cases

They are exchanging inconvenience for lack of functionality

[+] LudwigNagasena|3 years ago|reply
>…and here it is in neomutt, which helpfully “renders” the HTML with lynx:

>…

>But that’s a solution born from practice: the fact remains that HTML email doesn’t generally degrade gracefully in clients that don’t support images or CSS.

I hope that companies don’t waste their time at such edge cases that are irrelevant for 99.9% of users such as Lynx graceful degradation. It reminds me of the stories about clients that want their website to look nice in IE6.

[+] hericium|3 years ago|reply
Multi-part MIME emails with both plain text and HTML content used to be a standard[1] or at least a good and common practice. Not including plain text version or making it "can't see newsletter? click here for web article" is an ongoing process of moving users from unsnoopable email to invigilated web. It's invisible to dumbed-down users, possibly saves corpos some money: devs can learn something that benefits business more and, depending on email volume, business may decrease spending a bit by removing few kilobytes of text from each message sent.

Imo a standard of sorts, being removed due to lack of knowledge or for money (savings on data transfer or profit from data gained from users reading "web version of email" and email clients loading remote content) doesn't make it an "edge case". The edge case here (necessity to parse and read HTML in pure email client) is an effect of businesses and developers not following standards in the first place.

Also, IE6 overstayed its welcome not due to edge user cases but due to corporate savings as well.

[1] https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1

[+] peterkelly|3 years ago|reply
The whole point of email verification is to verify that the user is able to receive email at the given address. Aside from the other issues mentioned here, the proposed solution does not achieve this.
[+] kahnclusions|3 years ago|reply
The ‘mailto:’ thing is a dealbreaker.

A _significant_ amount of users, especially non-technical ones, won’t have a properly configured email client. They’ll click signup, it’ll open Outlook or Apple Mail, and then they’ll give up on your app in frustration because they can’t register.

There’s very limited room for shifting the signup paradigm in a way that won’t leave N% of your potential users unable to sign up.

[+] Merovius|3 years ago|reply
> In both cases, the proof is the same: “email verification” means “the user controls an email address.”

No. In one case the proof is "the user receives E-Mails under this address" and in the other it is "the user can send E-Mails under this address".

The service is generally interested in proving the first. For recovery, service-related notifications and for unsolicited marketing. So all the things mentioned in making the "normal" flow undesirable, are actually exactly what the service owner wants to protect against. If their E-Mails don't get to you because they end up in spam or just get dropped by a misconfigured Mailserver, they can't spam you and you won't be able to use this account for recovery.

I agree that the reverse flow is easier (though TBF, mailto links also require the user to have set them up to work, which not all users have on all their devices). But I don't think that's where the incentives align.

[+] throwaway892238|3 years ago|reply

  1. Attacker goes to website, initiates signup up for an account
  2. Website generates mail link for attacker to send
  3. Attacker uses mail link to spoof email
  4. Account is opened for the attacker using email address
This spoof attack works for both account creation and password reset (account takeover).

This attack is impossible if the website sends a verification email to the user's inbox which only the person who controls the inbox has access to.

Hence, while the proposed method is interesting, it exposes the user to an unnecessary attack vector.

[+] fefe23|3 years ago|reply
This suggestion makes things worse.

If you make a service and ask for email validation, you want validation that the user has that email address. Since anyone can spoof e-mails from any address, getting an email from the address is not validation for anything.

You need to send an email to the account and get a response to validate the email address is real. Otherwise it's trivial for me to go to any web site and pose as you. All I would need to do is spoof your email address.

Edit: I agree that the current state of affairs is unsatisfactory :-)

[+] edmundsauto|3 years ago|reply
So instead of an HTTP syncronous POST request, you want to trade it for a brittle implementation of a to-be-implemented-standard that requires the sending email client to be on point? And that I will wait while my outgoing email server isn't delayed?

And what if I use my ISP provided email, and move? Am I just locked out of accounts?

I appreciate the thought process of rethinking our assumptions. But this is not fruitful - the tradeoffs are trivially obvious and unacceptable failure modes.

[+] bergenty|3 years ago|reply
This isn’t better. I don’t have a default mail client and a lot of people don’t. You could open up the browser page to login but by the time you’re logged in all context is lost and you won’t get the pre setup outgoing mail ready to click send.
[+] kazinator|3 years ago|reply
This is quite wrong. The purpose of e-mail verification is to validate that the user creating the account controls the specified e-mail; that he or she can receive at that address.

This author has created a silly flow whereby the user proves that they are somehow able to use e-mail to reflect back some authorization cookie to the server. You can do that without even having an e-mail address; any Internet-connected host can send e-mail, using any sending identity it wishes. The fact that you can contact my server via HTTPS and via SMTP, and relay some information between the two, means absolutely fiddlesticks.

> The “reverse” flow should make intuitive sense: we’re proving that the user controls the specified email address by challenging them to send us an email from it.

Right; here is the core misconception on the part of the author: that receiving an e-mail from someone proves that they control a certain e-mail address. (Imagine the flow being used for password recovery, yikes! Anyone forging an e-mail from you gets your account?)

The second problem with this idea is that it increases the server complexity. Now it has to process e-mail. Conventional e-mail validation doesn't have to receive e-mail; all it does is send confirmation mails. Which, I repeat, any Internet-connected host can do; and it can do so without becoming a mail server.

A third problem is user experience. It complicates things for the user. The handling of mailto: URLs is tenuous. Not everyone has it set up correctly. Many users have some webmail account like Gmail, but when they click on some mailto: URL, some default application comes up that doesn't use their Gmail account. They don't use that application and are prompted to set up that application.

[+] nickjj|3 years ago|reply
I like that there's still options out there around authentication that I haven't seen before but this method is hostile towards folks without a dedicated email client. Personally I only ever login through web mail and have no client so `mailto` isn't a shortcut.

I still think magic links are a good compromise. With a user / pass you still need to verify your email as a separate step. A magic link sends you an email and you're technically verifying your email every time you use it, so it removes an extra step.

It's also really useful for being able to login from multiple devices without needing to keep your password synced with a multi-device password manager. That's especially handy on mobile because even if you're around your desktop with your password manager who the heck wants to manually type `pm'TWZS"$G39c(Es:k@v3-*IhP#DUbows:=='"Z],Ud"R8XO` as a password into your mobile phone?

Lastly, it lets you put the burden of MFA on your email account. If your email account is protected by MFA then your app using magic links is protected by MFA. It means users don't need to give out their phone number or hook up another site to their authenticator app.

I really don't understand why so many folks hate magic links. They solve a number of problems in a pretty reasonable way at the cost of having to wait 5 seconds for 1 email to verify your email at which point the site can set a cookie for a year and you don't have to deal with logging in by email for another year.

You can also protect sensitive actions by requiring folks to verify their email to replicate how a site might ask for your password to do something like change billing details or whatever makes sense for your app.

[+] brightball|3 years ago|reply
I actually tried something similar to this on a small social networking service about 5 years ago. There was a requirement for people to interact with their groups via email by replying to other emails or by sending a message to an email address for the group.

For both of these, I made the case that we should require the incoming emails to pass either a domain aligned SPF or domain aligned DKIM check (essentially what DMARC would do).

Since most people are going to be using a 3rd party email service, this just works by default. If they aren't, we shouldn't have any issue with forcing this requirement as it meets absolute basic email standards.

The higher ups let me do it and we ended up creating an instructional email that would bounce back to failed messages with the reason why, an explanation that we couldn't risk someone being impersonated on the platform and a suggestion that they use the app or website to interact until the issue could be resolved.

In the short time that I was there after that feature launched, we never ran into a problem.

[+] clement_b|3 years ago|reply
Guy calls current flow clumsy. Comes up with alternative reliant on mailto :)