[bartbutler]

This is Bart, Proton CTO here. For clarity, the issue mentioned here only impacts Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email.

The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural issue in a library that Bridge uses to implement IMAP. When there are network issues, this library returns errors to email clients.

Unfortunately, there are hundreds of email clients, and some email clients don’t handle errors properly, and this leads to desynchronization.

Our error tracking shows this does not happen often (1-2% of Bridge users) and the symptom is usually incorrect display of messages or read/unread status which is fixed with an inbox resynchronization. There are cases where a combination of a desynchronized mailbox and a specific series of user actions can lead to accidental email deletion, but this is far rarer than desynchronization. Our implementation tries as hard as possible to avoid this. If you find you are missing an email, our implementation works around the issue by placing it in a users’ All Mail folder.

As Bridge is open source, updates on this issue have always been publicly posted on GitHub. Addressing this issue at the source requires replacing the core IMAP library. Unfortunately, there are no FOSS IMAP libraries that are sufficiently well maintained. Therefore, the solution is to build our own IMAP library called Gluon, which we have been focusing on since this issue was reported to us. You can follow the progress of this open-source project here: https://github.com/ProtonMail/gluon

We are not refusing to fix the problem. The only possible solution is writing a new open-source IMAP library which we can maintain ourselves to ensure this class of errors cannot occur again. We have doubled the size of the team working on this this year so it is a priority for us.

We’re confident that this addresses the main sources of desynchronization and will be available in the beta version of Bridge by the end of the year.

[cryptonector]

The fundamental problem is that `UID`s in IMAP kinda suck because assigning persistent, unique IDs to emails in a store is a hard problem because doing that for mbox- or maildir-like stores is hard because those predated any notion of remote email access protocols.

Thus in practice IMAP servers generally assign `UID`s ephemerally per-session, which means that clients can't rely on the stability of `UID`s, which means that clients have to re-obtain `UID`s before operating on emails via IMAP even if they have cached those emails locally. `UIDVALIDITY` exists to help clients cache and invalidate `UID`s. The RFC has text about this.

A bridge from IMAP to something else (which is basically what every IMAP server ever is) needs to deal with this. To make `UID`s stable requires keeping state.

Clients should really not assume stable `UID`s. Instead clients should `SEARCH` or list to get [temporarily] valid `UID`s then use those to delete etc.

[survirtual]

Why is this complex…just take a sha3 hash of the email body + timestamp, then b58 it or format any way you want. Instant uuid per email.

[twleo]

> Bridge is open source, and as a result relies upon open-source components

I don't get it. Bridge is open source does not imply it should relies upon open-source components.

> Addressing this issue at the source requires replacing the core IMAP library.

Why building an IMAP library from scratch instead of fixing/forking go-imap? Even a temporary fix to go-imap when you are developing gluon? Another repetitive work which does not guarantee the mentioned issues will be resolved completely.

[cryptonector]

> I don't get it. Bridge is open source does not imply it should relies upon open-source components.

It could be open source and depend on proprietary components, but then the public wouldn't be able to build and use it from source.

[ploum]

What I don’t understand is why there’s no effort from Proton to expose the underlying protocol between Protonmail.com and the Bridge.

This protocol should be an open source effort, allowing mail clients to implement it and other provider to implement it on their own server.

This could clearly be a major move, making unencrypted IMAP a thing of the past, allowing direct competitors (tutanota? Mailfence?) to collaborate on the bridge and on the ecosystem and targeting directly the only competitor worth talking about : Gmail.

[tacker2000]

I think this happens to far more than just 1-2% of users, and re-synching the inbox every few weeks is nothing a paying customer should have to do.

These issues have been around since I started using bridge 3 yrs ago. So im sorry but my patience is running out soon.

I just renewed my yearly membership, but if these bugs concerning the MAIN FEATURE of proton arent taken care of in the next few months than i will be looking for alternatives.

[31337Logic]

Thank you for the prompt and clear explanation.

[XCSme]

Please add API support to Proton Drive, or any way to upload files automatically so I can back up my data on Proton Drive.

[euroderf]

Nobody else has asked it, so I will: DOES Proton Bridge work OK with mutt ?

(If so, I will make the jump from the free plan.)

[jrootabega]

I think you would experience the same problems as any IMAP client would. And seeing how this bug is being handled, it doesn't look like paying for bridge access is a good bet to make, IMO. You'll never be confident that the bridge is not going to have a problem. How many emails would you be willing to lose or have trouble with?