Stalwart is, from what I’ve read, an excellent JMAP server.
JMAP is, from what I’ve read, a great protocol for building an E-Mail (and now also others) client on top of.
Since I would like an innovative way to access my E-Mails, but do not want to self-host, I would find it interesting to use Stalwart as the server component of an E-Mail client: Data is somehow synced into Stalwart via the “ugly” protocols and I get a nice API to build an elegant client on top of.
My basic research shows that something like IMAP-IMAP sync seems to be a thing. Has anyone done something like this, perhaps even with Stalwart? (this of course grows in complexity for each new protocol to be proxied).
I believe having this kind of setup easily accessible could jumpstart a new generation of E-Mail clients on top of JMAP because it (relatively elegantly?) circumvents the chicken-egg problem by allowing all existing IMAP mailboxes to be accessed via JMAP.
I would just like to highlight the "excellent" bit here. It is a tour de force, a masterpiece. Beautifully factored software. They've done an excellent job incrementally building trust and delivering. Really cool project, and I'm so glad it exists.
I also did not realize it is… primarily developed by a single person? The author. Incredible:
This should be pretty straightforward to do with an IMAP <-> IMAP syncing tool, like mbsync [1]. You'd run it periodically in the background to sync the remote IMAP to Stalwart's local IMAP server, and Stalwart can then automatically serve that via JMAP, doing the translation internally.
I was originally thinking you'd need to go remote IMAP <-> maildir <-> Stalwart IMAP, which would be really complicated, but I think the IMAP <-> IMAP should work fine.
Reading weirdly many “but there are no clients” posts. Of course something has to come first. You can’t really develop a client without a server implementation and Stalwart is essentially the first server implementation of JMAP.
With Stalwart in place, there’s finally a reason to develop a client for JMAP.
I hope y’all are aware that Mozilla’s new mail service will use it, so that is likely going to give JMAP a big push!
I'm cautiously optimistic that Stalwart is a game-changer. I would normally stick with a simpler platform (like mox) if I wasn't interested in the groupware space. I've previously tried Nextcloud and SoGo and left disappointed, and have more or less been waiting for a project like this to come along.
Nextcloud was such a terrible experience for me (the file sharing/storage was good, but the groupware aspect was incredibly buggy). But knowing that Nextcloud is partnering with Stalwart to hopefully overhaul their stack, Opencloud is developing their JMAP integration, and Mozilla/Thunderbird is using it too (they already have a webmail in development here: https://github.com/thunderbird/stormbox)... we might finally see some exciting development in this space. And now is also a ripe time, as there seems to be a perfect storm of people wanting to get away from Big Tech platforms.
While JMAP seems to scratch every itch of a sucker for proper web API design, I’m wondering if the design space for new protocols should really be constrained to layers on top of HTTP. Is there really any new-ish binary protocol these days?
Stuff like file sharing or groupware, mail, calendars, and so on—these things could be a lot more efficient and don’t really need the overhead of JSON as the message interchange format, IMHO. Then again, a lot of solid thinking went into these things, so there probably are a lot of good reasons that I’m not aware of.
Email was never a binary protocol. Notoriously so, it's why MIME types and MIME encodings get so complicated.
Most of the "old internet" protocols (email, FTP, even HTTP itself) were bootstrapped on top of built-mostly-for-plaintext Telnet. HTTP as the new telnet has a bunch of improvements when it comes to binary data, request/response-based data flows, and some other considerations. HTTP/3 is even inherently a binary protocol, it's lack of "telnet-compatibility" one of the concerns about switching the majority of the web to it.
vCard/vCal/iCard/iCal were also deeply "plaintext formats". JSON is an improvement because it is more structured, even more efficient, than those predecessors. JSON may not look efficient, but it compresses extremely well and can be quite efficient in gzip and Brotli streams.
I feel like "JSON over HTTP" is a subtle improvement over "custom text formats over telnet", even if it doesn't sound like "binary protocol efficiency" at first glance. Especially as HTTP/3 pushes HTTP more efficient and more "binary", and arguably "more fundamental/basic" with HTTP/3 even taking over more roles in the TCP/UDP layer of the internet stack. (Telnet would never try to replace TCP.) HTTP isn't the worst bootstrap layer the internet could use to build new protocols and apps on top of. Sure, it would be neat to see more variety and experiments outside of the HTTP stack, too, but HTTP is too useful at this point not to build a bunch of things on top of it instead of as their own from-scratch protocol.
> I’m wondering if the design space for new protocols should really be constrained to layers on top of HTTP
It shouldn't. For some cases it helps, but other times it doesn't. Sometimes it helps but there would be better ways to do it, making it on a simpler protocol or making an entirely new protocol (which might or might not use TCP; sometimes it is better to use TCP and sometimes not) depending on the specific case.
> Stuff like file sharing or groupware, mail, calendars, and so on—these things could be a lot more efficient and don’t really need the overhead of JSON as the message interchange format, IMHO
I dislike JSON. I think it has many problems, and that DER is a better format.
(There are also the "small web" protocols such as Gemini and Scorpion and Spartan and Titan, which avoids some of the complexity of HTTP; I had considered using DER-over-Scorpion rather than JSON-over-HTTP. It is also possible to use SSH, although SSH does not have virtual hosting.)
How much overhead do you think fetching emails over http takes? Fetching text documents with HTTP seems like the prime usecase for it. And it's the only way you could have something that works in web clients.
I'm struggling to think of any real benefits to not using HTTP other than it would be more interesting.
> Is there really any new-ish binary protocol these days?
HTTP/2 and HTTP/3 are binary protocols. And if you replace the JSON with CBOR, then even the payload becomes binary.
The reason for using HTTP is that the semantics are right. HTTP is a state transfer protocol, and ultimately, that's 90% of what you need for sync.
The other 10% is for subscriptions, updates, with versioning, and patches. You can get these by adding the Braid extensions (see braid.org) which upgrade HTTP from a state transfer to a state synchronization protocol. (I work on Braid.)
People think serializers are easy, when suddenly this makes a whole raft of choices your problem. You can write your own, and now you have two problems; or you can build on top of capnproto or grpc or, if you really want to get retro, ASN.1 and inherit the problems of your serializer framework.
JSON by comparison has simple, obvious limitations that more people are familiar with dealing with.
There's also the tendency to tie your protocol to implementation. The Microsoft Exchange "protocol" didn't get reverse engineered for so long because it's basically the COM structure of Outlook fed through (if I remember rightly) DCOM-RPC.
My hot take is that if the web had been based on binary protocols rather than HTML and JSON, the fragmentation of tooling that can instead be standardized as view-source and network-tabs would have decimated (as in removed 90% of) the hackers that make up this community, and software engineering in general.
There's no magic. Nothing sacred. Nothing that you aren't allowed to understand, intuitively. Nothing where you aren't allowed to imagine "what if it also had X?" The web is yours. The computer is yours. As an industry, we burn some incremental percentage of bandwidth to give you the keys to the kingdom, and to allow you, new developer, to be one of us.
In an age when LLMs feel like magic boxes to tech-minded people new to development, we need this more than ever.
JMAP is only JSON over HTTP because that's what all the libraries support. Any data format which provides hashes and arrays would work fine, over any transport. So you could use CBOR or protocol buffers or whatever, over any channel you like.
I really hope Fastmail implements the JMAP spec for calendars and contacts soon. They’ve had the mail part of the spec implemented for a while, but it still requires CardDAV/CalDAV for contacts and calendar access.
We're working on it. We still use an unholy set of earlier versions of JMAP internally for our contacts and calendars; in particular the caldav_sync code is gnarly - I wrote it over 10 years ago when I knew less about calendars than I do now! It's still using an earlier branch of what became the perl Net::CalDAVTalk library interally, even though our frontend API is an almost-up-to-date version of what will become the JMAP Calendars spec eventually.
The downsides of developing and testing this stuff as we were writing it up!
We've finished rewriting the objectid generation to give smaller sized and more sortable IDs (they're an inverse of nanosecond internaldates now, plus some extra magic on the low bits for IMAP appends to avoid clashes)... which we wanted to speed up and reduce disk usage for the offline mode.
Next up is indeed updating to the latest spec on calendars and contacts. Files might take a bit longer, I really want to do some desktop clients for the files system, we have a really nice files backend at Fastmail which is only accessible via our interface or WebDAV right now.
What good is a protocol like JMAP as long as common clients like Thunderbird, K-9 Mail, the iPhone email client and others don't support it? Without some concerted effort it will never take off. Then there is the question of what problem it solves that isn't already solved by existing solutions.
What next? Replacing Sieve with something cumbersome, but JSON based?
There is no good desktop implementation of MUA with old technologies (IMAP, Sieve), will all this JMAP help?
I don't think so.
What is profit to have good server with new good (assume it is good, I'm not sure, but lets assume) protocols without good client?
IMAP4 is underused by modern clients: it allows to effectively store client configuration on server, nobody implements it on client side. It allows to configure per-folder Sieve scripts, nobody implements it on client side. Nobody implements good Sieve client (with folder name autocomplition and such) even for global script, not to mention per-folder ones. Heck, there is no good Sieve editor! (I know about Sieve client built on Electron, it is not good, it is incomplete and buggy).
Servers are solved problem (sendmail, exim, postfix, dovecot, cyrus). Clients are not, they stagnated at the moment GMail was announced.
Solved problem? Dovecot can't handle UTF-8 to this day. Things like Stalwart do try and change that and actually support non-legacy features. (Though I think Stalwart doesn't do IMAP NOTIFY either.)
Clients on the other hand have actually kinda moved forward, Apple Mail works with IMAP servers and offers features that people only got with Gmail before. But there are many other examples as well.
> What is profit to have good server with new good (assume it is good, I'm not sure, but lets assume) protocols without good client?
You need both. You could say, what profit is a good client without a server? By that reasoning, we never stake a step forward without a complete solution.
Now a better mail implementation is just a client away.
We need better client support for JMAP. Apple Mail, Thunderbird, Outlook (as if), and so on. I'm surprised some of the smaller ones like Canary or Spark don't implement it as a product differentiator.
Outlook ? Lol. In 2 years-time Outlook will only connect to MS365 and that's it. You're betting MS will switch to JMAP ? Lol again.
P.S. ("New" Outlook already only connects to MS365 servers and then stores your credentials and data on Azure, while they proxy to your actual IMAP/SMTP server )
I’ve got a friend who’s been pitching me on building a new email client for years. “I’ll do it if we exclusively use JMAP.” “okay does that include Gmail and Apple/iCloud accounts?” “Nope.”
I could sort of see dual-supporting Gmail's proprietary API and JMAP, but unless the #2-5 competitors support it… what’s the point? (sorry to put on the pessimism hat)
There isn’t really a great motivating feature or use-case driving client or server adoption.
To be honest, I’m not sure why end-users would want JMAP for e-mail access.
It would be interesting if they do successfully roll out all of these additional RFC proposals providing a cohesive “groupware” protocol covering calendering, contacts, file shares, etc, we see notable server implementations, and interest is enough to drive client support.
It’s such a breeze to self-host your own email server using Stalwart. It has been a new era for email self-hosters like myself since these kind of fully integrated email servers like Stalwart appeared. Another good one but not as actively maintained is Maddy.
I'm setting up Stalwart right now, migrating from my current Maddy+Postfix+Dovecot+Rspamd setup. Not exactly my experience.
The documentation is not great - I'd say it's just about barely enough to get an overall idea, but there's no one proper single definitive overview of what options exist, what are their possible values, what are the defaults, and how they relate to each other. Maddy docs, despite looking a bit sloppy, were a lot easier to get through. IMHO Stalwart makes it unnecessarily difficult to write a non-minimal static configuration file, hooking everything up correctly.
To be fair, maybe there is a page like that but I haven't found it, despite trying.
I know the Web UI allows to do the configuration by clicking through the forms, but this approach conflicts with declarative deployment practices. In my case it's giving me nondescript 500 errors in the UI with "Failed to write local configuration" in the logs because the .toml file is read-only.
Yes, are there any decent JMAP web mail client that we can use?
I have asked sooo many times since Stalwart first was introduced, but not got a straight answer. It is just FastMail or Topicbox. I want something like roudcoube or wildduck that can be used over https that I can self-host!
But is there any real benefit over Postfix + Dovecot other than "it's new and written in Rust?" Postfix and Dovecot have been around for decades and respect the Unix philosophy of doing one thing and doing it well.
I worked on the iCalendar, CalDAV, and CardDAV parsers at Apple in 2010 or so, and I see no reason to believe that today’s Macs, iPhones, and Watches, are using anything more modern.
I haven’t been there in more than a decade. I really am curious what the response in Apple (and Google) is to this spec.
Of all of their proposals, this is the most interesting part to me.
I researched what it would take to implement a full calendaring server once, and after reading all the RFCs, just backed away slowly from the whole idea and never thought about it again.
Running Stalwart in production for ~20 heavily used accounts for some company and no problems so far! The simplicity for such a complex stack and flexibility of deployments is off the charts!
Out of curiosity do you front-end SMTP with postfix to have many queues/MX entries and a battle hardened front-end or is Stalwart handling inbound connections directly? Im thinking of moving from Dovecot to Stalwart so family members have more modern features on my fallback domains about half of my domains do not use Fastmail. In multiple companies I had several Postfix inbound servers to keep the internet from touching Exchange directly and have multiple nodes for companies to quickly hand off to in multiple locations.
Can you share what's your solution for filtering incoming spam? I've had to abandon Stalwart because its spam filter is so ineffective and inconsistent.
For those needing to deal with customers/clients/internal teams with Google Workspace/Outlook and wanting JMAP-style (though not JMAP) modern JSON APIs, Nylas might be a viable option: https://www.nylas.com/
Nylas pricing has gotten better recently, but is still quite high though - at $1.50/connected account/month at scale, it's likely material to your per-user margin if it's part of your SaaS offering.
But if you have a use case where this is a no-brainer (like capturing/analyzing/building custom real-time UI around your internal sales team's emails) then it's remarkably powerful.
I tried to set up stalwart, but I didn't understand exactly what it wanted me to do. It's a webserver (for webmail and admin) and mail server in one, but I already run a webserver and I already have cert infra. So I couldn't figure out what dns settings to use and what ports to reverse proxy. And how to get it to play nice and share the certificate. Seems like stalwart has been designed as if it is the only thing running on a machine, with sparse documentation for any other setup. I tried getting help on the discord server but the tone seemed to be of a sort of "it's quite obvious, you should already know this". At that point, it's so much friction to reverse engineer it that it might just be easier to set up dovecot and postfix.
I have had similarly been unable to get as much help from their Discord as I had hoped.
That said, I'd give this another shot. What I discovered is that Stalwart is incredibly flexible and designed to be used in any configuration you want, and the documentation and examples are incredible, compared to most other projects. BTW, there is no webmail yet, but you are right that there is the admin that is served over HTTP/S, as well as MTA_STS, JMAP. The default listeners get you started and it's kind of out-of-the-scope of Stalwart in terms how you want your server to interact with those listeners. But whether you containerize it or setup a reverse proxy, the documentation has a lot of examples, including how to pick up the certificates, if not managed by Stalwart itself.
I landed on a Caddy for HTTPS and haproxy for proxy protocol on just JMAP, and the remaining TCP services directly binded to the host for the main mail services.
Traefik is another example that I had initially prepped and got working that also handled TCP connections with proxy protocol where needed. The Stalwart documentation was very helpful with that.
In short, I was impressed with how easy it was to integrate Stalwart into whatever setup I wanted, and how open the developer is to different setups. For instance, mox (while awesome in its own right) considers containerized setups to be not-recommended, and generally expects it to live on its own server. Stalwart is flexible, but the side effect is that it is overwhelming at first... but becomes quite elegant once you get the hang of it.
The installation instructions seem to agree with you:
1. curl this shell script
2. Run it as root.
Anytime you see that, you can assume the software wants to take over the whole box and isn't likely to make any attempt to play nicely with other services/software/users already running on the same host.
Interesting I didn't know that JMAP for File Storage is a replacement for WebDAV. My experience with WebDAV is that it has traditionally been sluggish and clunky; I'm not sure if that's due to the protocol or the backend. I'm hoping this could be a huge improvement.
> They are robust, widely adopted, and battle-tested. Yet, their XML-based design is notoriously verbose, inconsistent, and difficult to implement correctly. Information is scattered across HTTP headers, XML payloads, and even embedded iCalendar data, creating endless compatibility and interoperability challenges between clients and servers.
Can others confirm if these problems are widespread? I get that these protocols are probably a pain to develop for but given they are "robust, widely adopted and battle-tested" it seems that is probably a solved problem. It's better to have one standard that is used everywhere than to have to choose between two standards.
I can understand why JMAP instead of IMAP given the latter's antiquated design. I don't see the advantage to clients in replacing WebDAV though, and the others are a bit iffy too. They'll need to make a way better sales pitch than 'JSON vs XML' (serialization ain't tough, XML is supported everywhere).
I guess contacts/calendar follows JMAP naturally when the clients already implement it, but that only applies in the 'already wrote a JMAP email client' case. Virtually any other case would rather stay with widely supported protocols?
As a JMAP client implementer, there’s a big plus, beside JMAP, in having an integrated server for email, contacts, and files.
Some setups take days even for email, so an easy, fast setup saves a lot of time.
Also, after code the CalDAV ↔ JSCalendar part, using only IANA time zones instead of scattered ones in CalDAV components makes things much simpler.
Does it seem like the calendar protocol will be able to replace the VTODO bit of ical so that Todo applications can be built on top of it? I've played around with ics files a bit for the tasks app in nextcloud and it wasn't a pleasant experience so I kinda dropped the project.
Anyone got a link to a better sales job on JMAP & friends?
It sounds awesome but the way it is intro'd here:
Over the past few years, the IETF has been redefining how email, calendars, and contacts are synchronized and shared. Building upon the success of JMAP for Mail, several new protocol extensions have been introduced:
JMAP for Calendars - A modern replacement for CalDAV and CalDAV Scheduling.
JMAP for Contacts – A powerful alternative to CardDAV.
JMAP for File Storage – A replacement for WebDAV-based file storage.
JMAP Sharing – A modern successor to WebDAV ACL.
JSCalendar - A clean, JSON-based evolution of iCalendar.
JSContact – A modernized, JSON-native successor to vCard.
...gave me pause. A protocol I've never heard even though I hang out here for an hour a day, was so successful, that it launched 6 new projects?
Sounds more like the parts of the web dev that give me ick (new and shiny; rush to copy new and shiny in other contexts; give it a year; and all of a sudden only 1 of the 6 actually was successful)
The big pitch for JMAP is for a modern web-tech-only approach to email/calendar/"groupware" servers. One reason to do that would be to make it easier to also build email/calendar/"groupware" clients entirely out of modern web-tech. Today most "web email clients" are bespoke to specific stacks/email servers. A dream of JMAP is that with the right CORS policy a single web client could interact with multiple JMAP servers, using only fetch/XHR.
The modernization efforts of JMAP are interesting, too. Most of the old protocols are a mess of bespoke plaintext formats full of quirks evolved over decades in a giant mess of different software. Even the stuff that was already web tech like WebDAV and its extensions CalDAV and CardDAV were full of quirks, violated some REST "rules", and originally intended for a different purpose (file shares/FTP replacement). JMAP is much closer to "plain REST" than WebDAV's complex HTTP protocol extensions/changes.
JMAP and friends are very niche, none of the "mainstream" email clients (that ship with most computers/phones) support it. So this feature being available is unlikely to grow the userbase, IMO.
Now JMAP is quite a bit nicer to use than IMAP's API, but IMAP's gravitational field is too strong to be supplanted. IMAP is also becoming somewhat of a niche protocol, as the majority of users use vendor proprietary protocols for accessing their emails on Gmail, Outlook/Hotmail, etc. So why invest the time to add a niche replacement for IMAP when the entire protocol is a second class citizen to mainstream email clients.
solarkraft|4 months ago
JMAP is, from what I’ve read, a great protocol for building an E-Mail (and now also others) client on top of.
Since I would like an innovative way to access my E-Mails, but do not want to self-host, I would find it interesting to use Stalwart as the server component of an E-Mail client: Data is somehow synced into Stalwart via the “ugly” protocols and I get a nice API to build an elegant client on top of.
My basic research shows that something like IMAP-IMAP sync seems to be a thing. Has anyone done something like this, perhaps even with Stalwart? (this of course grows in complexity for each new protocol to be proxied).
I believe having this kind of setup easily accessible could jumpstart a new generation of E-Mail clients on top of JMAP because it (relatively elegantly?) circumvents the chicken-egg problem by allowing all existing IMAP mailboxes to be accessed via JMAP.
conradev|4 months ago
I also did not realize it is… primarily developed by a single person? The author. Incredible:
https://github.com/stalwartlabs/stalwart/graphs/contributors
abound|4 months ago
I was originally thinking you'd need to go remote IMAP <-> maildir <-> Stalwart IMAP, which would be really complicated, but I think the IMAP <-> IMAP should work fine.
[1] https://isync.sourceforge.io/mbsync.html
Fire-Dragon-DoL|4 months ago
keeganpoppen|4 months ago
solarkraft|4 months ago
With Stalwart in place, there’s finally a reason to develop a client for JMAP.
I hope y’all are aware that Mozilla’s new mail service will use it, so that is likely going to give JMAP a big push!
audelair|4 months ago
Nextcloud was such a terrible experience for me (the file sharing/storage was good, but the groupware aspect was incredibly buggy). But knowing that Nextcloud is partnering with Stalwart to hopefully overhaul their stack, Opencloud is developing their JMAP integration, and Mozilla/Thunderbird is using it too (they already have a webmail in development here: https://github.com/thunderbird/stormbox)... we might finally see some exciting development in this space. And now is also a ripe time, as there seems to be a perfect storm of people wanting to get away from Big Tech platforms.
AndrewDavis|4 months ago
Just to clarify. Stalwart is the first to have JMAP contacts and calendars. Cyrus has had JMAP mail since the beginning of JMAP.
tristan957|4 months ago
no_wizard|4 months ago
philipwhiuk|4 months ago
It's the first implementation of JMAP applied to stuff other than Mail.
WhyNotHugo|4 months ago
You can now synchosise any combination of CalDAV, JMAP, and filesystem.
calvinmorrison|4 months ago
9dev|4 months ago
Still, it’s an interesting space, I think.
WorldMaker|4 months ago
Email was never a binary protocol. Notoriously so, it's why MIME types and MIME encodings get so complicated.
Most of the "old internet" protocols (email, FTP, even HTTP itself) were bootstrapped on top of built-mostly-for-plaintext Telnet. HTTP as the new telnet has a bunch of improvements when it comes to binary data, request/response-based data flows, and some other considerations. HTTP/3 is even inherently a binary protocol, it's lack of "telnet-compatibility" one of the concerns about switching the majority of the web to it.
vCard/vCal/iCard/iCal were also deeply "plaintext formats". JSON is an improvement because it is more structured, even more efficient, than those predecessors. JSON may not look efficient, but it compresses extremely well and can be quite efficient in gzip and Brotli streams.
I feel like "JSON over HTTP" is a subtle improvement over "custom text formats over telnet", even if it doesn't sound like "binary protocol efficiency" at first glance. Especially as HTTP/3 pushes HTTP more efficient and more "binary", and arguably "more fundamental/basic" with HTTP/3 even taking over more roles in the TCP/UDP layer of the internet stack. (Telnet would never try to replace TCP.) HTTP isn't the worst bootstrap layer the internet could use to build new protocols and apps on top of. Sure, it would be neat to see more variety and experiments outside of the HTTP stack, too, but HTTP is too useful at this point not to build a bunch of things on top of it instead of as their own from-scratch protocol.
zzo38computer|4 months ago
It shouldn't. For some cases it helps, but other times it doesn't. Sometimes it helps but there would be better ways to do it, making it on a simpler protocol or making an entirely new protocol (which might or might not use TCP; sometimes it is better to use TCP and sometimes not) depending on the specific case.
> Stuff like file sharing or groupware, mail, calendars, and so on—these things could be a lot more efficient and don’t really need the overhead of JSON as the message interchange format, IMHO
I dislike JSON. I think it has many problems, and that DER is a better format.
(There are also the "small web" protocols such as Gemini and Scorpion and Spartan and Titan, which avoids some of the complexity of HTTP; I had considered using DER-over-Scorpion rather than JSON-over-HTTP. It is also possible to use SSH, although SSH does not have virtual hosting.)
SchemaLoad|4 months ago
I'm struggling to think of any real benefits to not using HTTP other than it would be more interesting.
toomim|4 months ago
HTTP/2 and HTTP/3 are binary protocols. And if you replace the JSON with CBOR, then even the payload becomes binary.
The reason for using HTTP is that the semantics are right. HTTP is a state transfer protocol, and ultimately, that's 90% of what you need for sync.
The other 10% is for subscriptions, updates, with versioning, and patches. You can get these by adding the Braid extensions (see braid.org) which upgrade HTTP from a state transfer to a state synchronization protocol. (I work on Braid.)
solarkraft|4 months ago
Absolutely yes, IMO. This significantly eases web client development.
pjc50|4 months ago
JSON by comparison has simple, obvious limitations that more people are familiar with dealing with.
There's also the tendency to tie your protocol to implementation. The Microsoft Exchange "protocol" didn't get reverse engineered for so long because it's basically the COM structure of Outlook fed through (if I remember rightly) DCOM-RPC.
wmf|4 months ago
btown|4 months ago
There's no magic. Nothing sacred. Nothing that you aren't allowed to understand, intuitively. Nothing where you aren't allowed to imagine "what if it also had X?" The web is yours. The computer is yours. As an industry, we burn some incremental percentage of bandwidth to give you the keys to the kingdom, and to allow you, new developer, to be one of us.
In an age when LLMs feel like magic boxes to tech-minded people new to development, we need this more than ever.
brongondwana|4 months ago
vamega|4 months ago
alberth|4 months ago
https://datatracker.ietf.org/doc/draft-ietf-jmap-calendars/
And Contacts was only 10-months ago.
https://www.rfc-editor.org/rfc/rfc9610.html
brongondwana|4 months ago
The downsides of developing and testing this stuff as we were writing it up!
We've finished rewriting the objectid generation to give smaller sized and more sortable IDs (they're an inverse of nanosecond internaldates now, plus some extra magic on the low bits for IMAP appends to avoid clashes)... which we wanted to speed up and reduce disk usage for the offline mode.
Next up is indeed updating to the latest spec on calendars and contacts. Files might take a bit longer, I really want to do some desktop clients for the files system, we have a really nice files backend at Fastmail which is only accessible via our interface or WebDAV right now.
drnick1|4 months ago
blacklion|4 months ago
There is no good desktop implementation of MUA with old technologies (IMAP, Sieve), will all this JMAP help?
I don't think so.
What is profit to have good server with new good (assume it is good, I'm not sure, but lets assume) protocols without good client?
IMAP4 is underused by modern clients: it allows to effectively store client configuration on server, nobody implements it on client side. It allows to configure per-folder Sieve scripts, nobody implements it on client side. Nobody implements good Sieve client (with folder name autocomplition and such) even for global script, not to mention per-folder ones. Heck, there is no good Sieve editor! (I know about Sieve client built on Electron, it is not good, it is incomplete and buggy).
Servers are solved problem (sendmail, exim, postfix, dovecot, cyrus). Clients are not, they stagnated at the moment GMail was announced.
Avamander|4 months ago
Clients on the other hand have actually kinda moved forward, Apple Mail works with IMAP servers and offers features that people only got with Gmail before. But there are many other examples as well.
mmooss|4 months ago
You need both. You could say, what profit is a good client without a server? By that reasoning, we never stake a step forward without a complete solution.
Now a better mail implementation is just a client away.
sylens|4 months ago
gtech1|4 months ago
P.S. ("New" Outlook already only connects to MS365 servers and then stores your credentials and data on Azure, while they proxy to your actual IMAP/SMTP server )
woodruffw|4 months ago
(This should not be interpreted as a defense of IMAP.)
WhyNotHugo|4 months ago
For regular desktop software, I’m not sure that it’s really an improvement over existing protocols.
Eric_WVGG|4 months ago
I’ve got a friend who’s been pitching me on building a new email client for years. “I’ll do it if we exclusively use JMAP.” “okay does that include Gmail and Apple/iCloud accounts?” “Nope.”
I could sort of see dual-supporting Gmail's proprietary API and JMAP, but unless the #2-5 competitors support it… what’s the point? (sorry to put on the pessimism hat)
frumplestlatz|4 months ago
To be honest, I’m not sure why end-users would want JMAP for e-mail access.
It would be interesting if they do successfully roll out all of these additional RFC proposals providing a cohesive “groupware” protocol covering calendering, contacts, file shares, etc, we see notable server implementations, and interest is enough to drive client support.
That’s a lot of “ifs”.
lifty|4 months ago
drdaeman|4 months ago
The documentation is not great - I'd say it's just about barely enough to get an overall idea, but there's no one proper single definitive overview of what options exist, what are their possible values, what are the defaults, and how they relate to each other. Maddy docs, despite looking a bit sloppy, were a lot easier to get through. IMHO Stalwart makes it unnecessarily difficult to write a non-minimal static configuration file, hooking everything up correctly.
To be fair, maybe there is a page like that but I haven't found it, despite trying.
I know the Web UI allows to do the configuration by clicking through the forms, but this approach conflicts with declarative deployment practices. In my case it's giving me nondescript 500 errors in the UI with "Failed to write local configuration" in the logs because the .toml file is read-only.
zenmac|4 months ago
I have asked sooo many times since Stalwart first was introduced, but not got a straight answer. It is just FastMail or Topicbox. I want something like roudcoube or wildduck that can be used over https that I can self-host!
drnick1|4 months ago
kridsdale1|4 months ago
I haven’t been there in more than a decade. I really am curious what the response in Apple (and Google) is to this spec.
frumplestlatz|4 months ago
I researched what it would take to implement a full calendaring server once, and after reading all the RFCs, just backed away slowly from the whole idea and never thought about it again.
matesz|4 months ago
Bender|4 months ago
jdalsgaard|4 months ago
quicksilver03|4 months ago
btown|4 months ago
Nylas pricing has gotten better recently, but is still quite high though - at $1.50/connected account/month at scale, it's likely material to your per-user margin if it's part of your SaaS offering.
But if you have a use case where this is a no-brainer (like capturing/analyzing/building custom real-time UI around your internal sales team's emails) then it's remarkably powerful.
woleium|4 months ago
edit: we use it on very resource constrained environments, the container version is too much overhead.
evrflx|4 months ago
dijit|4 months ago
reachableceo|4 months ago
I am most curious
apitman|4 months ago
bruce343434|4 months ago
audelair|4 months ago
That said, I'd give this another shot. What I discovered is that Stalwart is incredibly flexible and designed to be used in any configuration you want, and the documentation and examples are incredible, compared to most other projects. BTW, there is no webmail yet, but you are right that there is the admin that is served over HTTP/S, as well as MTA_STS, JMAP. The default listeners get you started and it's kind of out-of-the-scope of Stalwart in terms how you want your server to interact with those listeners. But whether you containerize it or setup a reverse proxy, the documentation has a lot of examples, including how to pick up the certificates, if not managed by Stalwart itself.
I landed on a Caddy for HTTPS and haproxy for proxy protocol on just JMAP, and the remaining TCP services directly binded to the host for the main mail services.
Traefik is another example that I had initially prepped and got working that also handled TCP connections with proxy protocol where needed. The Stalwart documentation was very helpful with that.
In short, I was impressed with how easy it was to integrate Stalwart into whatever setup I wanted, and how open the developer is to different setups. For instance, mox (while awesome in its own right) considers containerized setups to be not-recommended, and generally expects it to live on its own server. Stalwart is flexible, but the side effect is that it is overwhelming at first... but becomes quite elegant once you get the hang of it.
bityard|4 months ago
1. curl this shell script
2. Run it as root.
Anytime you see that, you can assume the software wants to take over the whole box and isn't likely to make any attempt to play nicely with other services/software/users already running on the same host.
To their credit, they do have a docker image (https://stalw.art/docs/install/platform/docker).
commandersaki|4 months ago
NoboruWataya|4 months ago
Can others confirm if these problems are widespread? I get that these protocols are probably a pain to develop for but given they are "robust, widely adopted and battle-tested" it seems that is probably a solved problem. It's better to have one standard that is used everywhere than to have to choose between two standards.
bmelton|4 months ago
Always relevant: https://xkcd.com/927/
yyyk|4 months ago
I guess contacts/calendar follows JMAP naturally when the clients already implement it, but that only applies in the 'already wrote a JMAP email client' case. Virtually any other case would rather stay with widely supported protocols?
sureglymop|4 months ago
However, doesn't stalwart already also support WebDAV though?
Gormo|4 months ago
I think we're about ten years past the point where "newer = better" was a reasonable starting presumption.
Mailtemi|4 months ago
Also, after code the CalDAV ↔ JSCalendar part, using only IANA time zones instead of scattered ones in CalDAV components makes things much simpler.
raybb|4 months ago
pluto_modadic|4 months ago
jasonriddle|4 months ago
lifty|4 months ago
batisteo|4 months ago
Ey7NFZ3P0nzAe|4 months ago
artooro|4 months ago
refulgentis|4 months ago
It sounds awesome but the way it is intro'd here:
...gave me pause. A protocol I've never heard even though I hang out here for an hour a day, was so successful, that it launched 6 new projects?Sounds more like the parts of the web dev that give me ick (new and shiny; rush to copy new and shiny in other contexts; give it a year; and all of a sudden only 1 of the 6 actually was successful)
WorldMaker|4 months ago
The modernization efforts of JMAP are interesting, too. Most of the old protocols are a mess of bespoke plaintext formats full of quirks evolved over decades in a giant mess of different software. Even the stuff that was already web tech like WebDAV and its extensions CalDAV and CardDAV were full of quirks, violated some REST "rules", and originally intended for a different purpose (file shares/FTP replacement). JMAP is much closer to "plain REST" than WebDAV's complex HTTP protocol extensions/changes.
ggm|4 months ago
https://datatracker.ietf.org/wg/jmap/history/
Bron is the principal of fastmail, who now own pobox. This is a serious activity.
candiddevmike|4 months ago
Now JMAP is quite a bit nicer to use than IMAP's API, but IMAP's gravitational field is too strong to be supplanted. IMAP is also becoming somewhat of a niche protocol, as the majority of users use vendor proprietary protocols for accessing their emails on Gmail, Outlook/Hotmail, etc. So why invest the time to add a niche replacement for IMAP when the entire protocol is a second class citizen to mainstream email clients.
JadedBlueEyes|4 months ago
SomaticPirate|4 months ago