How is ejabberd or XMPP in general doing these days?
I remember Nintendo adopted it for their push platform [1] but it was old news. Are there any interesting development and/or adoption stories in recent years?
Modern versions of ejabberd and MongooseIM are much better than earlier versions in regards to scaling. But XMPP is still a mess. If you need to develop a custom messaging app, it's much better to start from a clean slate, rather than trying to fit your data and interaction models into an XMPP framework. As a transport you can use Websockets, MQTT, and/or GraphQL.
The main problems with the earlier versions of ejabberd were:
1. Using Erlang strings (linked lists of UTF32 codepoints) instead of UTF8 binaries, which led to 16x more RAM use.
2. Over-reliance on Mnesia for session management which is a maintenance nightmare when used in large clusters. Now it can be replaced by Redis or your own DB.
3. Non-scalable default implementations for mod_muc (Multi-User Chat) and mod_pubsub.
4. Absence of built-in mobile push support.
That was before WhatsApp, which later committed upstream many of their scalability improvement to the BEAM VM.
Source: I scaled an old ejabberd cluster to 500K concurrent users, and wrote my own Erlang-based Long-Polling/Websocket Pub/Sub servers and MQTT brokers scaling to 1M concurrent connections per node in a cluster. Also Elixir-based specialized messaging systems.
As of 2 years ago, there were good XMPP clients for every platform except the iPhone. On the server side, both ejabberd and Prosody are in great shape. There is zero excitement for administering them.
With ejabberd, your scalability problem is that you don’t have enough users to load a single node. So you have a popularity problem, but not a technical one.
To get back to familiar technical ground, try Synapse. A node can support, like, 3 concurrent users.
I wish you would joking. But it doesn't depend so much on the users but more on the size of the room / amount of events and the amount of other servers in the room.
I hope someday someone pulls off an memory/cpu efficient matrix protocol implementation. https://conduit.rs/ looks good and there is still work on construct the c++ server: https://github.com/jevolk/charybdis - but both only implement a subset of the rather huge and complicated spec at the moment.
There are several talks by WhatsApp people who describe how they're scaled their systems. They run on FreeBSD and used Dtrace to quickly find bottlenecks in the BEAM VM. BEAM has a very good builtin instrumentation, but if you're only looking on them, you're missing a lot. Also they committed upstream many of their improvements.
Not only WhatsApp, but also the initial implementation of APNs (Apple Push Notification System) was based on ejabberd. They even used the same port numbers.
I did an ejabberd scaling training in SF in 2011, and there were several Apple guys. It doesn't mean that they're used ejabberd for sure, but it's a hint.
I like the federation abilities of xmpp and ran ejabberd for a previous job. It’s perfect for an organization … in theory. Client support for the protocol was always hit or miss though.
Have file transfers over xmpp gotten any better in the last decade?
Haven't tested much. But I think HTTP upload is reliable and has been ok for me on Snikket server (Prosody) between Snikket clients (Conversations) and between Snikket and Gajim.
[+] [-] flakiness|3 years ago|reply
I remember Nintendo adopted it for their push platform [1] but it was old news. Are there any interesting development and/or adoption stories in recent years?
[1] https://www.process-one.net/blog/ejabberd-nintendo-switch-np...
[+] [-] nivertech|3 years ago|reply
The main problems with the earlier versions of ejabberd were:
1. Using Erlang strings (linked lists of UTF32 codepoints) instead of UTF8 binaries, which led to 16x more RAM use.
2. Over-reliance on Mnesia for session management which is a maintenance nightmare when used in large clusters. Now it can be replaced by Redis or your own DB.
3. Non-scalable default implementations for mod_muc (Multi-User Chat) and mod_pubsub.
4. Absence of built-in mobile push support.
That was before WhatsApp, which later committed upstream many of their scalability improvement to the BEAM VM.
Source: I scaled an old ejabberd cluster to 500K concurrent users, and wrote my own Erlang-based Long-Polling/Websocket Pub/Sub servers and MQTT brokers scaling to 1M concurrent connections per node in a cluster. Also Elixir-based specialized messaging systems.
[+] [-] upofadown|3 years ago|reply
The XMPP newsletter[2] is a good summary of what is going on in the world of XMPP.
[1] https://snikket.org/
[2] https://xmpp.org/categories/newsletter/
[+] [-] aidenn0|3 years ago|reply
[+] [-] akho|3 years ago|reply
To get back to familiar technical ground, try Synapse. A node can support, like, 3 concurrent users.
[+] [-] nisa|3 years ago|reply
I wish you would joking. But it doesn't depend so much on the users but more on the size of the room / amount of events and the amount of other servers in the room.
However Matrix solves a different problem https://github.com/jryans/awesome-matrix#research so it's a bit unfair to compare both.
I hope someday someone pulls off an memory/cpu efficient matrix protocol implementation. https://conduit.rs/ looks good and there is still work on construct the c++ server: https://github.com/jevolk/charybdis - but both only implement a subset of the rather huge and complicated spec at the moment.
[+] [-] alberth|3 years ago|reply
I’m sure they have a wealth of knowledge related to scaling since WhatsApp was initially based on ejabberd.
[+] [-] nivertech|3 years ago|reply
Not only WhatsApp, but also the initial implementation of APNs (Apple Push Notification System) was based on ejabberd. They even used the same port numbers. I did an ejabberd scaling training in SF in 2011, and there were several Apple guys. It doesn't mean that they're used ejabberd for sure, but it's a hint.
[+] [-] imoverclocked|3 years ago|reply
Have file transfers over xmpp gotten any better in the last decade?
[+] [-] leetnewb|3 years ago|reply