Hi HN! I'm Alex, and I've been researching and writing about WebSockets for a while now. I'm the author of the recently released WebSocket Handbook. AMA about the WebSocket tech, the realtime web, Ably or anything related to Liverpool FC.
Hi! I want to create a web app like Google docs where multiple users can collaborate in real time to edit the document together (using a special link like gdocs generate). I want to save the docs in a MySQL db (no firebase)
My questions are:
1) Since multiple people are working together how does one manage conflicts, i.e. 2 people sending different edits simultaneously.
2) If one clients gets disconnected (4g) and then reconnects later how does it sync the changes it made during it was offline?
I recently watched this RAFT presentation (1) and I think I would need to use something like this?
What other alternatives are viable?
Also can I make it happen using just PHP, Javascript and MySQL?
It uses operational transformations ("OT") to manage conflicts, and it saves the data in MySQL. Technically any Django DB backend will work for storage, but the public demo instance uses MySQL.
One of the reasons I made this thing was to show that realtime apps don't need to require heavy frameworks or unusual databases. And it loads super fast.
I don't think you need Raft if you have a central database storing the document. You could also consider using CRDTs instead of OT, which may be more powerful but also more challenging to develop.
CRDT solve a part of your problem, and an important consideration is whether or not you want off-line editing. If you don't need off-line editing, then a WebSocket can do it.
I'm actually using my project to build a collaborative IDE (designer like Figma): http://www.adama-lang.org/
I'm going to be launching it as a SaaS soon so people can spin up a new back-end without managing an infrastructure.
As with REST APIs, you'd want to be able to authenticate to a websocket-based API using either basic auth, or a bearer token-based auth scheme. Unfortunately, the browser websocket API doesn't allow you to specify arbitrary headers in the websocket request, so it's typical instead to have credentials supplied via a query param (such as "accessToken" for a bearer token) in the wss request.
It's the same thing as HTTP. Websocket starts off as an HTTP request with cookies, headers etc. Use those just like HTTP to authenticate, and your Websocket server should pass the user data to the websocket object
In a Golang app I'm writing now, I have middleware that authorizes requests. Authenticated requests have a header with a JWT token. I have an endpoint for websockets where if an authenticated request comes in (the handshake) that request is then upgraded to a Websocket connection. This is the cleanest authentication implementation I've ever used thus far, and I wasn't able to achieve the same thing in Node when I was using socket.io.
I'm sure there are repercussions to this on the client-side, but I haven't gotten to that point yet. I'm still writing the server and testing it using automated integration tests.
If I understand correctly, websockets is a thin layer on TCP that does buffer data so that the application get the whole message instead of chunks. I recommend using wss to secure the websocket so that it can't be hijacked, then you don't need to send a token in each message and can do an application layer handshake once. Basically the first websock message from the client would be an authentication message with a password, token or what not.
We are monitoring it closely and super excited about what WebTransport will provide, which is both a more reliable and in many cases more performant transport. However, much like WebSockets, it's still quite low level and only provides a basic communication protocol. As such, like we have done with SSE, HTTP, Websockets and MQTT, our service focusses on what developers can enable on top of these lower level transports, such as presence, deltas, history of streams, limitless scale and fan-out of data, and the list goes on https://ably.com/platform.
When WebTransport reaches prime time, I'm confident we'll be supporting it.
superasn|4 years ago
My questions are:
1) Since multiple people are working together how does one manage conflicts, i.e. 2 people sending different edits simultaneously.
2) If one clients gets disconnected (4g) and then reconnects later how does it sync the changes it made during it was offline?
I recently watched this RAFT presentation (1) and I think I would need to use something like this?
What other alternatives are viable?
Also can I make it happen using just PHP, Javascript and MySQL?
Thanks
(1) http://thesecretlivesofdata.com/raft/
jkarneges|4 years ago
It uses operational transformations ("OT") to manage conflicts, and it saves the data in MySQL. Technically any Django DB backend will work for storage, but the public demo instance uses MySQL.
One of the reasons I made this thing was to show that realtime apps don't need to require heavy frameworks or unusual databases. And it loads super fast.
I don't think you need Raft if you have a central database storing the document. You could also consider using CRDTs instead of OT, which may be more powerful but also more challenging to develop.
born-jre|4 years ago
ofcourse u will need websocket to sync changes.
AlexTDiaconu|4 years ago
CRDTs can be the answer. We actually wrote about them recently (https://ably.com/blog/crdts-distributed-data-consistency-cha...), and there is more coming soon as our Chief Of Scientist and his team are researching CRDTs and building demos!
hammersmith|4 years ago
mathgladiator|4 years ago
I'm actually using my project to build a collaborative IDE (designer like Figma): http://www.adama-lang.org/
I'm going to be launching it as a SaaS soon so people can spin up a new back-end without managing an infrastructure.
friendlydog|4 years ago
AlexTDiaconu|4 years ago
JSdev1|4 years ago
dimgl|4 years ago
I'm sure there are repercussions to this on the client-side, but I haven't gotten to that point yet. I'm still writing the server and testing it using automated integration tests.
z3t4|4 years ago
mathgladiator|4 years ago
I recently wrote about the Woes of Websocket: http://www.adama-lang.org/blog/woe-of-websocket with an errata based on HN feedback: http://www.adama-lang.org/blog/more-websocket-woe
The depth of this topic is very interesting, and I'm excited as I'm building some of the final pieces for my SaaS (which could compete with Ably).
httgp|4 years ago
matt_oriordan|4 years ago
When WebTransport reaches prime time, I'm confident we'll be supporting it.
Ostrogodsky|4 years ago
lelanthran|4 years ago
Do people still watch football?
What are the viewership numbers for Liverpool FC?
Does Man United still matter?
:-)
stichers|4 years ago
mobilio|4 years ago