Composable single-purpose modules that communicate over a standard interface can be more easily achieved without involving a network and the complexity that comes with it.
IMO, there are only a few cases where the added network traversal make sense.
1. There's some benefit to writing the different parts of the system in different languages (e.g. Go and Python for AI/ML)
2. The teams are big enough that process boundaries start to form.
3. The packaging of some specific code is expensive. For example, the Playwright Docker image is huge so it makes sense to package and deploy it separately.
Otherwise, agreed, it just adds latency and complexity.
It's actually really weird, if you think about it, that point 1 should involve the network. We should be able to just call a function in one language from a function in another language.
Actually this happened to me once. We had two components that needed to talk to each other - one with an Erlang server and C client library that communicated over a socket with a proprietary protocol - and the other in node.js. The first attempt was to write a C translator that took requests over another socket with a different proprietary protocol, but this one was proprietary to us so we could use it directly from node.js. The second, much better attempt was to learn node's C++ module interface and write a C++ node module wrapper around the C client library.
This third-party Erlang component benefited from being an independently restartable process and therefore needing some RPC, but we also had a mess of C/C++ components inter-connecting over RPC that in reality probably didn't need to be separate processes, but for some reason we'd already decided that architecture before we started writing them.
CharlieDigital|1 year ago
1. There's some benefit to writing the different parts of the system in different languages (e.g. Go and Python for AI/ML)
2. The teams are big enough that process boundaries start to form.
3. The packaging of some specific code is expensive. For example, the Playwright Docker image is huge so it makes sense to package and deploy it separately.
Otherwise, agreed, it just adds latency and complexity.
immibis|1 year ago
Actually this happened to me once. We had two components that needed to talk to each other - one with an Erlang server and C client library that communicated over a socket with a proprietary protocol - and the other in node.js. The first attempt was to write a C translator that took requests over another socket with a different proprietary protocol, but this one was proprietary to us so we could use it directly from node.js. The second, much better attempt was to learn node's C++ module interface and write a C++ node module wrapper around the C client library.
This third-party Erlang component benefited from being an independently restartable process and therefore needing some RPC, but we also had a mess of C/C++ components inter-connecting over RPC that in reality probably didn't need to be separate processes, but for some reason we'd already decided that architecture before we started writing them.