top | item 37141560

(no title)

elisbce | 2 years ago

The real reason why we can't easily replicate Twitter/Facebook/Google is because we don't have the distributed storage/caching/logging/data processing/serving/job scheduling/... infrastructures that they have built internally that are designed to provide some level of guaranteed SLAs for the desired scale, performance, reliability and flexibility, not because it is hard to replicate the application logic like posting to timelines. That's also why Threads were built by a small team rather quickly -- they already have the battle-tested infras that can scale.

Any attempt to build a simplified version of the ecosystem will face the same fundamental distributed system tradeoffs like consistency/reliability/flexibility/... For example, one of the simplifications may be mixing storage/serving/ETL workloads on the same node. And the consequence is that without certain level of performance isolation, it could impact the serving latency during expensive ETL workload.

For Rama to be adopted successfully, I think it is important to identify areas where it has the most strengths, and low LOCs might not be the only thing that matters. For example, demonstrating why it is much better/easier than setting up Kafka/Spark and a database and build a Twitter clone on top of that while providing similar/better performance/reliability/extensibility/maintainability/... is a much stronger argument.

discuss

order

No comments yet.