I see so many blog posts from folks tweaking an algorithm or examining a data structure or describing a programming practice that is perceived to be superior to another (with varying quality, but that's the internet).
There is a relative Boötes Void of info being posted about relational data architecture and strategies. I came to realize most developers just regard the database as a dumb bit bucket, just as the author here used to. When all you know is an ORM, a database looks pretty dumb. Just model the relational data to match your object model?!
Smart developers care a lot more about the data structures they use than their choice of programming language or even algorithm. And the relational database engines manage the most fundamental data structures at scale. But folks will do almost anything to avoid learning the details of their database engine to point they'll shun even looking at SQL.
It's madness. As an industry we leave so much data validation, performance, and flexibility on the floor while fighting with access troubles further downstream when there's not much we can do about it anymore.
Some foreign key and check constraints could have prevented more data integrity issues than I can count. Some well placed indexes can literally solve performance problems from top to bottom. Row-level security at the data layer has exposed and mitigated so many bugs in app code.
To echo the author: "Don't design your database like a front end developer"
To make it sound more serious (or more funny, in this example), add three more zeroes to these numbers. Right now the whole site downloads the amount of data equivalent to 5000 recipes slightly compressed in transit, and also takes 250MB+ of RAM.
I actually wish all e-stores downloaded and cached all info for at least the current shopping section. Because the current behavior is to wait several seconds for each set of filters you may change.
It's nice to be ready for huge volumes of data, but also don't lie to yourself by thinking that your store will ever have inventory as large as 0.001% of that of Amazon. It will always be the same 1 MB that you'll query hundreds of times per second instead of fetching it once per client and letting their device do the job much faster.
ttfkam|3 years ago
There is a relative Boötes Void of info being posted about relational data architecture and strategies. I came to realize most developers just regard the database as a dumb bit bucket, just as the author here used to. When all you know is an ORM, a database looks pretty dumb. Just model the relational data to match your object model?!
Smart developers care a lot more about the data structures they use than their choice of programming language or even algorithm. And the relational database engines manage the most fundamental data structures at scale. But folks will do almost anything to avoid learning the details of their database engine to point they'll shun even looking at SQL.
It's madness. As an industry we leave so much data validation, performance, and flexibility on the floor while fighting with access troubles further downstream when there's not much we can do about it anymore.
Some foreign key and check constraints could have prevented more data integrity issues than I can count. Some well placed indexes can literally solve performance problems from top to bottom. Row-level security at the data layer has exposed and mitigated so many bugs in app code.
To echo the author: "Don't design your database like a front end developer"
wruza|3 years ago
I actually wish all e-stores downloaded and cached all info for at least the current shopping section. Because the current behavior is to wait several seconds for each set of filters you may change.
It's nice to be ready for huge volumes of data, but also don't lie to yourself by thinking that your store will ever have inventory as large as 0.001% of that of Amazon. It will always be the same 1 MB that you'll query hundreds of times per second instead of fetching it once per client and letting their device do the job much faster.
smitty1e|3 years ago
The related awakenings are rude when data arrive at scale.
mrnoone|3 years ago