[Shrezzing]

I think these examples still loosely fits the author's argument:

> There are some cases where big data is very useful. The number of situations where it is useful is limited

Even though there are some great use-cases, the overwhelming majority organisations, institutions, and projects will never have a "let's query ten petabytes" scenario that forces them away from platforms like Postgres.

Most datasets, even at very large companies, fit comfortably into RAM on a server - which is now cost-effective, even in the dozens of terabytes.

[teleforce]

Big data is not only about storage size but also about processing e.g. RAM. In the next coming years the trend is that there will be more (read exponential) IoT sensor devices than we can ever imagine and the nature of their data will be mostly big data in the sense of size (storage) and the analysis (RAM). Just check the latest cars or EV cars, many of them have hundreds of sensor devices and these devices are already connected to the Internet.

Another upcoming example is the latest 5G DECT NR+ standards (the first non-cellular 5G), it will only fuels these massive accumulation of datasets and these monitoring sensor devices do not even get connected to the Internet (think of private factory networks) [1].

Apparently there are limited number of human using or having sensors but for non-human based devices the sky is the limit. For human based communication the data is very limited, we rarely communicate with each others and most of our data now is based on our intermittent media consumptions while streaming audio/video [2]. For IoT sensors devices, they mainly have regular and frequent interval sampling that probably in the ranges of every seconds, minutes, hours, etc. Some if not most of this data is not clean data, there are raw data, and raw data is inherently big and huge compared to the data, for example raw image data vs JPEG data, where the former can be several time bigger in size and processing requirements.

[1] DECT NR+: A technical dive into non-cellular 5G:

https://news.ycombinator.com/item?id=39905644

[2] 50 Video Statistics You Can’t Ignore In 2024:

https://www.synthesia.io/post/video-statistics