[itroot]

It's a complex question. I had experience of working with ~60petabytish system back in 2016, and there a lot of things to cover (not only storage):

* network access - do you have data that will be accessed frequently, and with high traffic? You need to cover this skewed access pattern in your solution.

* data migration from one node to another, etc...

* ability to restore quickly in case of failure.

I would suggest to:

* use some open-source solution on top of the hosted infrastructure (Hetzner or similar is a good choice)

* bring in a seasoned expert to analyze your data usage/storage patterns, maybe there are some other ways to make storage more cost effective, that simply moving out of AWS S3.