[dxxvi]

So, you want a place to store many files in a short period of time and when there's a new file, somebody must be notified?

Have you ever thought of using a postgresql db (also on aws) to store those files and use CDC to publish messages about those files to a kafka topic? In your original way, we need 3 aws services: s3, lambda and sqs. With this way, we need 2: postgresql and kafka. I'm not sure how well this method works though :-)

[ravedave5]

I've dealt with images in a database and it was a disaster, the transfer times are garbage.

[jrochkind1]

Like put the video blobs themselves in postgres data columns? Does putting very large (relative to what you normally put in postgres) files in pg work well? Genuine question, i do not know, I've been considering it too and hesitant about it.

[dxxvi]

I already checked with AI before putting the comment :-)

1GB with the bytea data type (https://www.postgresql.org/docs/current/datatype-binary.html) and 4TB with the BLOB data type (https://wiki.postgresql.org/wiki/BinaryFilesInDB).

[dxxvi]

> I've been considering it too and hesitant about it

Why hesitant? Just ask AI. It'll tell you how to do it and then you can experiment it yourself.