Kinesis supports buffering - up to 900 seconds or 128mb. So you are way out on your cost estimations. Over time queries can start costing more due to S3 Requests, but regular spark runs to combine small files solves that.
I haven't even got to kinesis or bandwidth or storage.
Even if you compress N objects through spark/etc your starting point would be the large number of writes first. So that doesn't change. The costs would be even larger considering even more medium sized PUT's that double the storage, add N deletes potentially. Have also heard that Athena, presto etc charge based on rows read.
bosky101|1 year ago
Even if you compress N objects through spark/etc your starting point would be the large number of writes first. So that doesn't change. The costs would be even larger considering even more medium sized PUT's that double the storage, add N deletes potentially. Have also heard that Athena, presto etc charge based on rows read.