Would the S3 inventory help here? That would allow you to get the list of all files (albeit on a delay similar to the lifecycle rule approach), which you could process offline to generate the DELETEs.
S3 inventory would cost $0.0025 per million objects listed [1], while LIST requests are $0.005 per thousand requests and each LIST request can return up to 1000 objects, making them $0.005 per million objects listed. For the "Infrequent Access" storage tier, LISTs cost double that.
So S3 inventory would be half price compared to LIST (or quarter price in IA storage class), but that's still small comfort if you're staring down the barrel of a bucket containing a large number of objects.
jffry|4 years ago
So S3 inventory would be half price compared to LIST (or quarter price in IA storage class), but that's still small comfort if you're staring down the barrel of a bucket containing a large number of objects.
[1] Management & analytics tab on https://aws.amazon.com/s3/pricing/
vdm|4 years ago
> LIST requests for any storage class are charged at the same rate as S3 Standard PUT, COPY, and POST requests.
I read this as LISTs do not cost double for infrequent access, even though other Tier 1 requests do.
unknown|4 years ago
[deleted]
lathiat|4 years ago
Is also mentioned in the article though they don’t calculate the price.