Why do small teams keep getting wrecked by AWS bills?

[DeborahEmeni_]

Saw someone on Reddit lose $86k from a compromised AWS account. I've heard way too many stories like this — misconfigured IAM, tokens in repos, no billing alerts...

If you're on a small team, how are you actually protecting yourself from stuff like this? Is there a sane setup that works without needing a full-time AWS security person?

[PaulHoule]

Gotta look at your monthly bill every day.

I've been wondering about the question of "where did the web go?" and how even technically savvy people have given up on blogging for behavioral sinks like Medium.

Part of the story is that $50 a month dedis have given way to the "free" plan on AWS or a system that costs $10 a month to run if you're not successful but has no upper bound on the bills if you are successful. So if you make a blog you are praying every night that you don't make it to the front of Hacker News and that you don't build up a large following because boy those egress charges will add up. People are furious now that they are getting eaten alive by the egress costs run up by AI bots but 10 years ago I was thinking "Boy Bing crawls my site twice as hard as Google and sends 5% of the traffic and Chinese webcrawlers crawl my site 5x harder than Google and send me no detectable traffic."

[DeborahEmeni_]

yeah makes sense. the idea that success = risk now is kind of wild. did you end up doing anything to limit that egress, or just ride it out?

[scarface_74]

They look at code on stackoverflow and the web that initializes the SDK resources that have you explicitly put the access key and secret key in code.

For instance, the correct way to initialize the s3 client in Python is

     s3 = boto3.client('s3')

The SDK will automatically get the credentials that are configured locally within your environment or the IAM role attached to your Lambda, EC2 instance, Docker (ECS, EKS) container runner etc.

Your access keys never need to be part of your repository.

[DeborahEmeni_]

yep this is the one. feels like a lot of people just copy-paste from stackoverflow without realizing they're hardcoding keys. have you seen this cause issues in prod or mostly in hobby stuff?

[QuinnyPig]

One of the best things you can do is enable AWS’s free Cost Anomaly Detector. I like getting pinged when something substantial changes.

[DeborahEmeni_]

yeah cost anomaly detector is underrated. do you usually set custom thresholds or just let it use the defaults?

[ArinaS]

https://old.reddit.com/r/aws/comments/1jupura/just_got_compr... - this post you're referring to?

[DeborahEmeni_]

yep that’s the one. rough situation but the replies were interesting

[chistev]

I like that you linked old reddit. Using the normal domain makes the site utterly unusable.

[jalapenos]

Because AWS doesn't have out-of-the-box costs killswitches.

[DeborahEmeni_]

yeah that’s the part that gets me too. feels wild that there’s no built-in way to cap spend at the account level. even a “pause all” button would go a long way

[neuroelectron]

Best way to defeat competition is to make sure they never get out of the gate

[DeborahEmeni_]

yeah exactly. feels like the pricing model alone keeps a lot of folks from even starting. one wrong config and you’re done before you ship anything.

[8b16380d]

We’re not since cost controls are secondary to building features.

[DeborahEmeni_]

makes sense tbh. i’ve seen teams take that approach and only think about cost once it’s already a problem. ever had that come back to cause issues later?