Does anyone else finds AWS and other Amazon services overly complicated?

[damacaner]

I think AWS and its signature system is making things more complicated than it should be, like this is a normal signing in process to API:

1. you request client credentials, which is normal.

2. construct request URL, normal.

3. add headers, eh, normal.

4. signature... fuck.

First you need to convert the URL you have from step 2, mash it with headers from step 3, add header keys to signed headers, then sum256 hash the payload and hex encode it.

Then you create a sign, add algorithm, request date time that is formatted with ISO8601 but all special characters stripped apart, add credential scopes, hash the canonical request you created at the first step.

Then, you calculate this abomination: HMAC(HMAC(HMAC(HMAC("AWS4" + kSecret,"20150830"),"us-east-1"),"iam"),"aws4_request")

after that you calculate this: signature = HexEncode(HMAC(derived signing key, string to sign))

after that you create an authorization header and add signature to it: Authorization: AWS4-HMAC-SHA256 Credential=AKIAIHV6HIXXXXXXX/20201022/us-east-1/execute-api/aws4_request, SignedHeaders=host;user-agent;x-amz-access-token;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924aEXAMPLE

...I mean what the fuck? I can understand why people choose Azure over AWS for the sake of freaking simplicity just by looking at this sign and request process. It feels overly-complicated. Does anyone feels the same while working with this abomination?

[gw98]

Ah that's nothing. Simple problem from a high level: static web site on apex domain.

What you should be able to do:

Click click done. Upload files to S3. Point CNAME at AWS.

What you have to do:

Create an S3 bucket and stick the files in it. Create a zone in Route 53 and import your old zone file. Change your nameservers at the registrar. Wait a bit. Go to ACM in the correct region and create a cert. Tell it to add the DNS entries to R53. Wait a bit. Create a CloudFront distribution making sure that you get all the options right which is quite difficult. While that's deploying, copy the IAM policy it generates back to the S3 bucket. Jump back to R53 and add a new A record pointing at the cloudfront distribution alias. Cross fingers and start praying that the whole stack works. If it doesn't spend several hours working out which thing you forgot to click.

What devops culture would have you do:

Play with CloudFormation for 2 days writing oodles of mind numbing YAML and realise that you have to create the ACM cert and CF distribution in a different region to your bucket but that's impossible. Try and work around this with recommended StackSets but realise they are so horrible that they are unusable. Spend an hour googling. Download terraform because everyone is gushing over it. Spend several hours learning the above and writing it in HCL constantly tearing down and creating resources until the whole thing limps along. Eventually realise you need to share this is someone and the thing is stateful so pay Hashicorp for TF Cloud. After the PO is approved with takes 2 weeks, check it in to git, cheer loudly, hand it over to a colleague who has a different version of terraform and then discover about tfenv and terraform upgrades.

I only do this because I'm paid by the hour. I'm not even a cloud or ops guy. I'm an electronics engineer who needs to eat. I eat well due to this mess but I know it's all so so so wrong.

[viraptor]

AWS is basically a "construct your own custom infrastructure" service and does that well. But if you're not going past a static page, it's not the best service - just go with vercel or netlify or someone similar instead.

The policies, cloudformation, r53 config, cloudfront all have their purpose and a lot of flexibility. But don't pay the complexity price if someone else can wrap it for you in a nice saas.

[yakubin]

100% this. A couple times I've considered moving my personal website to AWS, but each time I got lost in the documentation trying to keep all the different moving parts in my head. At the end of the day managing a single VPS running debian+nginx+certbot[1] with a Gandi domain is orders of magnitude simpler.

[1]: I'm probably going to move that to FreeBSD/OpenBSD with OpenBSD's relayd+httpd+acme-client at some point, but the general idea stays the same.

[stblack]

I envy you.

I am using a simple S3 bucket to serve image files.

We’re getting close to busting the free tier web traffic limit.

I can’t figure out where, or how, to enter my boss’s credit card so we can start paying for the service.

I’m not kidding. I’ve looked everywhere. Several times. I’ve probably spent two hours on this.

[YPPH]

>Go to ACM in the correct region and create a cert

Glad you included the "correct region" qualifier. The number of times I've mistakenly created the certificate in my local region, rather than the one designated for CloudFront...

[TheNewsIsHere]

I recently created a static site at a zone apex without Route53. Everything else just works, well, as you illustrate.

> Play with CloudFormation for 2 days writing oodles of mind numbing YAML and realise that you have to create the ACM cert and CF distribution in a different region to your bucket but that's impossible.

The discrepancies in cross-region support specifically with ACM drives me nuts.

I noped away from Terraform back in 2019. At the time, the state management component was too immature, and it was a little.. lose with credential security.

You may like Ansible far more if you’d like something a bit more declarative, readable, and with “batteries included.” I have a close friend who is an EE and he finds tools like Ansible much more intuitive and enjoyable than Terraform and CloudFormation.

Every tool has its master.

[lifebeyondfife]

I created a CloudFormation script specifically to do this (including CloudFront for CDN). Yes, there's reasonable complexity, but it's a trade-off of security and configuration. As others noted, the documentation is good, and this took me a weekend to write including learning CloudFormation from scratch.

https://github.com/lifebeyondfife/simple-static-website

[quickthrower2]

I prefer log into Hostgator, click cPanel, click File Manager, drag files across. Any day!

Ok I missed: point DNS to Hostgator name servers. Click the button to install lets encrypt.

[hdjjhhvvhga]

> I eat well due to this mess but I know it's all so so so wrong.

That's the point - I do this too. I like the fact that AWS jobs pay well. I'm fair and I always point out to my clients that AWS is expensive and it will cost them more in the long run, and the vendor lock-in is considerable - but they don't care as "everybody is doing it." Well, that's fine with me, I also like to eat well.

[philliphaydon]

I learned cloudformation in a day. Everyone on HN said use terraforms. Tried to learn terraforms and it made things complicated. Went back to CF.

[fhfuewidxjhe]

best part about debuging you HCL/CL is when you cannot delete some resource and it won't tell you why besides "is in use".

I created account after account at first and then had hell deleting them entirely. was still easier than figuring out was using a vpc that had nothing visible attached to it

[zoover2020]

Are you aware of the AWS CDK?

Doing that in Infra as Code shouldn't take two days IMO.

[brunooliv]

If you don't use the SDK, how can you judge anything as being "overly complicated"? I mean, I don't know about you, but, last time I checked, signatures, certificates, security and all that stuff IS SUPPOSED to be super complicated because it's a subject with a very high inherent complexity in and of itself. The SDK exists and is well designed to precisely shield you from said complexity. If you deliberately choose not to use it or can't for some reasons then yes... The complexity will be laid bare

[naasking]

> I mean, I don't know about you, but, last time I checked, signatures, certificates, security and all that stuff IS SUPPOSED to be super complicated because it's a subject with a very high inherent complexity in and of itself.

Actually, security is not supposed to be complicated. "Complicated" is the anti-thesis of "secure".

[Gigachad]

HN users routinely try to do things the obtuse way and then complain when its hard. Throw in something about the SDK being spyware or not following the unix philosophy.

[BackBlast]

Have you ever tried to import the AWS SDK into a front end client? It's huge. Last time I tried it, it added multi MB of JS to my SPA, so I could do a relatively "simple" call using it. Yuck.

It did not tree shake cleanly with my build system and I eventually ended up just yanking AWS from the stack entirely.

[asah]

Curl is my preferred SDK.

[orf]

Your complaint isn’t about AWS, it’s about the authentication scheme. I find it to be pretty neat, especially when it’s flexible enough to create signed URLs for any method and send them to third parties. We use that as a basis for service-to-service auth. It’s cool and flexible. But overall this complaint seems pretty shallow. You’d just use the client SDKs they publish, or if you want to really go off the beaten path you could just use the signing methods from those SDKs with your own request/response calls.

And if you’re building your own SDK (why?), well there is a lot more complexity down the line once you get past authentication.

[omnibrain]

> And if you’re building your own SDK (why?)

There are more languages than C++, Go, Java, JavaScript, Kotlin, .NET, Node.js, PHP, Python, Ruby, Rust & Swift.

[mcqueenjordan]

You HMAC the region so that in case a region is compromised, other regions aren't as well. You HMAC the service so that in case a service is compromised, other services aren't as well, you HMAC the timestamp for obvious reasons (time bound the signature), the outer "aws4_request" HMAC, I'm sure there's a good reason for. Maybe just versioning? Not sure.

Also: All of this is handled in the SDKs. Anyone implementing this themselves either isn't using the right libraries or has a very special use case.

[dividuum]

Sounds like OP is having a bit of a Chesterton's Fence moment. It's clearly complicated, but you describe why this is probably implemented the way it is. The scheme reminds me of Macaroons: https://blog.gtank.cc/macaroons-reading-list/

[naasking]

I'm sure there are "reasons" why the HMACs are layered, the question is, does using HMACs actually add useful security properties here, or is this layered HMAC really just a way to generate a cryptographically secure id? If the latter, then can't you just generate that directly rather than needing to gather all of the right information and HMAC it in just the right way?

[carabiner]

I did AWS training at the Amazon offices in Seattle for data science. I was blown away by the configuration... I have recompiled linux kernels and configured iptables as a teenager, and this was an entire galaxy of more complexity. It took us 6 hours to the point where some of us had a Jupyter Notebook running. Many people didn't make it though.

[rippercushions]

Were you doing things the hard way on purpose? It's literally one click to spin up a Jupyter notebook on GCP, arguably less if you use Colab (instant access to shared instances), and I'm sure AWS has a similar service.

https://cloud.google.com/vertex-ai-workbench

[crazygringo]

Why?

That doesn't make any sense to me. What problems were you running into? Did you need to configure a bunch of super-non-standard stuff or something?

[goodpoint]

Artificial complexity to justify lock-in and artificial salaries.

[kureikain]

I though its just me who find it overly complicated.

With people who use AWS SDK its all abstracted out. But there is time I just want to send a damn `curl` to download a S3 file and yes, doing the dance in bash isn't easy.

There is time I wrote a Lua plugin for openresty to fetch s3 and. I have to trial and error with lot of debugging. The ordering. the timestamp format...all of that...

[darrenf]

> But there is time I just want to send a damn `curl` to download a S3 file

You can use curl's `--aws-sigv4` option, then: https://curl.se/docs/manpage.html#--aws-sigv4

[gw98]

It's even worse. We use presigned URLs (from the SDK) and S3 for file uploads. That's one fresh hell you don't want to get involved with. One of those seemed like a good idea at the time things...

[antonvs]

What does Azure’s security system for object storage look like? Because if it’s not doing something like this, then I have questions about their security.

The reality is is that what AWS is doing here is all fully justified and normal, and that’s why SDKs exist. Like many developers, you’re being a bit clueless about security here. Instead of complaining, I recommend educating yourself - look into capability security and signed tokens in general, and you’ll understand what AWS is doing and it’ll make you a better developer.

[dan-robertson]

Whether or not AWS is overly complicated, I don’t think this is a good example.

Any decent api should have some kind of signatures which work roughly like:

  signature = hmac_sha256(secret, url+payload+time+etc)
  set_header(…, auth_header(signature))

And the point is to prove that you know the key without sending it to AWS / your logs, and to prove that the request came from someone with the key ;and not eg someone replaying a message from logs which was either old or modified).

Most of the mess seems to be due to AWS wanting to limit the power of the secrets that they store, presumably in case they’re compromised. You can get a reasonably good idea of their architecture by looking at how you construct the derived secret – each inner layer will be more tightly controlled than the one around it.

[gregjor]

Lots of APIs do HMAC signing, not really something I get worked up over.

I use AWS and Azure. Each has their advantages and disadvantages. To answer if something seems "overly complicated" I would have to know, compared to what? AWS certainly has lots of complexity, but so does any services with so many features and moving parts and potentially catastrophic misconfigurations and security threats. Compared to managing servers in a rack I prefer AWS.

[mabbo]

First, most users are using the SDK that Amazon provides.

But second, most of this complexity is in the name of security. I have plenty of criticism of my former employer (Amazon), but I have never once felt that AWS didn't take security seriously.

Use the SDK. Be happy that it will keep your calls very secure.

[cpach]

Makes a lot of sense IMHO.

Because of their size they are a very interesting target for cyber-criminals. If Amazon wouldn’t make security a top priority it would all tumble down pretty quickly.

[perryizgr8]

Can confirm this was a intense 12 hours of work to get it working properly. Their example and explanation in the docs is also very vague and borderline misleading at points.

Another example is Google cloud run vs elastic container service. Cloud run is dead easy to get up and running. ECS is a mess of confusing terminology and unnecessary complexity. I literally do not care what class of machine you will use to run my container. If I cared that much I would have used a proper EC2 instance.

[throwaway894345]

Have you looked at ECS Fargate? That should manage the instances for you transparently. It’s been around for probably five years or so.

[tqwhite]

Yes it's too complicated. It's horrible. It's the Windows of cloud services, surpassed in obscurity and nastiness only by Microsoft's own Azure, which is a nightmare.

In both cases, the UI and UX designers should be shot. I understand that they are supporting a lot of complicated things. It's a huge fail, unusable by anyone who hasn't dedicated their lives to learning.

[neilv]

You probably want to be using one of the popular libraries to do that.

AWS documentation isn't consistently good, and there are lots of decoys. So, if you wound up in some local minima that made it look like you had to implement the signing and such yourself, try a Web search that includes the name of your chosen programming language.

(A long time ago, I had to implement the AWS client API from scratch, but that's because I was using a fringe language that didn't have such libraries.)

[adaml_623]

I like your language of decoys and local minima. It's very easy to follow out of date documentation and find yourself wasting time in a cul-de-sac

[arielcostas]

My main problem with AWS has always been the console UI being horrendous and inconsistent, the poor documentation and having far too many services.

Also, AWS not having something like Azure's resource groups or GCP's projects and instead telling you to create accounts for different projects and environments, and using AWS Organizations or however they call it sounds like a huge PITA.

[benjaminwootton]

IAM is the one for me. I can stumble through the basics, but trying to keep everything least privilege adds a layer of pain to everything relatively complex that I try to do.

[KaiserPro]

Yeah this is normally why you'd use the SDK, because its just horrific when you're outside.

However, having said that, compared to FAANG internal tools, its actually not that bad. It's at least vaguely consistent

[damacaner]

I am trying to work with MWS, and AWS SDK of GoLang horribly fails when tried to use standalone (I only need the signer part), but oh my oh my.

https://github.com/canercetin-randomguy/club-noira/blob/main...

This is only the auth&sign part. and it doesnt work. and it is already 150 LoC. god.

[iLoveOncall]

> However, having said that, compared to FAANG internal tools, its actually not that bad. It's at least vaguely consistent

Actually I find a lot of Amazon's internal tools to be a breeze to use compared to AWS.

Basically all the painful Amazon-specific stuff is done for you so it makes everything very easy.

Some stuff is bad obviously but what we use the most is pretty freaking good.

[rco8786]

I find basically all of AWS to be a nightmare of complexity. Probably great if you're in devops or infrastructure or something, but the story of "write code and deploy it" is horrific.

[ejb999]

That is why they give you the SDK - if you choose to not use it, and unnecessarily make your work more difficult - thats on you.

Most people don't build their signatures by themselves - they use the SDK provided.

[ninefathom]

I don't necessarily think that AWS is overly complicated. What I do think is that the entire ecosystem of tools is very powerful, and often times finding the "easy button" for a given task (and there usually is one) is far more time-consuming than it should be.

Many a time* I wished that Amazon would have a single easy-to-find documentation page listing common use cases and the de facto standard third-party tool (and there usually is one) to make a given case easier.

Fun side note: as a security professional, I cannot even begin to tell you how many times I've heard app owners claim that "AWS handles security for us" or some variation thereof.

*It's been a few years since I messed with AWS in depth; for all I know such a page may exist now.

[DangitBobby]

AWS is an absolute nightmare of complexity. Try out GCP. With minimal experience, you'll find you can actually accomplish things in the cloud console without consulting a single piece of outside documentation! It's wild.

I like this article which echos some of my experience. https://nandovillalba.medium.com/why-i-think-gcp-is-better-t...