Forget CDK and AWS's insane costs. Pulumi and DigitalOcean to the rescue

[jmspring]

Pulumi is really a royal piece of shit. Why the f*ck am I writing code to do "deployment". In C# --> new Dictionary<string, object> when dealing with a values.yaml for instance. The whole need to figure out when and when not to use Apply.

Give me Terraform (as much as I hate it) any day.

[stackskipton]

As SRE dealing with former Pulumi, "Hey Devs can use code to deploy infrastructure" is not great idea you think it is. I've seen some real ugly conditional behavior where I'm like "Is this or is this not going to run? I honestly can't tell."

[Longwelwind]

I would agree with you, if HCL wasn't a bad language in itself:

* You can't make have variables in an import block (for example, to specify a different "id" value for each workspace)

* There is no explicit way to make a resource conditional based on variables. Only a hacky way to do that using "count = foo ? 1 : 0"

* You can't have variables in the backend configuration, making it impossible to store states in different places depending on the environment.

* You can't have variables in the "ignore_changes" field of a resource, making it impossible to dynamically ignore changes for a field (for example, based on module variables).

* The VSCode extension for HCL is slow and buggy. Using TS with pulumi or TFCDK makes it possible to use all the existing tooling of the language.

[Hawxy]

As much as I like it, I find C# to be too inflexible of a language for infrastructure code. I tried with Pulumi for a while but moved to TypeScript as it works so much better. Structural typing makes your life a lot easier.

[cruffle_duffle]

> Give me Terraform (as much as I hate it) any day

Terraform sure is a quirky little DSL ain’t it? It’s so weirdly verbose.

But at the same time I can create some azure function app, setup my GitHub build pipeline, get auth0 happy and in theory hook up parts of stripe all in one system. All those random diverse API’s plumbed together and somehow it manages to work.

But boy howdy is that language weird.

[postalrat]

Why are people templating yaml for terraform like they templated html in php in 1996?

[arkh]

Tried Pulumi thinking "it's gonna abstract all the k8s specifics". Welp no, still need to know and understand K8s so I still don't see the value from those kind of tools. In which case why not use something like Pkl to generate my yaml from some sensible code-like structures?

[rusty-jules]

What about pulumi's declarative yaml interface which can be exported from type-safe languages like cue? https://www.pulumi.com/blog/extending-pulumi-languages-with-...

[nuker]

> Give me Terraform (as much as I hate it) any day.

Just use CloudFormation. Easy to write, declarative, vars (Parameters and Output exports). Trick is not to pile everything in one Stack. Use several.

[klysm]

Apply is really straightforward. The dictionary stuff is very annoying overhead but it’s nice keeping everything in one language.

[nothrabannosir]

For anyone deliberating between Pulumi and CDK let me recommend what I consider the best of both worlds: CDKTF, Hashicorp’s answer to Pulimi (my quote not theirs).

It’s got everything you want:

- strong type system (TS),

- full expressive power of a real programming language (TS),

- can use every existing terraform provider directly,

- compiles to actual Terraform so you can always use that as an escape hatch to debug any problems or interface with any other tools,

- official backing of Hashicorp so it’s a safe bet

It’s a super power for infra. If you have strong software dev skills and you want to leverage the entire TF ecosystem without the pain of Terraform the language, CDKTF is for you.

(No affiliation)

https://developer.hashicorp.com/terraform/cdktf

[ranguna]

Cdktf is good, but it's not amazing. You are still constrained by terraform syntax like `count = condition? 1 : 0` , instead of doing a normal` if` statement. And there's a fairly good amount of times where you need to use terraform iterators instead of doing a normal for/forEach/map/reduce.

But all in all, it works. It's just a bit limited on what you can do with the actual language.

[eadmund]

> - full expressive power of a real programming language (TS)

I suppose TypeScript does count as a real programming language, in that it’s Turing complete. But I can use Pulumi from (they claim) any programming language. Specifically, I can use it from Go. Why would I add TypeScript to my project when I can live in one language?

> - official backing of Hashicorp so it’s a safe bet

Given the number of folks leaving the Hashicorp platform, I think it’s arguably no longer a ‘safe bet.’

[ivantop]

How is compiling to terraform a positive? I'd rather debug python than python-compiled-to-terraform.

[terminalbraid]

Does Typescript offer a strong type system?

[turtlebits]

I wish CDK was fully baked enough to actually use. It's still missing coverage for some AWS services (sometimes you have to do things in cloudformation, which sucks) and integrating existing infra doesn't work consistently. Oh and it creates cloudformation stacks behind the scenes and makes for troubleshooting hell.

[Aeolun]

> sometimes you have to do things in cloudformation, which sucks

All of CDK does things in cloudformation, which made the whole thing stillborn as far as I’m concerned.

The CDK team goes to some lengths to make it better, but it’s all lambda based kludges.

[roncesvalles]

CDK is an abomination and I'm not sure why AWS is pushing it now. A few years ago all their Quick Starts were written in CloudFormation, now it's CDK that compiles to CloudFormation. Truly a bad idea.

Just write CloudFormation directly. Once you get the hang of the declarative style and become aware of the small gotchas, it's pretty comfy.

[LunaSea]

The biggest hurdle I've encountered is cross-stack resource sharing, especially in case of bidirectional dependencies like KMS keys and IAM roles.

[x0x0]

I also had a really rough go with cdk. I personally found the lack of upsert functionality -- you can't use a resource if it exists or create if it doesn't -- to make it way more effort than I felt was useful. Plus a lack of useful error messages... maybe I'm dumb, but I can't recommend it to small companies.

[yieldcrv]

Cant use bun to deploy CDK, CDK fails as it looks for package-lock yarn-lock or pnpm’s exclusively

So dumb. Trying to move to SST for only that reason

but if you add cdk to the path, you can still deploy, its just that your cicd and deployment scripts are not all using bun anymore

[petcat]

Kubernetes no thanks. Terraform + Kamal [1] on Digital Ocean is the way I deploy/run apps now.

[1] https://kamal-deploy.org/

[mati365]

Plain Podman systemd integration is way more powerful and secure, as it does not mess with firewall and allows to run rootless containers using services. It's even possible to run healthchecks and enforce building images just before starting service making on-demand containers using systemd-proxyd possible. Check example: https://github.com/Mati365/hetzner-podman-bunjs-deploy

[stackskipton]

I've looked into Kamal but it feels so "It's as complex as Kubernetes but isn't so support is going to be nightmarish."

Why is this better then Ansible + Docker Compose?

[mplewis]

Kamal is simply NIH K8s made by an unreliable company with poor leadership. No thanks, not for my prod infra!

[thinkindie]

Pulumi genAI-based documentation is trashed. I've moved to terraform and i was able to achieve much better results in shorter time thanks to higher documentation level for terraform.

[tholm]

Worth noting that most of the terraform documentation for classic pulumi providers (providers build on top of TF providers) is still relevant to Pulumi.

[mavdi]

Hi everyone,

We've gone through a lot of pain to get this blueprint working since our AWS costs were getting out of hand but we didn't want to part ways with CDK.

We've now got the same stack structure going with Pulumi and Digital ocean, having the same ease of development with at least 60% cost reduction.

[vundercind]

Keep an eye on reachability and performance. I’ve seen DO consistently perform terribly and/or drop connections for months (that is, didn’t look like some brief routing glitch somewhere) for some US and Canadian routes (not, like, Sri Lanka or something) on excellent Internet connections. The fix was moving to AWS, problem gone. It felt like a shitty-peering-agreements issue.

[skywhopper]

Please change the title text unless you add some discussion of the cost differences to the page you linked. However useful your tool is, nothing on this page mentions AWS or costs.

[Aeolun]

I don’t think Digital Ocean is all that much better for pricing, but using Pulumi over CDK is a pure win as far as I’m concerned.

[JamesSwift]

Agreed. On the bright side, I was able to migrate managed k8s on DO to managed k8s in GCP with very minimal work since it was managed via pulumi.

[CSMastermind]

Yeah, I've been really disappointed with Digital Ocean so far. Not just from a pricing perspective but from a customer service perspective.

Anyone using CDK should switch to Pulumi though.

[thelittleone]

Perhaps Pulumi with Vultr is also worth a look.

[fulafel]

Why's everyone going away from declarative? Terraform, CloudFormation, AWS Copilot etc have a lot of virtues and are programming language agnostic.

Using a complex programming language (C++ of the browser world) just for this has a big switching cost. Unless you're all in on TS. And/or have already built a huge complex IaC tower of babel where programming-in-the-large virtues justify it.

[jnsaff2]

> Why's everyone going away from declarative?

If I had to guess it's because

- more imperative background developers need to work with infrastructure and they bring over their mindset and ways of working

- infrastructure is more and more available through API's and it saves a lot of effort to dynamically iterate over cattle than declaratively deal with pets

- things like conditionals, loops and abstractions are very useful for a reason

- in essence the declarative tools are not flexible enough for many use cases or ways of working, using a programming language brings infinite flexibility

Personally I am more in the declarative camp and see the benefits of it, but there is certain amount of banging ones head against it's rigidity.

[1dom]

Complex programming languages for infrastructure code get used when people who are more comfortable using complex programming languages to solve their problems are given the problem of infrastructure and ops.

It is classic "every problem is a nail to the person with a hammer". Complex languages - by definition - can solve a wider variety of problems than a simple declarative language but - by definition - are less simple.

Complex languages for infra - IMO - are the wrong tool for the wrong job because of the wrong skills and the wrong person. The only reason why inefficiencies like this are ever allowed to happen is money.

"Why hire a dev and an ops when we can hire a single devops for fractionally less?" - some excited business person or some broken dev manager, probably.

[vundercind]

Declarative has in-practice meant “programming, but in YAML” more often than not, which is hell. YAML’s not even a good format for static data, and it’s even worse when you try to program in it.

[anothernewdude]

Terraform isn't really declarative. It's declarative right up until the point at which it isn't, where it falls apart. I need a declarative deployment right up to the application layer, which is where terraform fails.

[pjmlp]

Because they like to spend endless hours debugging infrastructure builds.

[nasmorn]

A small CDK project is a lot more readable in my opinion. It doesn’t have a ton of yml files where your config is spread out

[fragmede]

Because sometimes you just need a for loop in a way that terraform's for_each/other DSL doesn't support

[nsonha]

declarative does not equate to config files

the property that equates to config files is "being static", which modern deployments are not.

[nextworddev]

Controversial opinion here: just use CDK. Learn cloud formation for advanced stuff. It’s really not that hard and pays dividends

[coredog64]

Just learn CloudFormation. It’s not that hard, and if you really want to write code, you can implement custom resources for all the times the service team let you down.

[__turbobrew__]

CDK is a second class citizen, it is missing implementations for many services and features. CDK was DOA as it should have been a requirement that when AWS added something to terraform it needed to be added to CDK as well.

[ptdorf]

In my experience AWS' CloudFormation is limited in the number of resources and exposed APIs than any of the CDK.

[mythz]

Hetzner has been our "expensive AWS cloud costs" saviour

We've also started switching our custom Docker compose + SSL GitHub Action deployments to use Kamal [1] to take advantage of its nicer remote monitoring features

[1] https://kamal-deploy.org

[KronisLV]

I’ve been pretty happy with something like Docker Compose or Docker Swarm and Portainer, but honestly it’s nice that there are other alternatives that strive for something manageable and not too complex!

[jmspring]

One thing about managing EKS with Pulumi, Terraform, etc. if you deploy things like Istio that makes changes to infrastructure. Do a Terraform destroy - no luck, you are hunting down maybe some security groups or other assets Istio generated that TF doesn't know about. Good times.

[skywhopper]

This title text is nowhere on the linked page. Please get rid of the editorialization. DO is not that much cheaper for a baseline instance.

[lysace]

Pulumi is very neat with straight AWS, too. I suspect this is the primary use case.

[giorgioz]

CDK APIs in JavaScript are very nice. It's a much much developer experience than Pulumi/Terra form and even Server less Framework. In our monorepo each service is in a separate folder with a folder called /infrastructure inside with a file called Stack.js that defines all the resources needed. When starting a new service we just copy one of the last similar services that we developed. We are able to deploy a new service in hours. Services are getting better and better with accumulation of nice to have features that you wouldn't have time to add to most services.

[lazzurs]

This doesn’t sound good to me. Would you do the same with some functional code rather than creating an external versioned library?

Terraform or CDK I would want a simple shareable thing that did the boilerplate that I called with any variables I needed to change.

[nasmorn]

My DO K8S cluster ist bugging me every couple of months to do an upgrade. I am always scared to just run it but moving shit over to a new cluster instead is so much work that I simply gamble on it. AWS ECS is worth over penny

[katdork]

DO's K8S is more equivalent to AWS's EKS offering, so of course ECS which abstracts away pretty much all of the other parts of K8s is going to require less maintenance. It's sort of a false equivalence to say ECS == that solution.

On EKS, you need to do the same version updates with the same amount of terror.

You do pay the extra for the further management to just run containers somewhere!

(you might want to say "every" instead of over, "is" instead of "ist")

[wordofx]

It’s only “insane costs” if you don’t know what you’re doing.

[postalrat]

Or need a good amount of ram. Which should be really cheap these days.

[yieldcrv]

and even if you do, it’s usually a system design problem that you’re maintaining

on one hand, I can see how this is an unfalsifiable standard, on the other hand I can see the utility of solving a friction for people that messed up

[mise_en_place]

EKS has become a clusterf*ck to manage and provision. This looks very useful. Bare metal k8s, even running on EC2, might be another option.

[GauntletWizard]

You don't choose EKS because it's easy to manage. You choose it because you intend to use the bevy of other AWS hosted services. The clusterfuck of management is directly related to that.

The alternative, which I feel is far too common (and I say this as someone who directly benefits from it): You choose AWS because it's a "Safe" choice and your incubator gets you a bunch of free credits for a year or two. You pay nothing for compute for the first year, but instead pay a devops guy a bunch to do all the setup - In the end it's about a wash because you have to pay a devops guy to handle your CI and deploy anyway, you're just paying a little more in the latter.

[trallnag]

What's your issue with EKS? I operate several very simple and small single-tenant clusters, and I have to touch the infrastructure only once a year for updates

[RoxaneFischer1]

I personally love terraform. It's easy to use and actually it's rigid framework allow to make less mistakes/way more readable than pulumi

[strzibny]

You can also simplify Kubernetes to just Kamal and things become instantly easier...

[pmarreck]

Anyone use Garnix? https://garnix.io/

[mplewis]

This looks too experimental for me to trust with production deployments.

[kristianpaul]

Is this an Ad?

[nextworddev]

GitHub has been littered with developer relations growth hacks recently.

[icar]

I strongly recommend sst.dev

[nixdev]

Digital Ocean isn't really a "real" cloud. Maybe use Digital Ocean if you're hosting video game servers, but no serious business should be on it.

[Sohcahtoa82]

I wouldn't even use DO for that, unless it's like a private server for just your friends.

I won't touch DO after they took my droplet offline for 3 hours because I got DDoS'd by someone that was upset that I banned them from an IRC channel for spamming N-bombs and other racial slurs.

[Dylan16807]

What's your definition of real cloud?

And can you name a real cloud that charges a half-reasonable price for bandwidth? I consider $10/TB to be half-reasonable.