top | item 37537689

Running one’s own root Certificate Authority in 2023

216 points| jandeboevrie | 2 years ago |wejn.org | reply

161 comments

order
[+] jordemort|2 years ago|reply
I really like step[1] and step-ca[2] for this, it's a lot less fiddly than having to drive OpenSSL directly.

1. https://github.com/smallstep/cli

2. https://github.com/smallstep/certificates

[+] infogulch|2 years ago|reply
Wouldn't it be nice if LetsEncrypt could issue you a (1) name constrained, (2) 90-day limited intermediate CA with just the (3) DNS-01 challenge? I argue that such an intermediate CA would have no more authority than a wildcard cert which you can get today, so they should be able to issue it. [1] Everything supports name constraints now, which used to be an issue but isn't anymore. [2]

Then stick it in step-ca and issue all your certificates with internal ACME.

This would solve a lot of problems, such as leaking private hostnames in the certificate transparency log, or hitting issuance rate limits on LE servers.

[1]: https://news.ycombinator.com/item?id=29811552

[2]: https://bettertls.com/

[+] mattpallissard|2 years ago|reply
I completely agree that dealing with openssl is fiddly at best when you RYO CA. But, if your org knows how PKI works it's not a big deal; a bash script or two in the simple case or a flask app at most complex.

If your org doesn't know how PKI works shouldn't you be paying a vendor that does?

[+] wejn|2 years ago|reply
That's really neat, thanks for the pointer. ;)
[+] obnauticus|2 years ago|reply
I eventually need to publish an article about how to run an HSM backed root CA on the cheap with m of n auth.

Using nitrokey and some glue scripts you can get the cost below $500. If anyone is interested, let me know.

[+] dmayle|2 years ago|reply
I've just started down that route. I've got the nitro key hsm2 in the mail, have heard the advice on using two levels (first root in the Key, and intermediary on the Device for easier revoking). I mainly want to issue client certificates so that I can expose internal sites on the public Internet via proxy without having to require a VPN for all of my users, though I'm also interested in certificate based SSH
[+] n0n|2 years ago|reply
Yes, please! I would be interested. Currently i'm fiddling around with vault as an ICA, so this sounds like a good next step
[+] Lienetic|2 years ago|reply
+1 sounds like an interesting read
[+] temren|2 years ago|reply
an article like that would be great!
[+] jiggawatts|2 years ago|reply
“Cultural” technical issues are so frustrating to me. A certificate is fundamentally just a type of credential, like a password, but for historical reasons they’re treated like getting citizenship papers. There’s always this ceremony even in scenarios where it makes zero sense — such as internal-use certificates used for a gRPC API server behind a load balancer.

Why - for the love of God why - can’t I just obtain a cert like this directly out of a secret store such as an Azure Key Vault!?

These things are already full hardware security modules (HSMs) with all of the capabilities required to run just about anything short of a public Root CA and maybe even that too.

But no.

NO!

Script it yourself. Make a root cert, “upload” it, make a cert, sign it, upload it, link it, renew it, re-configure it, and on and on. Oh… you wanted a CRL too? A 1kb file? Ha-ha! No. Make it and host it yourself!!

It’s absurd.

So many services depend on a simple CA->cert setup: VPNs, API auth, load balancer back-ends, clusters, etc…

But my mark my words: no cloud will have a trivial turnkey solution this decade.

This is because running a CA is culturally accepted to be a hard problem. It is! If you’re DigiCert. It isn’t if you’re building a three-server internal use cluster. But the problem is hard, you see? That accepted fact! Everyone knows it! Ceremony is required. We can’t just hand your sever a 1kb credential file! That would be… unconventional!

It’s just not the way things are done, so stop asking.

[+] gorkish|2 years ago|reply
It’s about damn time that we should be able to get intermediate signing certs that are domain limited from a public ca and issue out own “real” certs. This is fully supported in the standards. Anyone offering this product affordably yet?
[+] mdekkers|2 years ago|reply
> Ceremony is required.

Ceremony, as in “a well established, rigorously implemented, meticulously documented, and diligently audited process” is required to establish and maintain trust. A CA promises that folks showing up bearing their certificate _are who they say they are_ - this doesn’t mean it is perfect (there are plenty of cases where public CAs went wrong) and some random person can place their trust in that assertion.

The whole _point_ of a PKI is trust, and why would this be any different for internal systems?

[+] diarrhea|2 years ago|reply
A password is a secret. A certificate is fully public (it’s useless if not). I don’t see how they’re similar in that dimension. Or how certificates are possible without a third party (a core reason for all the ceremony). Passwords get away with not needing a third party involved because they’re a prearranged process. Certificates aren’t, I need to be able to hit any website at any point, for the first time.
[+] c0l0|2 years ago|reply
Actually running a CA, even if only for private purposes, without certain regret down the road involves more than creating an OpenSSL cnf file, creating a root cert/key, and running with it. That said, it's a starting point. If you're looking to use more modern (i.e., faster) crypto than RSA keys, maybe check out my sping on a CSR generator wrapping `openssl`, available at https://johannes.truschnigg.info/code/tls_req_gen

If you need a self-signed cert instead, maybe try https://johannes.truschnigg.info/code/tls_cert_gen

[+] codetrotter|2 years ago|reply
I run a squid proxy with TLS intercept on a raspberry pi, with my own CA.

I have things set up so that the RPi connects to a WiFi, and then a cable from the RPi goes to another WiFi router.

I connect my MacBook Pro to that other router.

This way the MacBook Pro cannot reach the internet.

Then I set the http and https proxy configs in Firefox so that it goes via the squid on the RPi. And I have the root CA from the RPi trusted in Firefox.

Additionally I have set some env variables and added my root CA cert to some cert storages on the computer, so that git can clone via squid, and I can install and update things with brew etc.

It works great :D

But then I tried to set up my iPhone to also connect to that WiFi. I think I managed to trust my root CA on the phone. But I couldn’t manage to set up the http/https proxy on the iPhone and so for now only the MacBook Pro can use it, and not the iPhone

[+] lini|2 years ago|reply
macOS uses certificate pinning for some .apple.com and .itunes.com sites. If you pass all your traffic through the proxy, some stuff like the app store will not work. Do you bypass the proxy for those or just let them fail?
[+] alexeldeib|2 years ago|reply
I think iOS has http proxy settings in the wifi configuration for a given network? Haven’t tried recently.
[+] denysvitali|2 years ago|reply
You can use a transparent proxy to avoid this
[+] NegativeK|2 years ago|reply
I went with DNS based Let's Encrypt for internal certificates, since I'm okay leaking my internal DNS names.

> An obvious downside of this is having to guard a bunch of secrets and the need to rotate the host certificates yearly – because Apple says so.

The guarding secrets thing makes me too uncomfortable with managing my own CA. I'm sure it'd be fine, but since there are other equivalent and safer ways to do it.. Name constraints are a thing in the spec for restricting your CA to specific domains (which is amazing,) but browser/etc support was crappy when I looked at it and maybe getting better? I don't understand why name constraints aren't implemented everywhere. Unless an enterprise environment is doing TLS inspection, name constraints are a way saner implementation.

[+] woodruffw|2 years ago|reply
> I don't understand why name constraints aren't implemented everywhere.

They have weird semantics, especially in scenarios with multiple prospective validation paths: path `EE -> ICA' -> ICA'' -> TA` might result in different name constraints than `EE -> ICA' -> ICA''' -> TA`, resulting in hard-to-debug validation errors (or successes) depending on the user's trust store state.

(I don't believe that's why Chrome doesn't support them, however. Chrome's stated reason[1] is that they don't support them on user roots because RFC 5280 doesn't require them to, which is IMO a correct reading of the spec.)

[1]: https://bugs.chromium.org/p/chromium/issues/detail?id=107208...

[+] mirchiseth|2 years ago|reply
I used to have my own local root CA as well but now trying the Let's Encrypt with DNS-01. What is the easiest combination of software to try it? I have failed miserably trying Opnsense + ACME client plugin + Cloudflare DNS + HAProxy / NGinx. I would get 100% ssllabs certs but somehow the reverse proxy won't forward to internal services. Next I am gonna go caddyserver for reverse proxy as it has SSL with LE inbuilt. Let's see.
[+] fiddlerwoaroof|2 years ago|reply
The one thing you can’t do with Let’s Encrypt is generate a certificate with a CN of localhost which, since browsers are getting really picky about mixed HTTP/HTTPS content, is a real issue with local development using certain web features.
[+] soraminazuki|2 years ago|reply
> Name constraints are a thing in the spec for restricting your CA to specific domains (which is amazing,) but browser/etc support was crappy

It's well supported now. I use it and it works for OpenSSL, Firefox, and Safari.

Personally, I don't think there's much to gain from using public PKI for internal infrastructure. I already manage secrets on my personal devices and this is no different. Also, being able to issue certs for .home.arpa domains is nice too.

[+] wejn|2 years ago|reply
Thanks, I've incorporated the name constraints into the article now. (it is indeed supported by Apple and FF just fine)
[+] hnarn|2 years ago|reply
> I went with DNS based Let's Encrypt for internal certificates, since I'm okay leaking my internal DNS names.

Lets Encrypt offers wildcard certificates, there is no reason to have internal DNS records exposed.

[+] aberoham|2 years ago|reply
Did you use elliptic curve instead of RSA?
[+] andrewstuart2|2 years ago|reply
I've done similar for something like 8 years with vault as my intermediate issuer, almost exclusively using cert-manager once that was mature enough, and my own little utility before that. It's so nice getting certs for side projects or self hosting in an instant and with an encrypted (pgp) offline (flash drive in a safe) CA I'm never really worried about having to reroll. Installing the CA is pretty trivial on most devices and means I don't have to worry about CTLs or rate limits, which is especially helpful when I'm hacking on a saas side project that ends up requesting 10+ certificates every test run.
[+] JackGreyhat|2 years ago|reply
Easy-rsa to the rescue. Been using it for a while, works great and makes life easier :)

Link: https://github.com/OpenVPN/easy-rsa

Summary from that page:

easy-rsa is a CLI utility to build and manage a PKI CA. In laymen's terms, this means to create a root certificate authority, and request and sign certificates, including intermediate CAs and certificate revocation lists (CRL).

[+] suckitsam|2 years ago|reply
It's funny how I go from "YEAH! CA's SHOULD be a quick one-liner!" to "Should laymen be generating Root CAs?" inside of like 10 seconds of scrolling.
[+] morpheuskafka|2 years ago|reply
Doesn't the first Apple link specifically say the 398-day limit doesn't apply to self-signed CAs?

> This change will affect only TLS server certificates issued from the Root CAs preinstalled with iOS, iPadOS, macOS, watchOS, and tvOS.

> This change will not affect certificates issued from user-added or administrator-added Root CAs.

The second link about the other restrictions (including <=825 days validity) does appear to apply to all CAs.

[+] wejn|2 years ago|reply
And yet, my homegrown root CA cert with 3650 days of validity hums along just fine...

[edited: but since I also want to have host certs that are on various internal servers, the short validity applies to them]

[+] pxeboot|2 years ago|reply
I am not sure if the same rules apply to 802.1x authentication, but we use self signed certs with 2 year validity for EAP-TLS and have never had any issues on iOS devices
[+] nemo|2 years ago|reply
This is not really an Apple thing, it's an industry trend (and a good one IMO). Apple's generally applying the same criteria Chrome is: https://chromium.googlesource.com/chromium/src/+/HEAD/net/do...
[+] xg15|2 years ago|reply
Seems Chrome is specifically making an exception for custom root CAs though:

> This will only apply to TLS server certificates from CAs that are trusted in a default installation of Google Chrome, commonly known as “publicly trusted CAs”, and will not apply to locally-operated CAs that have been manually configured.

[+] galaxyLogic|2 years ago|reply
I'm building an app whose GUI runs in the browser talking to a local http server. The app (if and when it is ready) would be distributed as a standalone executable. A bit like Atom/Electron I guess.

What I haven't figured out yet is how the browser-GUI could talk to its local backend-server over https. Can an exe contain its own root-certificate-authority somehow in a way that the app-exe can work without having to update that certificate part ever?

[+] 1vuio0pswjnm7|2 years ago|reply
Trivia question: What is the limit, if any, for how many DNS names (SANs) one can include in a single self-signed certificate. It is common to see TLS proxies that will generate certificates on the fly as SNI in clientHell is received. Is this due to a limit on how many SANs one can include in a single certificate.

(Maybe the performance degradation of generating a certificate on the fly is less than using a certificate that includes 100s of SANs.)

[+] fsmv|2 years ago|reply
I made a web server / microservices thing that issues certs for clients from a CA root it automatically generates. Then internal reverse proxy connections use that cert so the whole path is TLD encrypted with full cert validation.

https://github.com/fsmv/daemon

[+] goku12|2 years ago|reply
Looks like step-ca/step-cli [1] and mkcert [2] have been mentioned. Another related tool is XCA [3] - a gui tool to manage CAs and server/client TLS certificates. It takes off some of the tedium in using openssl cli directly. It also stores the certs and keys in an encrypted database. It doesn't solve the problem of getting the root CA certificate into the system store or of hosting the revocation list. I use XCA to create and store the root CA. Intermediate CAs signed with it are passed to other issuers like vault and step-issuer.

[1] https://smallstep.com/docs/step-ca/

[2] https://github.com/FiloSottile/mkcert

[3] https://hohnstaedt.de/xca/

[+] linsomniac|2 years ago|reply
Shameless plug, there's also https://github.com/linsomniac/rgca

I've been using it at work for the last year for our certs and it's been quite nice. It can do pre/post hooks as well, so it directly commits the updated CA serial files to our git repo.

[+] remram|2 years ago|reply
I tend to treat service TLS certificates more like shared keys than PKI. Too many pieces of software don't handle revocation, it's easier to regenerate the CA and the entire set of certificates when you change your setup.
[+] kubanczyk|2 years ago|reply
If you control all the clients/browsers (i.e. you can immediately modify the required trust stores), you don't have any use for PKI whatsoever.

PKI's use case begins with shipping a trust store into the wild, where it will run unchanged for months or years.

[+] goku12|2 years ago|reply
The better solution is to do what vault does. Use only ephemeral certificates for servers and clients. It wouldn't be too hard to change them every week or so using the ACME protocol.
[+] leonletto|2 years ago|reply
There have definitely been many guides and I took a stab at this a few months ago https://github.com/leonletto/ca-for-labs. I tried to make it simple enough for anyone who is wanting to build an internal lab. Happy to receive any feedback or requests. No web interface yet. Thinking about building an interface that conforms to the ejbca api?
[+] wkat4242|2 years ago|reply
It's actually really difficult to import custom CAs now on Android. Since version 7 you can only import into the user store and apps ignore this one by default so it's pretty useless.
[+] lostmsu|2 years ago|reply
This is more like "making and installing own root certificate".

I expected the article to be about actually being a certificate authority that every browser will trust by default.