I think it's good advice to not pass secrets through environment variables. Env vars leak a lot. Think php_info, Sentry, java vm dumps, etc. Also, env vars leak into sub-processes if you don't pay extra attention. Instead, read secrets from a vault or from a file-system from _inside_ your process. See also [1] (or [2] which discusses [1]). Dotnet does this pretty good with user secrets [3].
> Instead, read secrets from a vault or from a file-system from _inside_ your process.
I’ve never liked making secrets available on the filesystem. Lots of security vulnerabilities have turned up over the years that let an attacker read an arbitrary file. If retrieving secrets is a completely different API from normal file IO (e.g. inject a Unix domain socket into each container, and the software running on that container sends a request to that socket to get secrets), that is much less likely to happen.
Disagree here. Basically if you use docker (which for most of the stuff you mention, you should), environment variables are pretty much how you configure your docker containers and a lot of sever software packaged up as docker containers expects to be configured this way.
Building a lot of assumptions into your containers about where and how they are being deployed kind of defeats the point of using containers. You should inject configuration, including secrets, from the outside. The right time to access secret stores is just before you start the container as part of the deploy process or vm startup in cloud environments. And then you use environment variables to pass the information onto the container.
Of course that does make some assumptions about the environment where you run your containers not being compromised. But then if that assumption breaks you are in big trouble anyway.
Of course this tool is designed for developer machines and for that it seems useful. But I hope to never find this in a production environment.
Remember we are mainly talking about dev envs here. If you put the secret key in a file...where do you put the file? In a common location for all the dotenv instances? One per dotenv instance? What if people start putting it as a dotfile in the same project directory?
Secrets are nasty and there are tradeoffs in every direction.
Environment vars propagate from process to process _by design_ and generally last the entire process(es) lifetime. They are observable from many os tools unless you've hardened your config and they will appear in core files etc. Secrets imply scope and lifetime - so env variables feel very at odds. Alternatively Env variables are nearly perfect for config for the same reasons that they are concerning for secrets.
Tl/Dr; in low stakes environments the fact that secrets are a special type of config means you will see it being used with env vars which are great for most configs but are poor for secrets. And frankly if you can stomach the risks, it is not that bad.
Storing secrets on the filesystem - you immediately need to answer where on the filesystem and how to restrict access (and are your rules being followed). Is your media encrypted at rest? Do you have se Linux configured? Are you sure the secrets are cleaned after you no longer need them? Retrieving secrets or elevated perms via sockets / local ipc have very similar problems (but perhaps at this point your compartmentalizing all the secrets into a centralized, local point).
A step beyond this are secrets that are locked behind cloud key management APIs or systems like spiffe/spire. At this point you still have to tackle workload identity, which is also a very nuanced problem.
With secrets every solution has challenges and there the only clear answer is to have a threat model and design an architecture and appropriate mitigations that let you feel comfortable while acknowledging the cost, user, developer and operator experience balancing act.
It handles task running (wipe local test db, run linting scripts, etc), environment variables and 'virtual environments', as well as replacing stuff like asdf, nvm, pyenv and rbenv.
Still somewhat early days, tasks are experimental. But looks very promising and the stuff I've tried to far (tasks) works really well.
I second mise, it's been a nice replacement for direnv, asdf and makefiles for my use case. Much faster, still compatible with the old configuration files when needed and all in one tool for the new projects. Awesome.
I’ve switched recently from asdf for managing language & tool versions and the ergonomics are much nicer (eg one command vs having to manually install plugins, etc., more logical commands) It’s also noticeably faster.
Regarding the env vars features, a couple of relevant Mise issues around people trying to integrate env var secrets using SOPS, 1Password, etc.
Seconded. I changed from pyenv to mise because pyenv was slowing down my shell startup (probably the shims, which mise doesn't use by default), and I'm slowly using mise for more stuff. Right now, I'm using it to auto-turn on virtual environments and add project scripts to the PATH, and it works very well.
I haven't felt the need to use it as a task runner yet, but that's probably because I'm used to having a bunch of shell and Python scripts in a `scripts` folder.
The only reason I use .env is because it’s dead simple and very obvious as to how it works to anyone.
If now someone has to read docs to figure out how to configure the app, I’d rather have them read docs for some other safer and more powerful configuration scheme.
I've dealt with problems from python-dotenv and very much prefer it as a command than as a library.
As an example, once I changed a .env fikr and unit tests started failing. Digging deeper into it, and lots of code was checking for .env to load its configurations, and would break without it. I'd prefer this not to happen, as our tests were executing based on outside, not version controlled, configurations.
After removing dotenv as a library and using it only as a command, we were able to separate configuration and logic, and not have .env files affecting our unit tests - we simply ran the application with dotenv command, and the unit tests without.
With leaking secrets being such a big concern, it seems wise to require that secrets be encrypted to use dotenvx. That is, it will only work with encrypted secrets. As others have commented, this doesn't eliminate the risk entirely, but I think having a tool that doesn't support unencrypted secrets at all, although a bit less convenient, is a win.
I too have been using sops for years, and agree -- dotenvx encryption seems very similar to sops.
I'd prefer an integration between dotevnx and sops where dotevnx handles the UX of public env and injection, while leveraging sops for secret management and retrieval. Additionally, being able to have multiple keys for different actors is important.
Having a single `.env.keys` file feels risky and error prone. dotenvx encourages adding your various env files, such as `.env.production`, to vcs, and you're one simple mistake away from committing your keyfile and having a bad day.
If sops is not to be integrated, dotenvx could take some inspiration where the main key is encrypted in the secrets file itself, and you can define multiple age key recipients, each of which can then decrypt the main key.
I've been using sops in production since at least 2017, plus it has excellent compatibility with containerized infra tools like helm and other infra tools like terraform (both technically using plugins, but helm secrets and the carlpetts terraform plugin have been around for ages and are widely used.
I don't think encryption is a good idea, and the reason is forming bad habits. Now developers have a very strong and non-ambiguous habit: never put .env files in version control (except may be for .example.env). However, with this, you'll get accustomed to commit .env in _some_ projects, so you'll easily slip and commit it in another project where the vars are not encrypted.
Encrypting secrets and committing them seems very convenient but I'm paranoid about these sorts of things. Can anyone tell me why this would be a bad idea?
One reason I can think of is that normally with secrets I actually don't keep any copies of them. I just set them in whatever secret manager my cloud environment uses and never touch them again unless I need to rotate them. Meaning there is no way to accidentally expose them other than by the secret vault being hacked or my environment being hacked.
With this approach if someone gets access to the encryption key all secrets are exposed.
The biggest issue with storing secrets in version control with the code is that past secrets are never relevant after they have been rotated. This makes rollbacks risky. Consider:
1. Create secret v1
2. Code v1
3. Deploy
4. Secret v2 (rotation)
5. Code v2
6. Deploy
7. Oops, need to roll back to v1 (from step 2)
8. Outage, because the secrets in step 2 are not the secrets from step 4
I've used git-crypt and sealed-secrets and the problem is always backing up the master key. sealed-secrets rotates it every so often so you need to go find it and copy it to 1password or whatever you use as your root of trust. (We used a calendar invite for this.)
git-crypt is easy, the master key doesn't rotate, so don't leak it. (Secret encryption key rotation is kind useless; it's nice that if you leak an old key newer secrets aren't leaked, but it depends on your underlying secret rotation policy as to whether or not that saves you any work. I have tended to do them in bulk in the past.)
On my last project we did disaster recovery exercises every 6 months, sometimes with the master key intentionally lost, and it wasn't that big of a deal. Restoring your infra-as-code involves creating the secret encryption service manually, though, which is kind of a pain, but not like days of downtime pain or anything. Of course, if the secrets encrypted your database or something like that, then losing the master key equals losing all your data. Hopefully your database backup has some solution for that problem.
Can a kind soul point me some documentation on how to put .env in a vault correctly? Possibly open-source solutions?
If the vault is password protected, aren't you just adding one more indirection and nothing more? How is that helpful, since now I have to write the vault password in clear-text somewhere such that my application can read the env file from the vault?
A lot of modern cloud deployments read from a secret management system or vault at deployment time, and the secrets are made accessible to the application through various indirect methods so they cannot be accessed later on (i.e. if someone were to gain access to a running Kubernetes container).
At no point does the application have access to the vault itself, and access to read the vault is guarded by IAM role permissions.
Add your actual dotenv to .gitignore. Use bfg to make sure that any traces of your dotenv aren't in your commit history. Use detect-secrets client-side commit hook to confirm this and also monitor for secrets leakage. Use sops to encrypt your dotenv into some other file that is tracked by git. Sops can integrate with secrets management solutions (Vault, AWS KMS, etc.). Done.
This is mildly complicated, but the alternative is storing config in a configuration server somewhere, which comes with its own can of worms.
It makes it easier to update all env at once, compare, and share.
It's not much help, but it helps me avoid a few annoyances.
On an unrelated note, I always find it a real headache to keep the naming convention
of the environments throughout the project.
It always ends up like a mixed bag:
* Juggling production/prod, staging/stg, and develop/dev,
* Inconsistent placement of env, e.g. prod-myproject or myproject-stg,
* Skipping env name sometimes, e.g. myproject-bucket for dev S3 bucket but prod-myproject-bucket for prod (though it's okay to emit env name for user facing places like URL),
* Inconsistent resource sharing between envs, e.g. same S3 bucket for local and dev but different DB, or same Kubernetes cluster with different labels for dev/stg but different cluster without a label for prod.
These inconsistencies often result from quick decisions without much thought or out of necessities,
and everyone is too scared to fix them anyway.
But it bothers me a lot and sometimes causes serious bugs in production.
Env vars over-share and files depend on local permissions. We should have a capabilities -like way to send secrets between processes. e.g., decrypt and expose on a Unix socket with a sha filename that can only be read from once, and then gets torn down. Share the file name, target can read it and immediately the secret is now at-rest encrypted.
Encryption based on config containing a whitelist of ssh public keys and what they can access, sort of like age.
On my phone so can’t double test, but can’t you get this by adding “export” in front of every line in your env file and then source before running command?
I suppose if you don’t want it to stay after execution i believe you can:
> $(source .env; my command)
I’m sure there is a fairly straightforward way to encrypt and decrypt a local file
Now take that a step further with Environment Modules and you can source them from any shell or scripting language: https://modules.readthedocs.io/ (without creating a subinterpreter)
dotenv has features that include conditional selection and ordered merging of env files, which are configurable by dotenv's runtime and buildtime APIs.
Importing a set of library and dependencies to handle reading a plain text file poses more risks than just leaving the file unencrypted.
You don't need to encrypt your keys, with what keys are you going to do so? Will you encrypt those?
if someone is in your server you are pwned anyways.
It's ok if you identify yourself as a cybersecurity dude and hold a cybersecurity role and you need to justify your livelihood.
But do it in a way where you don't bother people. It's ok if you bother devs, but then you go on and bother users with 4FA, 5 rule passwords, systems that can't answer subpoenas because you have encrypted your sense of self.
When you are improving security at the expense of every other variable, that's annoying, but when you keep "improving security" at the expense even of security, is the point where people will start ignoring and hiding shit from you
completely agree. convenience is terribly underlooked in security judging by the reactions in this thread.
if the "secure" methods aren't being used because of 4FA and 5 rule passwords and 30 day expiries (don't get me started on this), then it is by default insecure because the devs will find more convenient ways, and thereby, less secure ways.
it's like storing passwords, i can't tell u how many times i've seen people use the same passwords everywhere because the rules are too restrictive. or just write it down somewhere public because it's too much work to get into the password manager and properly add it
i'd be willing to put big money down that a LARGE chunk of passwords for apps that require at least 1 number or symbol largely end in `!` or `1` at the end.
luckily i do think passkey is a step in the right direction with good convenience and overall ux
We’ve been pushing for committing encrypted secrets for many years now, and have written an open source spec and implementation in multiple languages: https://github.com/neosmart/securestore-rs
I got so excited, but it doesn't seem to support multiple keys and seems overly eager to encourage people to leave a valuable key lying around on disk.
So if a single dev machine is compromised, all of your prod secrets are exposed?
I wish this were closer to sops with support for gpg and or ssh keys. Because sops is a great idea locked in a questionable codebase.
Thanks, but I would rather go with the imperfect setup that I understand than an allegedly perfect setup with dozens of third party dependencies that I don't understand.
Doubly the case now that env is natively supported by node now.
This is quite a common remark when it comes to Javascript. I rarely see the same being made about Rust libs, which also rely heavily on external dependencies.
Something I have done for secrets is use a syntax in environment variables to tell the process to go a key vault for the secret.
So we can have
FOOPW=pw1
when testing locally, but
FOOPW="{vault1:secret1}"
in production. Env vars are processed simply by running a regex with callback that fetches secrets from vaults. This is quite flexible and has the advantage of being able to inject the secrets in the same place as other configuration, without actually having the secrets in environment variables or git etc (even encrypted)
Hah! I've done something similar. In my case, I could have something like this in my .env file:
SOME_CONFIG_OPTION = @AWS::some_config_option
And I've written a config library that knows that when a config value starts with `@AWS::' it needs to resolve the config option to an actual value by reaching out to AWS's Secrets Manager service and looking it up there, in which case it receives the value and caches it locally so that subsequent references to this configuration option don't require an additional call out to the cloud.
Even without backwards compatibility, why not just a breaking change in the same project? Don't get it either.
I've never used it (knowingly) but if I did and wanted to use this new version/project even the CLI name change to append 'x' would be annoying (I'd probably alias /symlink it).
It was added in a backwards compatible way, but the author decided to make a breaking change with this release.
The previous (IMHO superior) version was generating a .env.vault and a .env.keys from a .env file. Leaving the .env plain text and .env.vault encrypted.
Yeah, this is a really weird post for me. Before dotenv existed I used Foreman. It worked basically exactly how "dotenvx cross-platform" works (as a global command line). I switched to node-foreman because I largely switched to working on node projects and didn't want devs to need to have ruby installed. Then, at some point I switched to dotenv, and I don't even remember why.
A lot of people here are saying that a vault is preferable to this, especially for secrets.
Is there a good primer on using vaults? I know how to query and insert into Azure Key Vaults, but architecting around it is unclear to me.
Things that come up for me:
- As (azure) key vaults don't support per secret access rights, where do I store secrets between deployments?
- Should I store connection strings to cloud resources, or just ask the resource for the connection string at deployment time (for Azure, a cloud function pretty much needs a connection string for most basic things. They say they are moving away from this but ...)
- A security warning is send if a key is accessed more then x/times per hour. Does that mean I should pull in the key from vault at deployment? Cache it after first call during runtime?
- Most of our 3rd party vendors gives us 1 and only 1 key. How do I manage that key between development, production and several developers? Right now we mostly forward the e-mail from the vendor with the key ...
I'm not comfortable that this tool is able to make HTTP requests, IMO it ought to be completely offline and not perform network operations. I did a search for requests and found some results pushing to and pulling from a 'hub'. But I couldn't figure out what the hub refers to.
I think for some reason people don't like such simple tools. I think it seems more reasonable to deploy a vault service running on kubernetes with jenkins. In some comments, they didn't even understand what the tool was for. Dear dude, you can travel between dimensions with this tool.
Secrets in env vars in production is not too secure either, ideally you’ll move to your app pulling secrets in-process from your infrastructure at boot-up or upon use. This also gives a nice advantage of not needing to rebuild the app or container or whatever to rotate a secret.
This just moves the problem to a different step. How are you going to manage access to said secrets, especially when your application lives off premises?
This seems to encourage committing encrypted secrets, which is a bad idea. Configuration and code should be in separate repositories. Secrets should be protected elsewhere.
The correct fix for “it’s too easy to accidentally commit .env files with secrets” is to not function (panic/throw) if there isn’t a suitable .gitignore/.dockerignore, not a specialized cryptosystem for .env files. This just creates a different problem.
I simply use an envdir outside of the project and update all my run scripts to use “envdir $CONFIG_PATH <whatever>”. Simpler and safer.
I don't think this is the best approach. I am building https://phase.dev which lets you import secrets (.env), encrypt secrets (end-to-end encrypted with keys you control), sync them to other services/platforms (think AWS, GitHub, Kubernetes), and inject them into applications at runtime (e.g., phase run node index.js).
Since node v20.06, has built in support for --env-file=.env on load... as for local(ish) encryption and pushing them into source control, I don't like this at all. I'm fine using a vault or secret distribution from either the environment host (k8s) or ci/cd deployment.
I do like to keep a .env.example that you can rename to .env and adjust as desired. I tend to have defaults for running a compose stack locally that close to "just works" as possible.
My beef with it is that it's written in Javascript whereas what it does is so simple that they should just code it up in Rust or Go and be done with it, particularly when it could be packed up as a tiny binary that doesn't require you to install Node, Python, the JVM or any other runtime or library.
Wait, people use the .env file for secrets?! I never even looked into this but just assumed this was a convenience for local dev to hold a local non-secret app config? In production you'd load the env from a secrets store like what kubernetes has or aws has for ec2 instances. You shouldn't ever have those secrets on your local machine... Because they are secret.
We implemented the exact same method for config encryption a year ago or so, using pub/private key auth and the same `encrypted:` prefixes for encrypted config values.
This is a great tradeoff: easy way to share configuration, easy way to edit non-encrypted config values, reasonable security for the private values.
Doesn't solve key rotation of course, but for small teams this is a great solution.
I've recently taken a simpler approach to handling secrets in .env files. Since I use autoenv and conda venvs for everything, I persist secrets outside all projects in appropriately-named files, so including them in a .env becomes similar to `source $HOME/.secrets/work__aws_access`. Also makes them easier to manage across projects.
So with this, I have to prefix every executable with dotenvx in order to utilize the env variables?
Can’t I somehow do this in the script itself so “ruby index.rb” is enough? I know I’m only saving a couple of characters in the command line but I’m asking out of curiosity.
Can someone help enlightening me. In terms of preventing leaks, since we still need to ensure that the .env.keys doesn't leak, wouldn't effort still be the same?
The ability to use arbitrary filename for.env is quite nice though!
The idea is that the private key is stored outside of your configuration directory (e.g. in your OS's keychain), so your dotenv file can contain encrypted key values without leaking the value itself.
How would this work with application-specific Intellij Run/Debug configurations? Would this require switching to a "Shell Script" configuration, thereby losing debugging capability etc?
Based on my skim, this doesn’t really solve anything? The private key to decrypt is still stored on machine in a similar place? All this is doing is obfuscating?
Wait, it's also doing command interpolation? I missed that. Where is the code for that? I thought it uses dotenv-expand for interpolation, which can't do commands. dotenv-expand is already a bit meh, because of how it resolves the variables after parsing the dotenv file. Meaning you can't have a verbatim single quoted string and also you can get a stack overflow doing this:
A=$B
B=$A
Anyway, I hope they don't do command interpolation on top of that (like Ruby dotenv does), because then you can inject code via environment variables (like in the Ruby version).
I recently looked into various dotenv implementations just for fun. They're all different. No unified syntax at all. A lot don't do proper parsing either, but just use some regular expressions (like this one), which means they just skip over what doesn't matches. I started to document all the quirks I could find and wrote my own dotenv dialect just for fun. Nobody use it! Anyway, here it is: https://github.com/panzi/punktum
The whole idea of using environment variables for configuration information is good, but ultimately flawed, and we are way past the point where this should continue to be the status quo.
Environment variables are great for configuration because:
- you can inherit them from a previous application or application(s)
- you can override them in each environment you run your app in
- you can pass them on to other applications
- they are globals that can be loaded by libraries
- they're not hardcoded in the code, so easier to change things without rebuilding, easier to reuse in different ways/environments/configurations
- the OS has primitives for them
- they're simple
Environment variables are bad for configuration:
- because (by default) when set in application, they are passed on to all future applications/forks/execs
- they are often dumped as part of troubleshooting and aren't considered confidential
- they can often be viewed by external processes/users
- there are restrictions on key names and values and size depending on the platform
- typical "dotenv" solution doesn't necessarily handle things like multi-line strings, has no formal specification
- no types, schemas
What we actually need that environment variables are being used for:
- configuration information passed at execution time that can change per environment
- loading or passing secret values
- development environments
- production environments
So what would be a good alternative?
- an application library ("libconfig") that can load configuration of various types from various sources in various ways
- support for configuration types: key-value, file/blob, integer/float
- support for confidentiality (require specific function to unseal secret values; in programming languages the intent would be you can't just print a stringified version of the variable without an unseal function)
- support for schema (application defines schema, throws exception if value does not match)
- support allowing a configuration to be overloaded by different sources/hierarchies
- support passing a configuration on to other applications
- support tracing, verbose logging
- truly cross-platform and cross-language with one specification, behavior for all
How would it work?
- devs can create a .env file if they want
- devs load 'libconfig' into app, use it to load their configuration values during development. library can have default sources, and even set env vars or an object internally, so no code needs to be written to use it
- in production, same code causes libconfig to look at cloud-native and other sources for configuration
- when debugging, secret confidentiality is maintained, tracing communicates sources of configuration, what was loaded, from where, etc
Super early days but we're building something very similar to what you are describing over at https://dmno.dev
Granted our solution is more javascript/typescript focused - and the config schema will be defined using TypeScript and piggyback on npm for sharing plugins/extensions. But the config will be usable in any language (and generate types for multiple languages) with deeper integrations coming soon.
The pluggable nature of our architecture also means you can encrypt secrets in your repo if you want to, or sync with other backends. Shouldn't be too hard to keep everything away from env vars either if that's what some folks want.
Would love your input, and to hear what you think!
This is the only goal and this tool archives it. In the simplest way. While keeping you as secure as you were before, manually setting envs on heroku, railway, aws, jenkins etc.
Can you do safer? Yes, yes you can with a secrets management service (e.g.: Hasicorp Vault). Is that way more complicated to setup in all envs? Oh yes, yes it is.
I _detest_ this kind of encryption. It's literally worse than useless. It makes life much harder during debugging, and it eventually leads to developers just storing the decryption keys locally.
For this kind of encryption to work, you need to supply the decryption key from some outside system (e.g. via env vars, AWS SSM, etc.). And if it can supply the key, then why not just use it for other important secrets directly?
It's also problematic from a secrets management perspective, because a big part of the perceived value of encryption is being able to check secrets into git. But because the encryption is tied to long-term keys, you have to design your security processes with the assumption that those keys will eventually be exposed and need to be "revoked" (ie: the secrets re-encrypted), and the "de-revocation" of those keys is hiding in your git history.
it might be easier to just store and checkout a single decryption key that only devops people know, vs storing hundreds of secrets.
while developers can move around their .env file across systems without worrying that they left plaintext secrets somewhere.
also it allows adding new secrets without knowing decryption key - I think it is important for collaboration
also most importantly: plaintext decrypted secrets are never stored on disk, and only kept in memory. I think it is also an improvement towards the regular doting
I don't get it. Dotenv is only good for local dev. Otherwise you should put your secrets in environment variables (the "env" in ".env"). That people put .env files in prod is a mistake itself, and the proposed fixes here seem to not really do much about that.
You can commit your secrets as an encrypted vault with this. Then decrypt it with a key where needed: locally, on CI, on prod, etc.
This is basically a simplified version of Hashicorp's Vault, GCP key vault etc. with some less granularity on user authentication.
It solves the issues around .env.example and is perfect for gitops. You have all your secrets for all your envs ready, while you only need to set a single env var (the private encrpytion key) on your specific hosting environment.
You could even use separate keys per env, eg.: to give access to a developer to staging only.
Mozilla's https://github.com/getsops/sops is another contender but with a more complicated (and perhaps more flexible) key management.
Secrets don’t belong in environment variables either. Place them in a vault and grant specific processes/identities permission to read and decrypt them.
Env vars are prone to leaking and best practice moves the goal post further. Devs love to dump envs to log files, child processes inherit them, admins can very easily sniff them.
Dotenv is an okay hack, but the root problem is that the operating systems we run these applications in don't have a fully conceived environment variable system.
I think you have a fair point that dotenvx doesn't get the implementation right, but it does at least seem to recognize where the problem lies and is trying to fix it from that angle. You have to start somewhere. Almost never do we get solutions right the first time. It takes iteration and experimentation and perhaps this (and others like it) can pave the way towards a better solution.
Just set those env vars in your IDE. Let your IDE or docker-compose or whatever read an .env file if you must. But don't do it directly from your application code, indeed you're one lazy dev away from putting an .env file on prod servers.
Using dotenv-like constructions is, in my eyes, an antipattern.
Seems odd to announce new features without calling out to the fact that libraries with the same name in other languages have had those same features for years. (for example, the dotenv gem in Ruby)
cimnine|1 year ago
[1] https://blog.diogomonica.com/2017/03/27/why-you-shouldnt-use... [2] https://security.stackexchange.com/questions/197784/is-it-un... [3] https://learn.microsoft.com/en-us/aspnet/core/security/app-s...
skissane|1 year ago
I’ve never liked making secrets available on the filesystem. Lots of security vulnerabilities have turned up over the years that let an attacker read an arbitrary file. If retrieving secrets is a completely different API from normal file IO (e.g. inject a Unix domain socket into each container, and the software running on that container sends a request to that socket to get secrets), that is much less likely to happen.
jillesvangurp|1 year ago
Building a lot of assumptions into your containers about where and how they are being deployed kind of defeats the point of using containers. You should inject configuration, including secrets, from the outside. The right time to access secret stores is just before you start the container as part of the deploy process or vm startup in cloud environments. And then you use environment variables to pass the information onto the container.
Of course that does make some assumptions about the environment where you run your containers not being compromised. But then if that assumption breaks you are in big trouble anyway.
Of course this tool is designed for developer machines and for that it seems useful. But I hope to never find this in a production environment.
darkwater|1 year ago
bhawks|1 year ago
Environment vars propagate from process to process _by design_ and generally last the entire process(es) lifetime. They are observable from many os tools unless you've hardened your config and they will appear in core files etc. Secrets imply scope and lifetime - so env variables feel very at odds. Alternatively Env variables are nearly perfect for config for the same reasons that they are concerning for secrets.
Tl/Dr; in low stakes environments the fact that secrets are a special type of config means you will see it being used with env vars which are great for most configs but are poor for secrets. And frankly if you can stomach the risks, it is not that bad.
Storing secrets on the filesystem - you immediately need to answer where on the filesystem and how to restrict access (and are your rules being followed). Is your media encrypted at rest? Do you have se Linux configured? Are you sure the secrets are cleaned after you no longer need them? Retrieving secrets or elevated perms via sockets / local ipc have very similar problems (but perhaps at this point your compartmentalizing all the secrets into a centralized, local point).
A step beyond this are secrets that are locked behind cloud key management APIs or systems like spiffe/spire. At this point you still have to tackle workload identity, which is also a very nuanced problem.
With secrets every solution has challenges and there the only clear answer is to have a threat model and design an architecture and appropriate mitigations that let you feel comfortable while acknowledging the cost, user, developer and operator experience balancing act.
neonate|1 year ago
unknown|1 year ago
[deleted]
sandstrom|1 year ago
https://mise.jdx.dev/
It handles task running (wipe local test db, run linting scripts, etc), environment variables and 'virtual environments', as well as replacing stuff like asdf, nvm, pyenv and rbenv.
Still somewhat early days, tasks are experimental. But looks very promising and the stuff I've tried to far (tasks) works really well.
ralgozino|1 year ago
horse666|1 year ago
I’ve switched recently from asdf for managing language & tool versions and the ergonomics are much nicer (eg one command vs having to manually install plugins, etc., more logical commands) It’s also noticeably faster.
Regarding the env vars features, a couple of relevant Mise issues around people trying to integrate env var secrets using SOPS, 1Password, etc.
- https://github.com/jdx/mise/issues/1617
- https://github.com/jdx/mise/issues/1359
maleldil|1 year ago
I haven't felt the need to use it as a task runner yet, but that's probably because I'm used to having a bunch of shell and Python scripts in a `scripts` folder.
mystickphoenix|1 year ago
renewiltord|1 year ago
treflop|1 year ago
If now someone has to read docs to figure out how to configure the app, I’d rather have them read docs for some other safer and more powerful configuration scheme.
pnt12|1 year ago
As an example, once I changed a .env fikr and unit tests started failing. Digging deeper into it, and lots of code was checking for .env to load its configurations, and would break without it. I'd prefer this not to happen, as our tests were executing based on outside, not version controlled, configurations.
After removing dotenv as a library and using it only as a command, we were able to separate configuration and logic, and not have .env files affecting our unit tests - we simply ran the application with dotenv command, and the unit tests without.
michaelmior|1 year ago
bluelightning2k|1 year ago
akvadrako|1 year ago
atkailash|1 year ago
[deleted]
mplanchard|1 year ago
Sops also integrates easily with AWS and other existing key management solutions, so that you can use your existing IAM controls on keys.
I mentioned in another comment, but I've been using it over five years at two jobs and have found it to be great.
[0]: https://github.com/getsops/sops
pmccarren|1 year ago
I'd prefer an integration between dotevnx and sops where dotevnx handles the UX of public env and injection, while leveraging sops for secret management and retrieval. Additionally, being able to have multiple keys for different actors is important.
Having a single `.env.keys` file feels risky and error prone. dotenvx encourages adding your various env files, such as `.env.production`, to vcs, and you're one simple mistake away from committing your keyfile and having a bad day.
If sops is not to be integrated, dotenvx could take some inspiration where the main key is encrypted in the secrets file itself, and you can define multiple age key recipients, each of which can then decrypt the main key.
hadlock|1 year ago
golergka|1 year ago
brigadier132|1 year ago
One reason I can think of is that normally with secrets I actually don't keep any copies of them. I just set them in whatever secret manager my cloud environment uses and never touch them again unless I need to rotate them. Meaning there is no way to accidentally expose them other than by the secret vault being hacked or my environment being hacked.
With this approach if someone gets access to the encryption key all secrets are exposed.
solatic|1 year ago
Atotalnoob|1 year ago
Typically, developers can’t change production secrets in vaults and need to follow some other protocols.
Encrypted secrets mean you deploy everything along side the secrets.
The developer experience is great, but the biggest issues I have faced while using Kubeseal were
1. Developers HAVE the secret in order to encrypt it. This can be not ideal as then they can use these secrets in production or leak them
2. The secret encryption key change causes the need to re encrypt everything.
3. People don’t understand the concept.
jrockway|1 year ago
git-crypt is easy, the master key doesn't rotate, so don't leak it. (Secret encryption key rotation is kind useless; it's nice that if you leak an old key newer secrets aren't leaked, but it depends on your underlying secret rotation policy as to whether or not that saves you any work. I have tended to do them in bulk in the past.)
On my last project we did disaster recovery exercises every 6 months, sometimes with the master key intentionally lost, and it wasn't that big of a deal. Restoring your infra-as-code involves creating the secret encryption service manually, though, which is kind of a pain, but not like days of downtime pain or anything. Of course, if the secrets encrypted your database or something like that, then losing the master key equals losing all your data. Hopefully your database backup has some solution for that problem.
rrgok|1 year ago
If the vault is password protected, aren't you just adding one more indirection and nothing more? How is that helpful, since now I have to write the vault password in clear-text somewhere such that my application can read the env file from the vault?
c0brac0bra|1 year ago
At no point does the application have access to the vault itself, and access to read the vault is guarded by IAM role permissions.
nunez|1 year ago
This is mildly complicated, but the alternative is storing config in a configuration server somewhere, which comes with its own can of worms.
hamasho|1 year ago
On an unrelated note, I always find it a real headache to keep the naming convention of the environments throughout the project. It always ends up like a mixed bag:
These inconsistencies often result from quick decisions without much thought or out of necessities, and everyone is too scared to fix them anyway. But it bothers me a lot and sometimes causes serious bugs in production.Fix: format
nunez|1 year ago
clord|1 year ago
bhawks|1 year ago
Also any system as described needs security audit and analysis to truly understand it strengths and weaknesses (or glaring compromises).
Alternatively - secrets via environment vars weaknesses and mitigations are well understood.
wodenokoto|1 year ago
I suppose if you don’t want it to stay after execution i believe you can:
I’m sure there is a fairly straightforward way to encrypt and decrypt a local filegkfasdfasdf|1 year ago
newzisforsukas|1 year ago
bogdan|1 year ago
eternauta3k|1 year ago
ruined|1 year ago
TZubiri|1 year ago
You don't need to encrypt your keys, with what keys are you going to do so? Will you encrypt those?
if someone is in your server you are pwned anyways.
It's ok if you identify yourself as a cybersecurity dude and hold a cybersecurity role and you need to justify your livelihood.
But do it in a way where you don't bother people. It's ok if you bother devs, but then you go on and bother users with 4FA, 5 rule passwords, systems that can't answer subpoenas because you have encrypted your sense of self.
When you are improving security at the expense of every other variable, that's annoying, but when you keep "improving security" at the expense even of security, is the point where people will start ignoring and hiding shit from you
stephenhuh|1 year ago
if the "secure" methods aren't being used because of 4FA and 5 rule passwords and 30 day expiries (don't get me started on this), then it is by default insecure because the devs will find more convenient ways, and thereby, less secure ways.
it's like storing passwords, i can't tell u how many times i've seen people use the same passwords everywhere because the rules are too restrictive. or just write it down somewhere public because it's too much work to get into the password manager and properly add it
i'd be willing to put big money down that a LARGE chunk of passwords for apps that require at least 1 number or symbol largely end in `!` or `1` at the end.
luckily i do think passkey is a step in the right direction with good convenience and overall ux
NegativeK|1 year ago
This is false and also a symptom of an all-or-nothing approach to cybersecurity, which isn't feasible in the real world.
ComputerGuru|1 year ago
curiousdeadcat|1 year ago
So if a single dev machine is compromised, all of your prod secrets are exposed?
I wish this were closer to sops with support for gpg and or ssh keys. Because sops is a great idea locked in a questionable codebase.
mplanchard|1 year ago
I've used it at two jobs now over about 5 years and have had zero issues.
jahewson|1 year ago
And where do I keep the key? In a secret store?
frithsun|1 year ago
Doubly the case now that env is natively supported by node now.
stiiv|1 year ago
unknown|1 year ago
[deleted]
iamsaitam|1 year ago
caliwagon|1 year ago
boundlessdreamz|1 year ago
In Rails, the entire file is encrypted unlike here where only the secrets are
vinnymac|1 year ago
kinduff|1 year ago
kevlened|1 year ago
dagss|1 year ago
So we can have
FOOPW=pw1
when testing locally, but
FOOPW="{vault1:secret1}"
in production. Env vars are processed simply by running a regex with callback that fetches secrets from vaults. This is quite flexible and has the advantage of being able to inject the secrets in the same place as other configuration, without actually having the secrets in environment variables or git etc (even encrypted)
pak9rabid|1 year ago
SOME_CONFIG_OPTION = @AWS::some_config_option
And I've written a config library that knows that when a config value starts with `@AWS::' it needs to resolve the config option to an actual value by reaching out to AWS's Secrets Manager service and looking it up there, in which case it receives the value and caches it locally so that subsequent references to this configuration option don't require an additional call out to the cloud.
It works surprisingly well.
ashenke|1 year ago
The URL in `dotenvx` points to https://gitub.com/dotenvx/dotenvx (gitub without the h)
ssahoo|1 year ago
hermanradtke|1 year ago
bluelightning2k|1 year ago
It would only break in cases where people's values specifically started with "encrypted:"
OJFord|1 year ago
I've never used it (knowingly) but if I did and wanted to use this new version/project even the CLI name change to append 'x' would be annoying (I'd probably alias /symlink it).
daralthus|1 year ago
The previous (IMHO superior) version was generating a .env.vault and a .env.keys from a .env file. Leaving the .env plain text and .env.vault encrypted.
triyambakam|1 year ago
VWWHFSfQ|1 year ago
[0] https://github.com/ddollar/foreman
edward28|1 year ago
[0] https://theforeman.org
itslennysfault|1 year ago
drchickensalad|1 year ago
wodenokoto|1 year ago
Is there a good primer on using vaults? I know how to query and insert into Azure Key Vaults, but architecting around it is unclear to me.
Things that come up for me:
- As (azure) key vaults don't support per secret access rights, where do I store secrets between deployments?
- Should I store connection strings to cloud resources, or just ask the resource for the connection string at deployment time (for Azure, a cloud function pretty much needs a connection string for most basic things. They say they are moving away from this but ...)
- A security warning is send if a key is accessed more then x/times per hour. Does that mean I should pull in the key from vault at deployment? Cache it after first call during runtime?
- Most of our 3rd party vendors gives us 1 and only 1 key. How do I manage that key between development, production and several developers? Right now we mostly forward the e-mail from the vendor with the key ...
politelemon|1 year ago
pictur|1 year ago
jitl|1 year ago
Etheryte|1 year ago
daralthus|1 year ago
`start: dotenvx run -f .env.local -f .env -- node index.js`
Instead of the -f flag, which now cannot be overriden, one could invoke it with
`DOTENV=.env.staging npm run start`
vinnymac|1 year ago
For example
DOTENV_PRIVATE_KEY_PRODUCTION
Would provide it with the information it needs to read .env.production
fiddlerwoaroof|1 year ago
sneak|1 year ago
The correct fix for “it’s too easy to accidentally commit .env files with secrets” is to not function (panic/throw) if there isn’t a suitable .gitignore/.dockerignore, not a specialized cryptosystem for .env files. This just creates a different problem.
I simply use an envdir outside of the project and update all my run scripts to use “envdir $CONFIG_PATH <whatever>”. Simpler and safer.
bradgessler|1 year ago
nimishk|1 year ago
Source: https://github.com/phasehq/console
poopsmithe|1 year ago
tracker1|1 year ago
I do like to keep a .env.example that you can rename to .env and adjust as desired. I tend to have defaults for running a compose stack locally that close to "just works" as possible.
I doubt I'd ever want to use this in practice.
PaulHoule|1 year ago
globular-toast|1 year ago
the_duke|1 year ago
This is a great tradeoff: easy way to share configuration, easy way to edit non-encrypted config values, reasonable security for the private values.
Doesn't solve key rotation of course, but for small teams this is a great solution.
skeledrew|1 year ago
Alifatisk|1 year ago
Can’t I somehow do this in the script itself so “ruby index.rb” is enough? I know I’m only saving a couple of characters in the command line but I’m asking out of curiosity.
difu_disciple|1 year ago
Does dotenvx support secrets managers?
blacksoil|1 year ago
The ability to use arbitrary filename for.env is quite nice though!
nightpool|1 year ago
kodeninja|1 year ago
sampli|1 year ago
ptdorf|1 year ago
tamimio|1 year ago
And the attackers will be after this file not the .env anymore.
It looks great nonetheless, especially the cross-language feature.
hooverd|1 year ago
panzi|1 year ago
I recently looked into various dotenv implementations just for fun. They're all different. No unified syntax at all. A lot don't do proper parsing either, but just use some regular expressions (like this one), which means they just skip over what doesn't matches. I started to document all the quirks I could find and wrote my own dotenv dialect just for fun. Nobody use it! Anyway, here it is: https://github.com/panzi/punktum
Direct link to the quirks of the JavaScript dotenv implementation: https://github.com/panzi/punktum?tab=readme-ov-file#javascri...
I've also tried to write a parser compatible to JavaScript dotenv (no x) in C++: https://github.com/panzi/cpp-dotenv
0xbadcafebee|1 year ago
Environment variables are great for configuration because:
Environment variables are bad for configuration: What we actually need that environment variables are being used for: So what would be a good alternative? How would it work?theozero|1 year ago
Granted our solution is more javascript/typescript focused - and the config schema will be defined using TypeScript and piggyback on npm for sharing plugins/extensions. But the config will be usable in any language (and generate types for multiple languages) with deeper integrations coming soon.
The pluggable nature of our architecture also means you can encrypt secrets in your repo if you want to, or sync with other backends. Shouldn't be too hard to keep everything away from env vars either if that's what some folks want.
Would love your input, and to hear what you think!
daralthus|1 year ago
This is the only goal and this tool archives it. In the simplest way. While keeping you as secure as you were before, manually setting envs on heroku, railway, aws, jenkins etc.
GitOps FTW
daralthus|1 year ago
swedonym|1 year ago
Aeolun|1 year ago
rednafi|1 year ago
nunez|1 year ago
I wonder if dotenvx ensures that .env is in .gitignore and yells loudly if it is not.
I encrypt my dotenvs with gpg, but that's hella esoteric and everyone shouldn't be forced to do that.
daralthus|1 year ago
yashbindal|1 year ago
jbverschoor|1 year ago
cyberax|1 year ago
For this kind of encryption to work, you need to supply the decryption key from some outside system (e.g. via env vars, AWS SSM, etc.). And if it can supply the key, then why not just use it for other important secrets directly?
tptacek|1 year ago
slt2021|1 year ago
while developers can move around their .env file across systems without worrying that they left plaintext secrets somewhere.
also it allows adding new secrets without knowing decryption key - I think it is important for collaboration
also most importantly: plaintext decrypted secrets are never stored on disk, and only kept in memory. I think it is also an improvement towards the regular doting
gregwebs|1 year ago
unknown|1 year ago
[deleted]
boxed|1 year ago
daralthus|1 year ago
This is basically a simplified version of Hashicorp's Vault, GCP key vault etc. with some less granularity on user authentication.
It solves the issues around .env.example and is perfect for gitops. You have all your secrets for all your envs ready, while you only need to set a single env var (the private encrpytion key) on your specific hosting environment.
You could even use separate keys per env, eg.: to give access to a developer to staging only.
Mozilla's https://github.com/getsops/sops is another contender but with a more complicated (and perhaps more flexible) key management.
rhplus|1 year ago
Env vars are prone to leaking and best practice moves the goal post further. Devs love to dump envs to log files, child processes inherit them, admins can very easily sniff them.
randomdata|1 year ago
I think you have a fair point that dotenvx doesn't get the implementation right, but it does at least seem to recognize where the problem lies and is trying to fix it from that angle. You have to start somewhere. Almost never do we get solutions right the first time. It takes iteration and experimentation and perhaps this (and others like it) can pave the way towards a better solution.
globular-toast|1 year ago
isoprophlex|1 year ago
Using dotenv-like constructions is, in my eyes, an antipattern.
pluc|1 year ago
l5870uoo9y|1 year ago
It wouldn't surprise me if many VPS use .env files.
unknown|1 year ago
[deleted]
bdcravens|1 year ago
smartplaya2001|1 year ago
https://nodejs.org/en/blog/release/v20.6.0#built-in-env-file...
unknown|1 year ago
[deleted]
languagehacker|1 year ago
[deleted]
throwaway240403|1 year ago
dymk|1 year ago
unknown|1 year ago
[deleted]
atulvi|1 year ago
[deleted]
usixk|1 year ago
[deleted]
jppope|1 year ago
[deleted]
unknown|1 year ago
[deleted]
Aerbil313|1 year ago
[deleted]