(Warning: I am being Dennis Downer) Meanwhile, we're still waiting on WebSockets and it's been nearly 10 years.
Everyone should read the PR https://github.com/nodejs/node/pull/48890. It doesn't traverse parent directories, it doesn't have good overwrite/merge logic, multiline support, or variable expansion - things that dotenv has supported for years (sometimes with add-ons). This is another half-useful implementation that the majority will end up ignoring, and keep using better-suited third party packages for.
Node is committee'd to death, and the comments in that PR are further proof. I've been in the Node ecosystem since 2010, and my money is on Bun and Deno to lead the way forward.
So there is chance it won't be massive security hole enabled by default! Great!
> it doesn't have good overwrite/merge logic, multiline support, or variable expansion - things that dotenv has supported for years (sometimes with add-ons)
99% of what I used env files for was just a bunch of key-values for db passwords so their minimal implementation is probably good enough for most. Then again, that's experience from other than node ecosystems, and if config requires "multiline support, or variable expansion" we either make proper config file or a template for CM to deploy.
Fun fact - I added a pretty feature complete WS connection in Node core (https://github.com/nodejs/node/blob/main/src/inspector_socke...) 7 years ago (it is well-tested - every debug/profiling session goes through it). There's no public API for it because I did not want to deal with politics.
There's rudimentary HTTP server as well (no TLS and such though). All in C++. Should be fairly fast, based on expertise of the Chromium team.
I think it is fine for node to have limited, basic, built in support for .env files, and then leave the opportunity for the ecosystem to add in more features.
Easy things should be easy, and hard things should be possible through npm install.
I'm the CTO of a popular Secrets Management platform. It's fair to say that I personally have a lot of experience with secrets and requirements around them, based on conversations with customers.
The primary missing feature here seems to be multiline support. That's super common for keys, certificates, and other JSON configuration. Based on the Node PR, they appear open to adding that later (big +1 on shipping incrementally).
The other missing feature that folks tend to heavily rely on is variable expansion. For that to truly work well, I recommend using a holistic platform like Doppler. That allows for expansion/referencing across environments and projects, like when you have multiple independent services that need access to the same set of secrets (e.g. database creds, error reporting tool, stripe key, etc). You can then update the secret once and have the change propagate to all the places it's used.
Lastly, I'd be remiss if I didn't mention the Doppler CLI and our own fairly unique support for .env files. We've traditionally taken a dim view of .env files because they represent a static, long-lived collection of sensitive information that lives offline. Often, these get checked into a git repo. This is probably fine for personal projects, but a major issue for companies and the security aware. However, .env files are a pseudo standard and folks want a way of continuing to consume their secrets via them. Our CLI's approach is to mount a named pipe that we can write secrets to when a reader attaches. That allows us to limit the amount of times the "file" can be read (e.g. once), it guarantees that the file's contents are unavailable once the application process dies, and it uses the same open/read interface as a standard file.
In all, this is an exciting development for Node. I'm glad to see more standard features make it into core and hope that multiline support is a fast follow.
I agree. Since you seem to know the area, do you have any comment on which contender (Bun or Deno) you see as most likely to take the crown, and also which you’d prefer to take the crown? By ‘the crown’ I mean the status of being the leading JS runtime outside of web browsers.
For me, Deno seems very well thought out, I’ve used it a fair bit and found it to work reliably and fast. I like the whitelist approach to system access although I worry that it could be a hurdle for adoption. I haven’t used Bun, just read about it. My sense is that Bun is more of a 1:1 replacement for Node, still tied to npm as a first class citizen, while Deno has a long term strategy to get away from npm (while supporting it as a legacy thing). Overall I’d prefer Deno to win, but I wonder if Bun has a better chance due to its closer parity with Node.
.env desperately needs a formal specification. I wrote a bit of code using some features python_dotenv offers but realized after the fact that other software might not parse it the same way. I’ve taken to instead doing my templating in Python itself via pydantic BaseSettings
Kinda surprising it took this long. I really hope Deno continues to grow in popularity. I greatly prefer the "batteries-included" and Typescript-first approach.
Why would they add built-in support when there's already a reliable well-supported package? Feels like bloatware and waste of resources. I'm sure there are lots of core functionalities that could be improved and optimised instead
There is a clear trend to absorb popular packages into the core and I believe the main reason is to reduce the number of dependencies. Deno, for example, goes on this direction. Given the recent security issues with npm packages, I think it's a good idea to reduce the number of dependencies.
A well-supported package, plus several tools that you can learn once and use the same for any language or combo of languages. And those will still work when you have to git-bisect back to a version of your project that doesn’t have built-in .env setting.
Shipping this specific thing with nodejs seems deeply pointless to me.
Plus, having my program auto-monkey with its own environment, even optionally, feels… wrong. I should define it (or something should on my behalf) and the program should accept what it gets. Else, why even use env vars? Just use a config toml or some shit. But that part reasonable people could disagree over.
Is it just me or does anyone else hate .env files? They accumulate unused cruft, it's not typed, and typically many systems in the same repo point at the same .env file. It all feels so ... implicit ... I guess.
For example in a code base I work in, docker-compose uses it, laravel picks up the same file and Vue / webpack does too. It's a big mess. There must be better solutions out there.
I personally prefer to stick to config files in whatever format is native to ecosystem (toml, yaml, ini). Env files fill the same role for apps which are configured by env variables, but you don't really need to use env variables there. It was forced by Docker some time ago as a single supported configuration format, but nowadays k8s happily accepts configs and mounts them as a file, so why jump the hoops with .env -> vars -> app?
I like them for defining a small number of env vars for my app that only get injected at run time. I don't like them when other random tools that aren't my app use them. Docker's insistence on automatically using the .env in your home directory is absurd.
The reason why we continue to use those is that they are easy to load in bash (source the file), can be hidden away in the env in production, don't have inheritance, and aren't yaml.
I feel you on the typing, but the format could accomodate such a tool. Code away! :)
Environment variables are all strings. That’s not .env files’ faults, it’s how env vars work.
If you don’t want/need to read env vars… why use .env? If you do need to, then you’re stuck treating them as string input, sure, but that’s got nothing to do with .env files. They just help set env vars. If you get rid of your .env file but are still reading from env vars, all you’ve done is tie one hand behind your back.
You can create a class or something that reads it in at application startup and throws up if something you expect isn't there. You can't really enforce a rule to not access `process.env` directly, but it at leasts gives you types to access the env should you choose to use the wrapper.
yeah I don't like environment variables because it's all stringly typed and the lack of a thing you deserialize the config file into means that practically speaking, you can read the values from anywhere in a program; figuring out what's permittable in environment variables is very often a disorganized mess. .env files take all of the messiness of environment variables and then ... put them into a file, which is typically the thing that people who are using environment variables are trying to avoid, so ... I dunno, just use a configuration file imho, .env files are the worst of both worlds.
When will React env variables work on docker images without any bullshit hacks? I mean when I have my frontend and backend in separate pods, use nginx to serve the frontend in kubernetes and want use BACKEND_URL as an env. Why is it so goddamn hard. Any high IQ people have tips on how to get this to work without some weird injections before building the image? Me dumb.
I'm honestly a bit confused by this statement, as "React env variables" can mean various things, depending on your setup.
Backend and frontend can be separate runtimes... The backend could be an API-only-server and an OS process with all permissions that come with it (reading env vars during execution as it pleases). The frontend, eg. a React SPA, could just be a bunch of built HTML/CSS/JS-files (type-checked, bundled, minified, etc.), served through a static file server (separate backend, nginx?) and interpreted through a browser engine on the client.
Are you asking for nginx to parse the served HTML/CSS/JS-files and replace unknown placeholders with env var values? If so you are prob asking for a SSR framework[0], meaning a backend that treats a React project similar to template files, and is able to inject env vars?
* most people are pre-building their packages. That means you can’t really modify anything at run time. It also means it can be extremely tedious to test changes and wait for the build process.
* there’s a string of places you have to properly “expose” ENV variables to a layer deeper down. This includes some build condos to make sure you’re not exposing sensitive BE configs. It can be challenging debugging the intermediate steps.
I'm not 100% sure the problem you're trying to solve, but I've solved a similar sounding one that may be of use to you too.
We needed to set env vars on React applications (the frontend) inside Docker containers. The containers were going to be built in CI/CD, and we didn't want different builds for each environment.
We solved this by adding a small JS script that is included in the base HTML. It pushes env vars into the window object, so they're accessible from window.env.
The JS file is generated by a Bash script that's ran whenever the container starts (by modifying the entry-point.sh file used by Nginx's Docker image).
The Bash script loads enumerates any environment variable with a given prefix.
Docker already allows you to set runtime environment variables from the run command, or via Docker Compose.
Im the end, the developer experience comes down to "docker run image:tag -e PREFIX_FOO=BAR".
Might be misunderstanding the problem, but what’s stopping you from using env variables like you specified? Create React App has this baked in by prefixing env variables with REACT_APP [1]; I’m assuming other configurations can achieve the same.
havent dug further in, wonder if it will support multiple files. we use two files, one for things that everyone should have in common and one for things that are different per individual and not committed
am I still the only one who thinks storing all of your secrets in plain text while using a package manager that allows arbitrary code execution is a terrible idea?
Nope, but if your dependencies are reasonable and don't do a bunch of stuff in pre/post-install you can disable it with `--ignore-scripts`.
IMO it would be much better if that was the default and dependencies had to ask to run scripts (perhaps with a whitelist in package.json), but unfortunately node did a bunch of "helpful" mistakes and now it's hard to roll back without breaking.
Maybe if they had denos permission model they could isolate so that the dependency could only read/write to it's own directory within node_modules.
What’s the motivation? I don’t want multiple things reading the file if I’m not working solely in nodejs—I want one tool reading it and setting env vars for all the rest—plus plenty of tools already exist to do this and most are trivial to install and use, and overall it seems like another way for things to go wrong and just a platform-specific Yet Another X burning person-hours to… what benefit?
But why though? .env (the library) has never made sense to me. There should be a singular binary called dotenv that then takes a command+args as arguments that then runs that command with the .env file's values in the environment. Why do runtimes need first-class support for this?
Environment variables should never be used. This was a known security hole already in the last century. Several OS solved this by config files that comes with file permissions. The correct solution on Linux and Windows at least. And still people use envs.
Why? Seems like it goes against the Unix principle. I've been using direnv to set environment for any software, containers, python virtualenvs and more, and it's working quite well. Will nodejs now conflict my direnv files?
[+] [-] andrew_|2 years ago|reply
Everyone should read the PR https://github.com/nodejs/node/pull/48890. It doesn't traverse parent directories, it doesn't have good overwrite/merge logic, multiline support, or variable expansion - things that dotenv has supported for years (sometimes with add-ons). This is another half-useful implementation that the majority will end up ignoring, and keep using better-suited third party packages for.
Node is committee'd to death, and the comments in that PR are further proof. I've been in the Node ecosystem since 2010, and my money is on Bun and Deno to lead the way forward.
[+] [-] ilyt|2 years ago|reply
So there is chance it won't be massive security hole enabled by default! Great!
> it doesn't have good overwrite/merge logic, multiline support, or variable expansion - things that dotenv has supported for years (sometimes with add-ons)
99% of what I used env files for was just a bunch of key-values for db passwords so their minimal implementation is probably good enough for most. Then again, that's experience from other than node ecosystems, and if config requires "multiline support, or variable expansion" we either make proper config file or a template for CM to deploy.
[+] [-] euos|2 years ago|reply
There's rudimentary HTTP server as well (no TLS and such though). All in C++. Should be fairly fast, based on expertise of the Chromium team.
[+] [-] madeofpalk|2 years ago|reply
Easy things should be easy, and hard things should be possible through npm install.
[+] [-] tompic823|2 years ago|reply
The primary missing feature here seems to be multiline support. That's super common for keys, certificates, and other JSON configuration. Based on the Node PR, they appear open to adding that later (big +1 on shipping incrementally).
The other missing feature that folks tend to heavily rely on is variable expansion. For that to truly work well, I recommend using a holistic platform like Doppler. That allows for expansion/referencing across environments and projects, like when you have multiple independent services that need access to the same set of secrets (e.g. database creds, error reporting tool, stripe key, etc). You can then update the secret once and have the change propagate to all the places it's used.
Lastly, I'd be remiss if I didn't mention the Doppler CLI and our own fairly unique support for .env files. We've traditionally taken a dim view of .env files because they represent a static, long-lived collection of sensitive information that lives offline. Often, these get checked into a git repo. This is probably fine for personal projects, but a major issue for companies and the security aware. However, .env files are a pseudo standard and folks want a way of continuing to consume their secrets via them. Our CLI's approach is to mount a named pipe that we can write secrets to when a reader attaches. That allows us to limit the amount of times the "file" can be read (e.g. once), it guarantees that the file's contents are unavailable once the application process dies, and it uses the same open/read interface as a standard file.
In all, this is an exciting development for Node. I'm glad to see more standard features make it into core and hope that multiline support is a fast follow.
[+] [-] cal85|2 years ago|reply
For me, Deno seems very well thought out, I’ve used it a fair bit and found it to work reliably and fast. I like the whitelist approach to system access although I worry that it could be a hurdle for adoption. I haven’t used Bun, just read about it. My sense is that Bun is more of a 1:1 replacement for Node, still tied to npm as a first class citizen, while Deno has a long term strategy to get away from npm (while supporting it as a legacy thing). Overall I’d prefer Deno to win, but I wonder if Bun has a better chance due to its closer parity with Node.
[+] [-] rat9988|2 years ago|reply
[+] [-] thiht|2 years ago|reply
Sounds good to me. If I need more one day I’ll use dotenv, but having basic support in the stdlib is great
[+] [-] gumballindie|2 years ago|reply
[+] [-] yxre|2 years ago|reply
[deleted]
[+] [-] lcfcjs|2 years ago|reply
[deleted]
[+] [-] SOLAR_FIELDS|2 years ago|reply
[+] [-] blitz_skull|2 years ago|reply
[+] [-] noman-land|2 years ago|reply
https://typestrong.org/ts-node/docs/
[+] [-] laurent123456|2 years ago|reply
[+] [-] dlojudice|2 years ago|reply
[+] [-] hotnfresh|2 years ago|reply
Shipping this specific thing with nodejs seems deeply pointless to me.
Plus, having my program auto-monkey with its own environment, even optionally, feels… wrong. I should define it (or something should on my behalf) and the program should accept what it gets. Else, why even use env vars? Just use a config toml or some shit. But that part reasonable people could disagree over.
[+] [-] rat9988|2 years ago|reply
[+] [-] _ZeD_|2 years ago|reply
[+] [-] jtwaleson|2 years ago|reply
For example in a code base I work in, docker-compose uses it, laravel picks up the same file and Vue / webpack does too. It's a big mess. There must be better solutions out there.
[+] [-] capableweb|2 years ago|reply
It kind of is, environment variables can only be strings, so why "type" it when you 100% know that everything within is 100% a String, always?
[+] [-] vilunov|2 years ago|reply
[+] [-] dubcanada|2 years ago|reply
I don't see any problem with 4 different things using it.
The point of it is basic, rather then VARIABLE=WHATEVER you just copy and paste a .env and it picks it up.
But you can also just use regular environment variables if you want.
[+] [-] moojd|2 years ago|reply
[+] [-] batmansmk|2 years ago|reply
I feel you on the typing, but the format could accomodate such a tool. Code away! :)
[+] [-] hotnfresh|2 years ago|reply
If you don’t want/need to read env vars… why use .env? If you do need to, then you’re stuck treating them as string input, sure, but that’s got nothing to do with .env files. They just help set env vars. If you get rid of your .env file but are still reading from env vars, all you’ve done is tie one hand behind your back.
[+] [-] s900mhz|2 years ago|reply
[+] [-] lghh|2 years ago|reply
[+] [-] zemo|2 years ago|reply
[+] [-] toastercat|2 years ago|reply
[1] https://www.npmjs.com/package/env-smart
[+] [-] user3939382|2 years ago|reply
[+] [-] Alifatisk|2 years ago|reply
Try cuelang.org
[+] [-] janmarsal|2 years ago|reply
[+] [-] aastronaut|2 years ago|reply
Backend and frontend can be separate runtimes... The backend could be an API-only-server and an OS process with all permissions that come with it (reading env vars during execution as it pleases). The frontend, eg. a React SPA, could just be a bunch of built HTML/CSS/JS-files (type-checked, bundled, minified, etc.), served through a static file server (separate backend, nginx?) and interpreted through a browser engine on the client.
Are you asking for nginx to parse the served HTML/CSS/JS-files and replace unknown placeholders with env var values? If so you are prob asking for a SSR framework[0], meaning a backend that treats a React project similar to template files, and is able to inject env vars?
[0]: Next.js - https://nextjs.org/docs/app/building-your-application/config...
[+] [-] SkyPuncher|2 years ago|reply
* most people are pre-building their packages. That means you can’t really modify anything at run time. It also means it can be extremely tedious to test changes and wait for the build process.
* there’s a string of places you have to properly “expose” ENV variables to a layer deeper down. This includes some build condos to make sure you’re not exposing sensitive BE configs. It can be challenging debugging the intermediate steps.
[+] [-] fckthisguy|2 years ago|reply
We needed to set env vars on React applications (the frontend) inside Docker containers. The containers were going to be built in CI/CD, and we didn't want different builds for each environment.
We solved this by adding a small JS script that is included in the base HTML. It pushes env vars into the window object, so they're accessible from window.env.
The JS file is generated by a Bash script that's ran whenever the container starts (by modifying the entry-point.sh file used by Nginx's Docker image).
The Bash script loads enumerates any environment variable with a given prefix.
Docker already allows you to set runtime environment variables from the run command, or via Docker Compose.
Im the end, the developer experience comes down to "docker run image:tag -e PREFIX_FOO=BAR".
[+] [-] CharanSriram|2 years ago|reply
[1] https://create-react-app.dev/docs/adding-custom-environment-...
[+] [-] miohtama|2 years ago|reply
Shdotenv helps to convert/reuse environment files across tools
https://github.com/ko1nksm/shdotenv
[+] [-] netcraft|2 years ago|reply
[+] [-] c0brac0bra|2 years ago|reply
[+] [-] aslilac|2 years ago|reply
[+] [-] SahAssar|2 years ago|reply
IMO it would be much better if that was the default and dependencies had to ask to run scripts (perhaps with a whitelist in package.json), but unfortunately node did a bunch of "helpful" mistakes and now it's hard to roll back without breaking.
Maybe if they had denos permission model they could isolate so that the dependency could only read/write to it's own directory within node_modules.
[+] [-] hotnfresh|2 years ago|reply
[+] [-] ilyt|2 years ago|reply
[+] [-] junon|2 years ago|reply
[+] [-] _pdp_|2 years ago|reply
[+] [-] AtNightWeCode|2 years ago|reply
[+] [-] INTPenis|2 years ago|reply
[+] [-] rbongers|2 years ago|reply
[+] [-] gjvc|2 years ago|reply