[dashwav]

I have a very hard time getting behind these complex configuration languages. To me what makes a configuration format good is the simplicity of reading the configuration of a program, and almost all of these languages are optimizing for feature complexity over readability. I think that all of the popular config formats (yaml, json, toml, etc) have issues, but none of the major issues with them have to do with being unable to represent a fibonacci sequence in their language.

To draw a direct comparison, when I look at the examples in the github repository, all I can think is "I would never want to have this be a source of truth in my codebase". While I get frustrated w/ whitespace in yaml and the difficulty of reading complex json configuration, if I need a way to programmatically load complex data I would almost always rather use those two as a base and write a 'config-loader' in the language that I am already using for my project (instead of introducing another syntax into the mix)

[q3k]

Conversely, I've been using quite a bit of Jsonnet in different projects for a few years now, and it's a life changer.

Here's a public example - using Jsonnet to parametrize all core resources of a bare metal Kubernetes cluster: [1]. This in turn uses cluster.libsonnet [2], which sets up Calico, Metallb, Rook, Nginx-Ingress-Controller, Cert-Manager, CoreDNS, ...

Note that this top-level file aims to be the _entire_ source of truth for that particular cluster. I know of people who are reusing some of the dependent lib/*libsonnet code in their own deployments, which shows that this is not just abstraction for the sake of abstraction.

Jsonnet isn't perfect, but it allows for actual building of abstraction layers in configuration, guaranteed pure evaluation, and not a single line of text templated or repeated YAML.

[1] - https://cs.hackerspace.pl/hscloud/-/blob/cluster/kube/k0.lib...

[2] - https://cs.hackerspace.pl/hscloud/-/blob/cluster/kube/cluste...

[AtlasBarfed]

Does a "configuration language" specifically incorporate features for "overlaid" or "unified from parts" configuration?

Much like layered dockerfiles, mature configuration often comes from several places: env vars, configuration appropriate for checkin to git (no secrets), secrets configuration, and of course the old environment-specific configuration.

All of that merges to "The Configuration".

Also, these seem close to templating languages.

I've done this several times with a "stacked map" implementation (much like the JSP key lookups went through page / session / application scopes, or even more convoluted for Spring Webflow.

[GordonS]

I also have a hard time for the same reasons. I'm torn though; I want a config format that's super easy to read, and that I can easily change anywhere with nothing more than sed/vim/nano/notepad - but I also want to avoid typos and formatting problems.

I'm not sure which is the lesser evil.

Actually, I think (as always), that it depends. For something simple like a config file for an app, JSON/YAML is usually fine.

But for something more complex, like IaC (Infrastructure as Code) definitions, I think perhaps "proper" programming languages might be more beneficial. I had a look at Pulumi just yesterday, and I very much like the idea of writing a simple C#/Typescript app to deploy my, when compared to something like HCL (HashiCorp Configuration Language) or bash scripts that wrap the Azure/AWS CLI tooling.

[liuliu]

Starlark is OK (it is very similar to Python except removed non-deterministic part). But the part where no type-hinting kicks in is when you try to read the actual underlying macros / functions the others provided. It is much harder to do without types (which Nickel seems would like to address).

Honestly, I would prefer anything that mimics popular languages to lower the bar of reading.

[FridgeSeal]

Have you tried Dhall? Static types and enough power to provide the tools you need, but deliberately not enough to allow full on arbitrary computation.

I've played with it briefly, along with the Kubernetes plugin, and it was a nice experience.

[vinceguidry]

Configuration isn't code isn't data. Data belongs in a database. Code belongs in a codebase. Configuration doesn't. Ideally config should be reduced down to keys and values and stored anywhere where it's easy to push to the environment where the code runs. I don't understand the immense expansion and proliferation of the config layer. I never touch code anymore. All I handle is config and tooling around it. YAML engineering.

Better configuration doesn't mean more ways to treat config like code, or data like config, or god forbid, code. It means treating config like config and code like code. Gitops just makes me sad. Truth should only flow in one direction. The first time I had to write a script to utilize the GitHub API to auto-update a code repo I died a little inside.

[pc86]

So we try to keep these three areas separate, and config generally ends up in our deployment pipeline. The problem is what do you do when code changes necessitate config changes? Adding/removing config properties, etc. We don't want 50 developers messing around with production deployment pipelines.

Doesn't something like this go a little way toward solving that problem?

[throwaway894345]

> I have a very hard time getting behind these complex configuration languages. To me what makes a configuration format good is the simplicity of reading the configuration of a program, and almost all of these languages are optimizing for feature complexity over readability. I think that all of the popular config formats (yaml, json, toml, etc) have issues, but none of the major issues with them have to do with being unable to represent a fibonacci sequence in their language.

Static languages like JSON and YAML are fine for toy configurations, but they don't scale to the most basic real-world configuration tasks. Consider any reasonably sized Kubernetes project that someone wants to make available for others to install in their cluster's. The project probably has thousands of lines of complex configuration but much of it will change subtly from one installation to another. Rather than distributing a copy of the configs and detailed instructions on how to manually configure the configuration for each use case, it becomes very naturally expedient to parameterize the configuration.

The most flat-footed solution involves text-based templates (a la jinja, mustache, etc) which is pretty much what Helm has done for a long time. But text-based templates are tremendously cumbersome (you have to make sure your templates always render syntactically correct and ideally also human readable, which is difficult because YAML is whitespace-sensitive and text templates aren't designed to make it easy to control whitespace).

A similarly naive solution is to simply encode a programming language into the YAML. Certain YAML forms encode references (e.g., `{"Ref": "<identifier>"}` is equivalent to dereferencing a variable in source code). Another program evaluates this implicit language at runtime. This is the CloudFormation approach, and it also gives you some crude reuse while leaving much to be desired.

After stumbling through a few of these silly permutations, it becomes evident that this reuse problem isn't different than the reuse problems that standard programming languages solve for; however, what is different is that we don't want our configuration to have access to system APIs including I/O and we may also want to prevent against non-halting programs (which is to say that we may not want our language to be turing complete). An expression-based configuration language becomes a natural fit.

After using an expression-based configuration language, you realize that it's pretty difficult to make sure that your JSON/YAML output has the right "shape" such that it will be accepted by Kubernetes or CloudFormation or whatever your target is, so you realize the need for static type annotations and a type checker.

Note that at no point are we trying to implement the fibonacci sequence, and in fact we prefer not to be able to implement it at all because we expressly prefer a language that is guaranteed to halt (though this isn't a requirement for all use cases, I believe it does satisfy the range of use-cases that we're discussing, and the principle of least power suggests that we should prefer it to turing-complete solutions).

[marcosdumay]

The use case of those executable configuration languages is that you often need to set the same setting on different programs, maybe even on different machines, but they must all reflect the same decision in different ways (like your services server can set a port to listen, so your firewall must set that port as open for internal traffic, and your applications must set that port as their data source).

That said, this one language does not look powerful enough for that. So I'm not sure where it can be used.

[wtetzner]

> That said, this one language does not look powerful enough for that. So I'm not sure where it can be used.

I mean, it's used to configure all of NixOS, so I'm not sure if that's true.

[cogman10]

Yeah... I honestly don't see the appeal of nickel.

It is sold as "You use this to generate configuration in other formats like JSON"... but why? Why would I want to use some language other than the target format to configure things? Why am I making my configuration a 2 step process? And even if I bought all of those reasons, why wouldn't I just use a general purpose language instead? Why have some esoteric language dialect whose only purpose is... making configuration files?

I'd much rather use Bash, python, perl, javascript, typescript, groovy, Java, kotlin, C++, C, Rust, erlang, php, awk, pascal, go, Nim, Nix, VB, Hax, coffeescript etc. Really, take your pick. Any well established language seems like a much better approach than something like this.

[aidenn0]

Here's a snippet for configuring a systemd timer on NixOS. Note that if I were to use the systemd configuration language, it would be spread across two files (the timer and the service itself)[1]. If I don't have "startAt" in the definition, it won't generate the timer file. If I spell it "statrAt" it will give me an error when I generate it (or in my editor if I have that configured). Note it's possible to fallback on using the json-like syntax to generate the ini-like systemd configuration files manually, which is useful to have when needed, but mostly it's about writing fairly simple functions that increase the signal-to-noise of the configuration file by removing boilerplate while at the same time detecting mistakes earlier.

  systemd.services.tarsnapback  = {
    startAt = "*-*-* 05:20:00";
    path = [ pkgs.coreutils ];
    environment = {
      HOME = "/home/XXXX";
    };
    script = ''${pkgs.tarsnap}/bin/tarsnap -c -f "$(uname -n)-$(date +%Y-%m-%d_%H-%M-%S)" "$HOME/ts" '';
    serviceConfig.User = "XXXX";
  };

1: Quick reference if you aren't familiar with systemd timers: https://wiki.archlinux.org/index.php/Systemd/Timers

[q3k]

Three things to address:

1) This doesn't have to be a two-step process. Specialized tools like kubecfg for Jsonnet will directly take a Jsonnet top-level config and instantiate it, traverse the tree, and apply the configuration intelligently to your Kubernetes Cluster.

2) General purpose languages are at a disadvantage, because most of them are impure. Languages that limit all filesystem imports to be local to a repository and disallow any I/O ensure that you can safely instantiate configuration on CI hosts, in production programs, etc. The fact that languages like Jsonnet also ship as a single binary (or simple library) that requires no environment setup, etc. also make them super easy to integrate to any stack.

3) Configuration languages tend to be functional, lazily evaluated and declarative, vastly simplifying building abstractions that feel more in-line with your data. This allows for progressive building of abstraction, from just a raw data representation, through removal of repeated fields, to anything you could imagine makes sense for your application.

Related reading: https://landing.google.com/sre/workbook/chapters/configurati...

[throwaway894345]

JSON and YAML don’t offer any abstraction. If you want to describe kubernetes resources in such a way that you can deploy the same resources to many environments with subtle differences between them (e.g., namespace names, DNS names, etc) you want something to let you abstract so you aren’t manually trying to keep disparate copies of thousands of lines of frequently-changing config in sync.

The reason you don’t use regular languages for this task is because you want to enforce termination (programs can’t run forever without halting, allowing someone to DoS your system) or reproducibility (the config program doesn’t evaluate to different JSON depending on some outside state because the program did I/O). If your use case involves users who can be trusted not to violate these principles, then a standard programming language can work fine, but this frequently isn’t the case.