mjibson's comments

CockroachDB uses it as well (and is almost certainly the reason Materialize does): https://www.cockroachlabs.com/blog/efficient-documentation-u...

Nice story: around 2016 the OP project was not yet open source, just a web app. Cockroach Labs wanted to automate using it into their docs build. I asked and was approved to donate a fairly good amount to the author. My argument was that that amount of money, though high, was less than my time to add a few other features we wanted (BNF pre-processing things).

Having programmed in both Rust and Go: I mostly didn't need what this does in Go because my problem was usually solved by panic'ing, which prints a stack trace of each go routine, allowing me to get enough of the way through figuring out my current problem (usually something was stuck waiting or sending). In tokio, there's no such print-stack behavior, so it's much harder to get a snapshot of what's going on. (I'm relatively new to tokio and Rust, so there's perhaps a handy "print all the tokio tasks and their stack traces" method, but I haven't come across it yet.)

If folks use this console thing for perf reasons and not debug reasons, then yeah, maybe cool to have in Go.

The setup you describe is very much not simple. I worked at a place with very good DBAs and our replication setup caused us more downtime than anything else. Cockroach and Spanner exist because many programmers observed that what you describe is hard.

I'll guess: money. Postgres is decades old and was designed when the internet was smaller. Doing a large, fundamental change like this requires an already experienced person (or maybe more than one) to devote a lot of time designing and implementing some solution. This time costs money. So some company must be willing to employ or pay some people to work full-time on this for months. Anyone qualified to work on this should be very expensive, so full costs to pay experts for months of their time would be in the ~$100-200k level. Much outside the donate-a-cup-of-coffee-each-month range, and outside of any small startup's budget, too.

This suggests that the various companies employing people to work on Postgres-related stuff (like Microsoft, perhaps due to their purchase of Citus) have more lucrative work they'd rather do instead of improve this at the design level.

This problem is now perhaps larger than open source is designed to handle because of how expensive it is to fix. Very few people (zero in this case) are willing to freely donate months of their life to improve a situation that will enrich other companies.

Regarding the difficulty of doing this: the blog post here describes how the concurrency and transaction safety model is related to connections, so any connection-related work must also be aware of transaction safety (very scary).

I wrote one of the listed tools (github.com/mjibson/esc) and am thrilled about this proposal. I think it's great and solves all the problems in a great way.

It's a single click. There's no double clicks in acme. Did you watch the youtube video? Acme is about 5% usable with a keyboard.

I've been using acme for ~6 years now and it's still my daily editor. I wrote a LSP client for it (https://github.com/mjibson/acre). acme is so weird because when you start out it's like "wait so I have to write little shell scripts to do everything?". But then it slowly dawns that larger programs (like acre) are possible that are much more interactive, like modern IDEs.

Which version of CockroachDB was that? A few years ago it had a rudimentary heuristic optimizer that made plenty of poor choices. Today it has a quickly improving cost-based optimizer that makes much better choices. Each release now has significant jumps in which kinds of queries it can produce fast plans for.

Money prevents it. It takes money to host things and pay people to work on infrastructure. While people often volunteer to contribute to OSS products because they like or use them, not many are willing to write infrastructure that can handle this kind of traffic in their spare time. Even if you can find someone to donate the time, you'd still need to fund that infra in some way. Having an infra company (say, Google donates a bunch of GCP credits) to cover the hosting costs still puts the project at risk if the host company decides to stop funding.

Super mega ultra hard. It would take so much time for us to learn rust, port everything including all tooling, and fix new bugs we introduce that we wouldn't add any new features (but lots of new bugs!) for like 2-4 years and the company would die.

The Control of Nature (https://en.wikipedia.org/wiki/The_Control_of_Nature) is a book that contains this essay and two others (one about Iceland attempting to divert lava flows using pumped ocean water, one about southern CA attempting to build housing in an area dominated by complementary mudslides and firestorms). The whole book is a really great (in John McPhee's unique style) description of what happens when humans attempt to restrict or alter the earth's natural changes.

The above Atchafalaya essay is eye opening about the Mississippi River and how its natural course has swayed back and forth hundreds of miles over the centuries. We have now decided these two rivers should stop moving, but the earth doesn't see it that way. When they hit the gulf, their flow speed lowers, dropping the carried sediment. This causes their mouth to move slightly to an area with less dropped sediment. Humans have built walls attempting to constrain movement, but that may be a long-term losing battle.

Recommended reading, and a nice entry point to McPhee if you haven't read him yet.

As a former Mormon, this is exactly correct and scary for that reason. I am ashamed that I used to think this way.

We have a conference room named Tardigrade.

sqlfmt gets pretty close to this: https://sqlfum.pt/?n=30&indent=4&spaces=1&simplify=1&align=2...

https://www.cockroachlabs.com/blog/sql-fmt-online-sql-format... was on here last week. It's a sql formatter based on the same paper prettier is.

As CockroachDB matures these kinds of things will slowly get added.

But sqlfmt doesn't (currently) have a goal to format all SQL, just cockroach SQL.

Is that a thing? The Java driver must actually parse those and convert them. Java is crazy. https://github.com/mjibson/sqlfmt/issues/25 is the issue to track '?' as placeholders.

The formatter adheres to CockroachDB rules which is why it acts like you describe. Case is insensitive in names unless you double quote it. Postgres-style (CockroachDB is one) uses "$1" for placeholders instead of "?". Adding "?" would complicate our grammar enough that it's not worth adding it.

sqlfmt is not super useful for non-postgres grammars.

sqlfmt is inspired by the style of gofmt (https://blog.golang.org/go-fmt-your-code). The reason a tool having an opinion about formatting is useful is that it removes humans from having to care about formatting. I didn't like how gofmt formatted my Go code at first. But since I couldn't care, I didn't care, and now I never care about formatting. Without an opinion, we spend lots of time quibbling about stuff that doesn't matter.

Author here. Based on feedback I am going to make a sqlfmt binary for easy editor integration.

You can currently already do this by using the CockroachDB binary (https://www.cockroachlabs.com/docs/releases/v2.1.0-beta.2018...) but understandably people want a more minimal solution.