top | item 45704419

Against SQL (2021)

82 points| charles_irl | 4 months ago |scattered-thoughts.net

77 comments

order

layer8|4 months ago

This is mostly all true, but there is little incentive for RDBMS vendors to implement and maintain a second query language, in particular a shared cross-vendor one. Databases are the most long-lived and costly-to-migrate dependencies in IT systems, so keeping the SQL-based interface in parallel for a long time would be mandatory. This is compounded by the standardized SQL-centric database driver APIs like ODBC and JDBC. Despite the shortcomings of SQL, there is no real killer feature that would trigger the required concerted change across the industry.

andai|4 months ago

The way I've heard this phrased is, for potential customers to justify switching to your solution, it can't be 10% better, it needs to be 10x better.

(And on top of that they need to clearly perceive the value of Strange New Thing, and clearly perceive the relative lack of value of the thing they have been emotionally invested in for decades...)

gavinray|4 months ago

  > This is compounded by the standardized SQL-centric database driver APIs like ODBC and JDBC.
The criticality of JDBC/ODBC as a platform can't be understated. The JDBC API is the dominant platform for data access libraries. Compare number of drivers for JDBC, ODBC, go/sql, etc.

Newer platforms like Arrow ADBC/FlightSQL are better-suited to high-volume, OLAP style data queries we're seeing become commonplace today but the ecosystem and adoption haven't caught up.

https://arrow.apache.org/adbc/current/index.html

https://arrow.apache.org/docs/format/FlightSql.html

YZF|4 months ago

In my day job the question of SQL and its role keeps coming up. Some people want to propagate SQL all the way to clients like web browsers. Perhaps operating over some virtual/abstract data and not the real physical underlying data (that's a whole other layer of complexity). This seems like a bad idea/API in general.

I'm not too familiar with GraphQL but on the surface it seems like another bad idea. Shouldn't you always have some proper API abstraction between your components? My sense for this has been like GraphQL was invented out of the frustration of the frontend team needing to rely on backend teams for adding/changing APIs. But the answer can't be have no APIs?

All that said there might be some situations where your goal is to query raw/tabular data from the client. If that's your application then APIs that enable that can make sense. But most applications are not that.

EDIT: FWIW I do think SQL is pretty good at the job it is designed to do. Trying to replace it seems hard and with unclear value.

lelanthran|4 months ago

> All that said there might be some situations where your goal is to query raw/tabular data from the client. If that's your application then APIs that enable that can make sense. But most applications are not that.

IME, the majority of responses sent to the client is tabular data hammered into a JSON tree.

If you generalise all your response to tabular data, that lets you return scalar values (a table of exactly one row and one column), arrays (a table of exactly one row with multiple columns) or actual tables (a table of multiple rows with multiple columns).

The problem comes in when some of the values within those cells are trees themselves, but I suspect that can be solved by having a response contain multiple tables, with pointer-chasing on the client side reconstructing the trees within cells using the other tables in the response.

That would still leave the 1% of responses that actually are trees, though.

chao-|4 months ago

Instead of a client dealing with a server that only presents unopinionated, overly-broad CRUD endpoints for core entities/resources, GraphQL is a tool through which the client tricks the server into creating a bespoke viewmodel for it.

gavinray|4 months ago

  > My sense for this has been like GraphQL was invented out of the frustration of the frontend team needing to rely on backend teams for adding/changing APIs.
GraphQL was borne out of the frustration of backend teams not DOCUMENTING their API changes.

It's no different ideologically from gRPC, OpenAPI, or OData -- except for the ability to select subsets of fields, which not all of those provide.

Just a type-documented API that the server allows clients to introspect and ask for a listing of operations + schema types.

GQL resolvers are the same code that you'd find behind endpoint handlers for REST "POST /users/1", etc

nawgz|4 months ago

Re: GQL - Explain to me what abstraction layer should exist between the data model and what data is loaded into the client? I’ve never understood why injecting arbitrary complexity on top of the data model is wise.

Perhaps unfettered write access has its problems, and GQL has permissions that handle this issue plenty gracefully, but I don’t see why your data model should be obfuscated from your clients which rely on that data.

nuc1e0n|4 months ago

To use SQL effectively a certain amount of training is needed. But people are trained to read and write and do arithmatic. How to understand and write simple relational database queries is a broadly useful skill that should be widely taught in schools.

When it comes to written English, perhaps that could do with some reforms just as with SQL. Yet the way we write remains mostly unchanged.

nesarkvechnep|4 months ago

I hold a very unpopular opinion of GraphQL. I think it’s a great internal querying API. Every web backend project I’ve worked on tries to implement an API for querying data and it’s usually either fast and inflexible or flexible but slow. GraphQL allows to strike a balance, flexible and reasonably fast, with ways to optimise further.

sema4hacker|4 months ago

For any language as large and complicated as SQL, it's easy to come up with a long list of design problems. The difficulty is designing something better, and then even more difficult than that is getting people to use it.

jampekka|4 months ago

Much of the critique is that it's large and complicated because of bad design.

"Because SQL is so inexpressive, incompressible and non-porous it was never able to develop a library ecosystem. Instead, any new functionality that is regularly needed is added to the spec, often with it's own custom syntax. So if you develop a new SQL implementation you must also implement the entire ecosystem from scratch too because users can't implement it themselves.

This results in an enormous language."

JaggerFoo|4 months ago

SQL is great. I've used it to implement knapsack optimization for Daily Fantasy Sports at scale. I use it in Big Data tools and RDBMS. It's pervasive in data tech.

Feel free to innovate and bring forth other RDBMS/Data query languages and tools, perhaps something may succeed and stick as long as SQL has.

Cheers

tqi|4 months ago

Most of these arguments against seem like personal preferences? For example, I understand it would be convenient to give special treatment to foreign key joins, but i personally find `fk_join(foo, 'bar_id', bar, 'quux_id', quux)` less easy to understand on it's own, without having to look up the underlying table structures to know which tables have which (ie is quux_id a column in foo or bar?). Not to mention I've never worked anywhere where foreign keys were consistently used, mostly for perf reasons.

andai|4 months ago

Take a drink every time you see a comment that didn't even open the article ;)

pphysch|4 months ago

I think "SQL" is fine, whatever, I'm used to working with multiple different query and programming languages and dialects. That includes the freedom to define abstractions over SQL that meet my personal needs.

Standard SQL is not helpful, though. If that (failed) experiment was ended, database implementations would have even more freedom to explore superior syntax. Prescriptive language standards are a mistake.

throwaway894345|4 months ago

I like that SQL is a standard, and it's mostly "fine". Sure, I have to constantly read the man pages because there are half a dozen different ways to do fundamentally similar things, and there are subtle differences between each vendor, and I keep running into silly errors like trailing commas. But it mostly works.

The stuff that is more painful is building any kind of interesting application on top of a database. For example, as far as I know, it's very hard to "type check" a query (to get the "type" returned by a given query). It's also hard to efficiently compose SQL. And as far as I know, there's no standard, bulletproof way to escape SQL ("named parameters" is fine when you need to escape parameters, but most of SQL isn't parameters). There's also no good way to express sum types (a "place" can be a "park" or a "restaurant" or a "library", and each of those have different associated data--I don't need a "has_cycling_trails" boolean column for a restaurant, but I do for a park). There are various workarounds, all deeply unsatisfying.

GuinansEyebrows|4 months ago

Question for people who actually write app and SQL code: besides convenience, what is the upside of working with JSON in SQL over having your app construct and parse JSON objects, but storing the data in a database using more primitive types? My relatively inexperienced brain is telling me that it’s probably over complex to store and manipulate JSON objects at the DB level.

taffer|4 months ago

I use Postgres JSON functions to return nested results. The database itself contains no JSON; just a well-normalised data model. However, the queries return nested JSON in the format required by the application, e.g. for rendering an HTML template or returning JSON to the client in a single round trip. Check out the old dogs can sort of learn new tricks in this great article: https://www.scattered-thoughts.net/writing/sql-needed-struct...

gavinray|4 months ago

  > what is the upside of working with JSON in SQL over having your app construct and parse JSON objects, but storing the data in a database using more primitive types? 
You use map-like structures (JSON/HStore, etc) for semi-structured user data that you CAN'T define/know a rigid schema for, ahead-of-time.

Think usescases like: Allowing users to write configuration rules, or lists of custom tag <-> value pairs for (whatever), things of these sorts

Izkata|4 months ago

We use json columns to store raw data that comes from API responses or structured files. Anything from there we actually use gets copied out into its own strongly-typed column, but the json column remains as a type of logging/tracking as well as the original source of truth for database migrations if we need to pull a new column out, or track down some odd bug.

YZF|4 months ago

I've occasionally stored JSON directly in a database. It really depends on what you do with this data. If you do need to query and manipulate the internals of that JSON object then you should extract that data into a proper schema. But sometimes e.g. this is just something the frontend uses and you never (or rarely) have to query the internals, i.e. you treat it as an opaque blob.

blef|4 months ago

Feels old when you see how it played out to become SQL for everything in the data ecosystem lately.

Even though SQL as flaws, maybe a lot, it has one upside which is: it's so easy to onboard people on it, in the data ecosystem (warehousing etc.) it means that we can do way much stuff faster than before and hire less technical people, which is great

qaq|4 months ago

Thing is it's good enough and extremely widely used. Given that there is close to 0 chance an alternative will take off.

lelanthran|4 months ago

> Thing is it's good enough and extremely widely used.

The real problem is not that "it is good enough"; it's that SQL is still better than many of the newer proposals.

I mean, sure, if newcomer tech $BAR was slightly better than existing tech $FOO, then maybe $FOO might be eventually replaced. What we are seeing is that the newcomers are simply not better than the existing $FOO.

gavinray|4 months ago

I work at (what was previously known as) Hasura.

Specifically: the connector bits that deal w/ translating Relational Algebra IR expressed as GraphQL nodes -> SQL engine-specific code.

The author's comments about lack of standardization and portability might not get across just how nightmarishly different SQL dialects are.

I might put together a list of some of the batshit-insane bugs we've run into, even between version upgrades of the same engine.

I really think folks would raise an eyebrow if they understood just how much variance exists between implementations in what might be considered "common" functionality, and the sorts of contortions you have to do to get proper shims/polyfills/emulations.

nevertoolate|4 months ago

What is the approach? Do you target a subset of sql which you compile onto or you have some runtime dynamic dispatch thing and fight for code reuse with the magic haskell tools?

grebc|4 months ago

It’s not a small decision to switch database vendors.

Worrying that your data query language works across multiple vendors DB’s is not a concern ever considered imho.

j45|4 months ago

I'm agnostic between relational/non-relational.

SQL isn't for everything.

Neither is starting with NOSQL thinking it might be better and then proceeding to spend way too many man years making it a relational database, when learning a bit of SQL would have handled it fine.

jampekka|4 months ago

The post is not against the relational model. It's against SQL.

> The relational model is great ... but SQL is the only widely-used implementation of the relational model ...

janpio|4 months ago

(2021)

j45|4 months ago

Trying to understand how the year is relevant - still new to folks and still seems relevant.

dalmo3|4 months ago

404.

...Or is that the joke?

procaryote|4 months ago

TL;DR;

* a list of things they don't like in sql

* a list of traits they think a replacement should exhibit by negating the first list

I was kind of hoping for some example of what this much better language should look like

Spivak|4 months ago

I think the much better language would be the "no language" database. Throw portability to the wind and just have the client ship the query plan directly. The frontend to the database is howerver you want to expose it in your language of choice. I don't think there's any hope of getting disparate db vendors to agree on a compatible frontend language. It seems easier to externalize it.

The closest existing database to this ideal is probably FoundationDB although it also externalizes the query planner, which I don't necessarily consider a downside.

3eb7988a1663|4 months ago

I like PRQL[0] - fixes stupid warts about SQL.

A few top line items:

  - trailing commas not an error
  - queries can be read/written in linear order, starting with from, and ending on select
  - trivial intermediary keywords (eg you define month_total, and then can re-use month_total in a following calculation, no need to duplicate the calculation logic)
  - no need for a separate `having` keyword when `where` can just be a filter on a group
There is nothing too ground-breaking about it. Just streamlines some logic into a more holistic experience.

[0] https://prql-lang.org/

cess11|4 months ago

Maybe I'm holding TFA wrong but to me it seems like they're hinting at wanting a Prolog-as-database that could also become widely used, unlike actual Prolog.

It's not hyper-performant and mega web scale but the object database and Prolog like query language that comes with Picolisp is quite fun and sometimes rather useful, and has helped me think differently about how to model things in the default SQL database engines.

thom|4 months ago

[deleted]

throwaway894345|4 months ago

SQL itself doesn't generate any value, relational databases generate value. SQL is just a frontend for them. Anyway, your snark could be applied to _literally any change_. Are you angry that cars are replacing horses? "Horses generate such incomparable value that there's a steady supply of pro-car posts, like fumes from the vast ocean of their constantly boiling piss". Don't like the cotton gin? "Slave labor generates such incomparable value...". There's not really anything of substance in this kind of comment.

zkmon|4 months ago

So I guess the author is trying to help a decision maker to make a decision when faced with a question of whether to use SQL or not. But in reality that question would be settled by other factors and contextual reasons rather than the arguments provided by the author.

For instance, analytics usecases favor SQL stores, as slicing and dicing is better done with row or column stores instead of document databases.

Also, Postgres is getting more popular for lot of usecases, so SQL is here to stay.

geysersam|4 months ago

> So I guess the author is trying to help a decision maker to make a decision when faced with a question of whether to use SQL or not.

That's not my impression. A decision maker today should typically make the decision to use SQL. I'm pretty sure the author would agree with that.

I think the target audience is language designers and tool builders. The author is urging people to envision and build new better interfaces to interact with relational data.