[mkleczek]

This.

In-RDBMS computation specified in declarative language with generic, protocol/technology specific adapters handling communication with external systems.

Treating RDBMS as a computing platform (and not merely as dumb data storage) makes systems simple and robust. Model your input as base relations (normalized to 5NF) and output as views.

Incremental computing engines such as https://github.com/feldera/feldera go even further with base relations not being persistent/stored.

[bambax]

Ha! I don't yet know much about 'incremental computing engines' but Feldera seem to be something I need. Because at some point I inevitably have to create materialized views to speed up some parts of the pipeline. Materialized views are of course a side effect and can become mildly dangerous if you're not careful to destroy/recreate them in time.

I was trying to think of a way to "only update new or changed rows" but it's not trivial. But Feldera seems to do exactly that. So thanks!