[marcklingen]

This is interesting! Do you plan to add support for distinguishing between blocking and non-blocking/async migrations? This seems to be a common challenge when using Clickhouse and needing to change the ordering of existing tables.

Context: When adding Clickhouse to Langfuse [1] to store all observability data, the migration DX was a big challenge. Some migrations need to copy/reorder data between tables which can take a long time and thus need be performed in the background/async. Many teams run Langfuse self-hosted and reducing downtime of an upgrade is important, thus, we added some custom logic to run these “background migrations” for now [2]. For regular migrations we use golang-migrate, works decently although DX isn’t as good as eg Prisma for Postgres.

[1] OSS LLM Observability, https://github.com/langfuse/langfuse

[2] https://langfuse.com/self-hosting/background-migrations

[vortex_ape]

Interesting, we didn't run into the need for blocking and async migrations for our use case, because we do incremental updates to our clickhouse schema. What do you mean by needing to change the ordering of existing tables? I'll check out the background migrations link tomorrow!