top | item 40134184

(no title)

guy4261 | 1 year ago

Looks pretty darn cool! Two questions please:

1. How does this compare to dbt? If we're already using dbt, why migrate?

2. Will you consider making a tool that tries to transpile SQL back to Hashquery models? This way I can work against my database, then merge the changes back to the model.

Good luck!

discuss

order

cpimhoff|1 year ago

> 1. How does this compare to dbt? If we're already using dbt, why migrate?

I actually think dbt and Hashquery work very well alongside one another!

dbt can help you normalize, clean, and materialize your data upstream, then Hashquery can be used to associate semantics to those output tables (measurements, synthesized attributes, common views) and query them.

So dbt can be the build/ETL part, and Hashquery can be the semantic layer/frontend for analytical queries.

> 2. Will you consider making a tool that tries to transpile SQL back to Hashquery models?

This is a really interesting idea!

I'm pretty skeptical we could make this technically feasible. Compilation from a higher level abstraction (Hashquery semantics) to a lower level abstraction is inherently lossy and can't really be done in reverse without a lot of noise.

Hashquery has a lot of escape hatches for raw SQL though if you need to access some functionality not yet implemented as part of the project. There's API to inline Hashquery structures inside of SQL fragments and visa versa.

peterhadlaw|1 year ago

I really don't understand the appeal of dbt vs a proper programming language. The templating approach leads to massive spaghetti. I look forward to trying out something like Ibis [0]

0: https://ibis-project.org/

BadHumans|1 year ago

You might not get the appeal of dbt because it is not meant for you. Dbt was made for data analyst who wanted the same engineering practices that software engineering teams have. The people who use it may have Python experience but they are not software engineers so dbt introduces concepts like testing and CI/CD to a group of people who don't historically use them.

mjirv|1 year ago

re: dbt, it looks like Hashquery is more for running analyses than doing in-data-warehouse transformations.

Unless I'm mistaken there's no native way in Hashquery right now to materialize the results back to your warehouse like you would with dbt.