ELT your data with PlanetScale Connect

[cosmiccatnap]

Planetscale is really being pushed on this site lately but all I see is theory and promises of doing things different for vague reasons. I hope I'm wrong or just not understanding something but it feels like a CEO trying to sell a product without understanding the technical reasons why a project like this hasn't existed before. I can't imagine that planetscale is going to solve something that Google, amazon, Facebook, apple, and Netflix all failed to solve for a similar use case and for well documented reasons.

[josteink]

I thought the common denominator for this was (and has been, for decades) ETL? Extract, Transform, Load and in that order, because what other order would make sense?

Getting such basic things wrong doesn’t exactly give the reader the impression that the writer knows the subject particularly well.

[edenlinger]

Its not a typo but describes a product difference. ELT is a common term for a new type of data eng workflow that isn't mapped to ETL. In most ELT products the raw data is loaded into and then transformed by the data warehouse tool. As opposed to the ETL pipelines many of us are used to where the data gets transformed in a separate process before being dropped into the DW.

[dominotw]

data warehouses are now processing engines that scale compute independently of storage. you can simply dump your raw data into warehouse and do your transform there. Hence ELT.

[mmcgaha]

They aren’t the only ones using ELT instead of ETL. I hear they are different but I have no motivation to even Wikipedia the topic to find out.

[RHSeeger]

ETL is the standard terminology that I learned, going back decades.

[moltar]

See dbt (data build tool) for an example of ELT workflow.

[NonNefarious]

Emergency locator transmitter your data?

[ushakov]

how many ETL/ELT projects do we really need?

i feel like every day of a month there's a new flavour coming out

[dominotw]

This isn't an 'elt project'. They are supporting ELT out of their planetscale database for their users.

[mjirv]

Step 1: write an open-source connector for an open-source ELT tool

Step 2: paywall customers’ access to their own data so you can brand it as a premium feature/product

Step 3: ???

Step 4: profit

[throwusawayus]

worse, theyre pricing is based on innodb_rows_read, even includes cached rows.. so every single ELT job costs $$$ based on how many rows you have, even if all in buffer pool cache already

double-worse, innodb_rows_read known to be buggy! example https://jfg-mysql.blogspot.com/2022/03/rows-examined-not-tru... -- this bug is in customer's favor but what if other bugs are not?!