[jftuga]

> General day to day is creating jobs that will process large amounts of input data and storing them into Snowflake

About how long do these typically take to execute? Minute, Tens of Minutes, Hours?

My work if very iterative where the feedback loop is only a few minutes long.

[Rietty]

Depends on the dataset anywhere from seconds to tens of minutes depending on preprocessing needed.

[Rietty]

Some of the largest are a few billion rows and we sample randomly when developing code then execute it on all