[ceh123]

We don't need first principals thinking every time, but having an understanding of why you can't just test 100 variations of your hypothesis and accept p=0.05 as "statistically significant" is important.

Additionally it's quite useful to have the background to understand the differences between Pearson correlation and Spearman rank, or why you might want to use Welch's t-test vs students, etc.

Not that you should know all of these things off the top of your head necessarily, but you should have the foundation to be able to quickly learn them, and you should know what assumptions the tests you're using actually make.