top | item 44563655

(no title)

wingspar | 7 months ago

Honestly asking, How did you validate your results?

discuss

order

jandrewrogers|7 months ago

In this particular case it was just a proof-of-concept, albeit at scale. We did not run a proper ground-truthing process but people actually running that type of data model in production could have ground-truthed the analytic model if they wanted to.

However, it turns out that thousands of people like to talk about their flights on social media, so we scraped that as a spot check and it mostly lined up perfectly. Good enough for a demo and it would have been difficult to come up with an alternative explanation for the patterns in the data.

The purpose of the PoC was to sell the data analysis infrastructure that made that type analysis possible at scale, it wasn't about the data per se. It was a compelling demo we invented given the data that happened to be available. Startup life.

jcranmer|7 months ago

> Good enough for a demo and it would have been difficult to come up with an alternative explanation for the patterns in the data.

For fun edge cases, there's always Antarctica, where you can travel from a US base (which looks like you're in the US) to a NZ base (which looks like you're in NZ) in a couple of minutes: https://brr.fyi/posts/credit-card-shenanigans

fsckboy|7 months ago

i don't have any special knowledge in this area, but just thinking about it idly while sitting here, "robbing their homes while they are away" comes to mind as a good proxy.

iterance|7 months ago

That seems like a risk, but not a validation method, unless you are feeling particularly bold.

wingspar|7 months ago

Basically a plot line on the show “Black List”. Had an inside guy at the post office who would forward people stopping mail delivery on vacation. Then used homes as safe houses.