[nomad86]

I don't know, I assigned this task to trusted specialists from India.

[maalber]

Wow, manually reviewing 50k photos is a lot! Would you be willing to share what the cost of that was?

[nomad86]

I created an app similar to Tinder that facilitates manual verification. Around 60 photos can be verified in one minute. The whole process took about a week and didn't cost much.

[karamanolev]

Doesn't sound that much. When I was playing with datasets, for simple tasks I only took around 3 seconds to classify an image. That's 1200/hour or on the order of 40 hours of work. That can't cost much when outsourced.

[moralestapia]

RLHF in the wild, nice.