[dvse]

Not surprising at all - kaggle (as currently implemented) is a fundamentally broken model. On top of the rather unpleasant "everybody pays auction" or "winner take all" system, they have a severe problem with metrics - the majority of the datasets are not anywhere near large enough to give stable out of sample error estimates, which means that in many cases the "winners" are barely better than random.

Perhaps they might be onto something with "kaggle prospect", but unless they pivot in some creative new direction, it's hard to see the service being very useful.

[antgoldbloom]

(Disclaimer: I work at Kaggle.)

We're actually using public competition as a way to find out who are some of the world's strongest data scientists. Once somebody has performed well in several public competitions, we start inviting them to private competitions where: 1. we invite 15 members; 2. prize money is generally much higher; and 3. everyone wins something (the higher the position the more one wins)

We're aiming to give many of the world's data scientists the opportunity to earn great full time incomes by competing in our competitions.

[dvse]

Thanks for the reply. If you are already running these private competitions it might be useful to advertise them a bit more openly (then you will need some formal criteria for joining to avoid user resentment, e.g. those in top 100). You are right that the system as is can help to find people who have some combination of persistence, general intuition for dealing with data and a modicum of modelling skills - "the world's best data scientists" is doubtful but certainly beats interviewing.