>> The average number of failures for those who failed at least once before success was 2.03 for NIH, 1.5 for startups and 3.90 for terrorist groups.
Regarding counter-terrorism, that would suggest that rather than more surveillance we instead need better reporting of and responding to failed terrorist incidents/attacks.
This reminds me of (I think) Richard Feynman's comment about the Challenger, to the effect that when the o-rings were eroded by 1/3 or whatever, they called that a safety factor of 3, whereas they should have stopped and said "wait, it wasn't supposed to do that at all, something is seriously wrong".
I'm not sure if it was him or someone else that pointed out when you do a post-mortem on a disaster, you generally find a history of near-misses, but people don't take near-misses as seriously as they should.
There was once a mathematician who got caught at the airport with a bomb. They took him in for questioning. When asked why he had a bomb in his suitcase he stated "the odds of there being a bomb on your airplane are less than one in ten million. So I figured what are the odds of there being two bombs on my flight?"
Do you see why this is similar? When an outcome (terrorism) becomes an input to its own predictor (past terrorism failures), the logic breaks.
I can't speak to venture funding or terrorism, but something seems really misleading to me about the NIH grant applications model.
My sense is that a grant, like an academic paper, is often basically well-received or not. So prior to going into it, if you're familiar with the details of the particular grant, you can get some sense of interest or not. The grant may have even been solicited in a certain sense by a program officer or something, so it's quasi-invited.
Those grants that are quasi-invited, or well-received, will basically involve polishing on subsequent revisions. Those that are totally unsolicited and not well-received it doesn't matter how much polishing you do often, it will not go anywhere.
I think where this becomes relevant to the paper is that this often has very little to do with the process of the grant revisions. It says little about "how someone responds to failure", and everything about connections with the grant agency, program officers, and luck. All that stuff that happens before the grant is even submitted plays into it, and this paper kind of ignores that.
You could say that it does speak to learning, in that you could ask "why doesn't the other scientist adjust their strategy?" To which I'd say, there are enormous pressures to submit anyway, and for someone who doesn't get good mentoring about what to target, or what agency to target, or whose research doesn't jive with the priorities of the division head at that time, or whatever, they might just not get it, and it might easily be over the 5 year period of the study.
Basically, at least with NIH grants I think this study is really misleading and potentially harmful, because it kind of suggests failed applicants just aren't learning, when I think what's really happening is that you have a mixture of two groups, one of which can learn something, and the other of whom it just doesn't matter if they learn or not, because the outcome has kind of been preordained.
That is, the process they predict from is an indicator of which mixture class the applicant belongs in, not the cause of the outcome.
>the faster you fail, the better your chances of success, and the more time between attempts, the more likely you are to fail again
This is confounded by the fact that the closer one is to success the more frequent their attempts tend to be. The article doesn't indicate the research took this into consideration.
e.g. Golfers takes shots more frequently the closer they are to a hole. But telling a golfer from the start to take a series of frequent short shots is bad advice.
Golf match should not be defined as a consecutive failures before a potential success, but more of a set of strategic actions before a potential success.
But yes, it is not defined what they took into account. e.g. Calling the same potential client hundred times in a row, can get you a restraining order only.
If qualitative input and output of an attempt would be taken in account, quantity of an attempt would not really be that significant.
It is also confounded by the element of ruin, the ability of a gambler to stay in the game based on their stake. It's reasonable to suppose that some of the lag in re-try times (particularly with regard to startups) is attributable to raising money, which will be much easier for someone who starts out wealthy to begin with, even if they're not directly investing their own funds.
[+] [-] itronitron|6 years ago|reply
Regarding counter-terrorism, that would suggest that rather than more surveillance we instead need better reporting of and responding to failed terrorist incidents/attacks.
[+] [-] perl4ever|6 years ago|reply
I'm not sure if it was him or someone else that pointed out when you do a post-mortem on a disaster, you generally find a history of near-misses, but people don't take near-misses as seriously as they should.
[+] [-] ummwhat|6 years ago|reply
Do you see why this is similar? When an outcome (terrorism) becomes an input to its own predictor (past terrorism failures), the logic breaks.
[+] [-] golemotron|6 years ago|reply
[+] [-] o09rdk|6 years ago|reply
My sense is that a grant, like an academic paper, is often basically well-received or not. So prior to going into it, if you're familiar with the details of the particular grant, you can get some sense of interest or not. The grant may have even been solicited in a certain sense by a program officer or something, so it's quasi-invited.
Those grants that are quasi-invited, or well-received, will basically involve polishing on subsequent revisions. Those that are totally unsolicited and not well-received it doesn't matter how much polishing you do often, it will not go anywhere.
I think where this becomes relevant to the paper is that this often has very little to do with the process of the grant revisions. It says little about "how someone responds to failure", and everything about connections with the grant agency, program officers, and luck. All that stuff that happens before the grant is even submitted plays into it, and this paper kind of ignores that.
You could say that it does speak to learning, in that you could ask "why doesn't the other scientist adjust their strategy?" To which I'd say, there are enormous pressures to submit anyway, and for someone who doesn't get good mentoring about what to target, or what agency to target, or whose research doesn't jive with the priorities of the division head at that time, or whatever, they might just not get it, and it might easily be over the 5 year period of the study.
Basically, at least with NIH grants I think this study is really misleading and potentially harmful, because it kind of suggests failed applicants just aren't learning, when I think what's really happening is that you have a mixture of two groups, one of which can learn something, and the other of whom it just doesn't matter if they learn or not, because the outcome has kind of been preordained.
That is, the process they predict from is an indicator of which mixture class the applicant belongs in, not the cause of the outcome.
[+] [-] kekebo|6 years ago|reply
[+] [-] Dowwie|6 years ago|reply
[+] [-] Fernicia|6 years ago|reply
This is confounded by the fact that the closer one is to success the more frequent their attempts tend to be. The article doesn't indicate the research took this into consideration.
e.g. Golfers takes shots more frequently the closer they are to a hole. But telling a golfer from the start to take a series of frequent short shots is bad advice.
[+] [-] keymon-o|6 years ago|reply
But yes, it is not defined what they took into account. e.g. Calling the same potential client hundred times in a row, can get you a restraining order only.
If qualitative input and output of an attempt would be taken in account, quantity of an attempt would not really be that significant.
[+] [-] vannevar|6 years ago|reply
[+] [-] fsloth|6 years ago|reply
[+] [-] GoToRO|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] marcosdumay|6 years ago|reply
[+] [-] lifeisstillgood|6 years ago|reply
[+] [-] reportgunner|6 years ago|reply
[+] [-] keymon-o|6 years ago|reply
I guess we can only wait for the title "Here is 10 reasons why you are a loser, Harry..."..
[+] [-] sqba|6 years ago|reply