My S/O used to work at a company AuntBertha. They're a "search engine for social services" but they are a for-profit company (a B-Corp).
Their biggest clients are hospitals and insurers. There are people called "frequent flyers" who come to the emergency room frequently and while they might only be 2% of clients they account for a huge share of costs. It's actually cheaper to hire social workers to provide these people with the things they actually need (i.e. treating the causes) instead of the symptoms.
Here's an example. Guy is low income. Cannot afford food & insulin for diabetes. So he chooses to not buy insulin and the food he does buy is high in sugar (think cheap whitebread). Goes into diabetic shock and gets sent to the emergency room. Instead it's much cheaper for everyone involved if a social worker finds out the problem, helps him research some foodbanks to cover his nutritional needs or gets him on a discount insulin program. It's not free, but it's much cheaper than trying to clean up a mess after the fact.
I often wish we in the US would separate out long-term medicine from insurance. It really violates any rational for insurance to have a continuous, known cost be part of that. Also, it ups the insurance cost for everyone else.
I wish we had a separate drug assistance program. Something like if you are making less than 200 or 300% poverty[1] the government will pay the full cost[2] and then just pro-rate it from there. Since we are talking long term drugs, its by definition something that would be applicable to preexisting conditions. I worry that it will not take the steps necessary to control fraud or run out of money like IHS.
This is a large problem in Canada as well. A relative is an ER nurse and she has endless amounts to say about the number of people who come in for these kinds of reasons, or less. The hospital she works at is in a town that was economically decimated in the late 80's and only just started to recover. The number of people who go into emergency because of alcoholism, self-neglect, loneliness, and of course (probably encompassing the rest) mental illness is staggering. She says it's one thing to know their names, but it's bad when you know how their [grand-/]children are doing in school, and who visited or did not visit last week. She is not shy about how much she thinks that social programs need to be expanded in accessibility. I agree.
From what I understand, much of the medical world feels the same way. That's one thing I would certainly add to Canadian health care -- some kind of minimum of public mental health and social work. There are a lot of programs already that don't attach a fee for the patient/subject, but they're not necessarily common knowledge or accessible to people as a "walk in and book" sort of service.
edit: To wit, I think the reasons social programs don't work is inaccessibility to those who really need them, lack of familiarity with them, and taboo (but this last one is probably another discussion). I think right now we need more of them, and they need to have access to proper resources and that their end benefit to society would be substantial. I'd be interested in seeing some projections surrounding that to be sure.
I participated in a social program funded by the gov in the 90's as the data person. It was a combined early childhood education / social service program. I won't discuss what I thought of the outcomes or the cost / benefits, but I think there is a piece a lot of people miss.
There are consultants in DC that take all the data from a program and summarize it. They have a contract and aren't always the folks who have the assistance contract on the grant. People are sadly, people, and sometimes the summarizing folks have some pretty powerful incentives to put a spin on the numbers, drop certain grants, and well, other things. Life is incentive based after all.
When I look at the outcomes, I often wish there was a way to get some anonymized raw take because I am a bit mistrustful of third-party analysis particularly when they aren't up on simple things like geographical and economic differences. Plus the tendency to roll all the results in one set of numbers ignores how big the USA is and that San Francisco is a bit different than Salt Lake City or Rosebud.
PS: one thing, it is amazing how many children start life with hearing problems - fix that early and save the rest of us a lot of money. Plus, fatalogic has it right in my book, life is complicated, one axis of action doesn't really cut it.
I'm glad that you mentioned hearing problems (ear infections specifically). They have been shown to affect child development in ways that aren't immediately intuitive, like hyperactivity and their ability to pick up phonics [1]. That obviously has an impact on their early learning, and consequently the rest of their lives.
As a former social worker I think most social programs don't work because they try to deal with multifaceted problems from a single point of action. It's like trying to fix a suspension bridge by replacing one cable while all the other cables are broken.
The tl;dr being in the mid 1900s when somebody actually DID try to measure the results of a mentor program for at-risk youth, they found the impact was slightly negative.
Actually fixing societal problems is a really hard problem and what actually works is often very counterintuitive. I think the whole field could do to let the data guide them better (also difficult because so many outcomes are "make people feel better", and most people are pretty bad at actually knowing how they are or will be feeling).
I've long had this thought about charity itself, it seems that a lot of charity is propping up bits and pieces of capitalism so we don't look at the root causes of why we "need" charity in the first place.
On one hand it's great that children around the US can have life saving surgery because so many donated to their cause, on the other hand would we all be better off donating to a lobbying cause for single-payer healthcare?
If we could get that passed, every child in the US could have life saving surgery if needed and it wouldn't be a matter of whether or not their community is charitable enough.
People are failing to recognize that it doesnt matter what percentage of programs fail/succeed. The metric that matters is the ROI on programs. If 25% of programs succeed, giving a 100x ROI, then they more than make up for the failed programs.
Unless we know a better way (UBI seems promising) to help people, then programs seem to be the best bet.
The problem is that people hear "75% of programs fail" and think "Dont start programs". This is like saying "75% of businesses fail therefore do not start a business" .
I appreciate the last section's perspective: that giving to the causes best supported by evidence, though very helpful, does not guarantee 10x (social) returns since measurement is imperfect and some big returns are likely from things that aren't fully studied yet. Besides how that makes sense in its own right, it sort of pushes folks to recognize that that the many useful strategies the effective altruism community has come up with still aren't a silver bullet for aid effectiveness.
Recognizing there's no silver bullet seems especially relevant to me because there's a folk interpretation of "most social programs don't work" that's more like "we spend tens of billions of dollars trying to alleviate poverty; why isn't it gone now?". And _that_ question only seems to make sense because people misjudge the size of the challenge relative to the amount spent on it and the returns you can expect from that money.
The U.S.'s yearly global health budget, if spent entirely on sub-Saharan Africa, would be under $20/person, and you don't, say, raise average income $10k with $20, no matter how wisely or creatively you invest it. Demanding that kind of return from charity is like expecting every investment to perform like early money in Google, except, perhaps, that the impact of your demand is less on the investor and more on the global poor.
Separately, it seems like a shame to me that in fields where we do have strong signs of a high multiplier, like bednets or deworming, the opportunities haven't already been fully exploited by governments and billionaires. Some of it is that large spenders are making their choices through a process much different from someone like GiveWell's, but some is also that governments, for example, spend a lot less of their budgets on non-strategic global-health aid than most think (<1% for the US). More about aid spending here: http://www.npr.org/sections/goatsandsoda/2015/02/10/38387558...
This is silly. The biggest social programs in America are Social Security (basically a UBI for old folks), Medicare (health care), and the #1 economic development success of all time, public education. The kind of social programs they're talking about in this article are so unambitious it's hardly surprising their results aren't measurable.
I would love to see more work-for-benefits programs. Where you are required to do some labour work for the community (cleaning wild areas, swiping side walks, planting trees etc.) in exchange for some better than average benefits. IMO everyone should be entitled to such a work. This way we would eliminate involuntary extreme poverty.
Many people who are currently homeless are not the bums you see on the side of the road. Many have assets (computers, clothes, maybe even cars) but have been temporarily displaced from a home. Usually due to unforeseen financial hardship like an unexpected medical bill, or an auto accident etc. These people don't have a safety net of family or friends they could borrow money from so they get evicted. These people may currently have a job but no place to stay when they leave form work.
Many of the successful homeless programs are "housing first" programs. Search for "colorado homes first program" to read more about it. In many cases it's also the cheapest way to do it as well. If you want to organize a group of currently homeless people to do some task, you have to coordinate travel, pick them up, have a van or a truck, hire a coordinator to plan the events and oversee them, and then at then on top of it pay for housing. It's actually cheaper just to provide housing.
Anyway, just wanted to say the issue goes deeper than "why won't these lazybones work".
> Where you are required to do some labour work for the community (cleaning wild areas, swiping side walks, planting trees etc.) in exchange for some better than average benefits.
All of the things you listed are geared towards improving the looks and desirability of communities, which will not work to benefit the people receiving the benefits in the current US housing system because of gentrification. Forcing poor people to work to receive the equivalent of less-than-minimum-wage in benefits to drive gentrification and real estate speculation will price those people out of housing in their own neighborhoods and make their plight worse.
This is not hyperbole or hypothetical, it actually happens today in many places in the United States. There was an article in the Guardian yesterday about homeless people in Portland being jailed and then forced to work for $1 a day to destroy the homeless camps that they had lived in:
I think this is a great idea, and we should apply it to insurance benefits in other areas. Go to the doctor? You have to clean the gutters in a street for a few hours. Claim for stolen property? You have to help paint fences. House burns down? Get on those overalls. Car accident? Time to dig a hole.
You wouldn't have to do it, you just wouldn't get full payment unless you did.
It would be good to get your read on whether this is absence of evidence, or evidence of absence. To me (I work in stats) it sounds like the former, but the general reader could get the impression of the latter, especially given the title. What's your take?
I think the title is phrased badly. You wouldn't say "most businesses don't work", because business is obviously doing just fine. Yet it's true, since most businesses fail in the first few years - 80% by some calculations. And you'd be laughed out of the room if you used this statistic to argue that businesses aren't working.
Thank you for including such a thorough and approachable conclusion. As someone with no background in statistics, it helped me understand the content.
Genetic mutations seem to have similar properties (by analogy). Most don't 'work', but evolution still uses random mutation to build better organisms. Progress is sensitive to the degree of selection pressure: too small, and unfit organisms will reproduce; too great, and even fit organisms will frequently die.
I wonder if there's an empirical way to determine the optimal degree of selection pressure (where "optimal" means the degree which most rapidly improves the organisms) and whether the same logic could be applied to determine what fraction of underperforming social programs to cut and replace with new programs.
The way of telling if it works is: statistical significance.
This "approach" to research was debunked as pseudoscience long ago (eg here is one from 1967[1], and that wasn't the first)... So the answer is: no one knows what works or doesn't, (edit: at least not based on the usual claimed method of drawing conclusions from the data).
We discuss this somewhat in the post - the line at which you say something "works" is arbitrary. We explore how the choice of definition changes the stat.
I would imagine most of them can't, it amounts to trying to bail out the ocean, one bucket at a time.
In less hyperbolic terms, many of these programs are attempting to tackle intractable societal problems when - even for ones that are relatively well-funded - the cost of organizational overhead, limited ability to cause lasting impact outside of their specific area of influence, etc., reduce what they do to a lot of good intentions and few meaningful results.
Depending on the "clientele," it amounts to attempting to provide assistance to people who really can't or don't want to be helped (hard drugs and the chronic homeless, for example).
"Is it fair to say that it's hard to create good social programs? And we should all remember that some social programs work far better than most people realize."
Something like that might not give the wrong impression...?
The big secret is social programs are largely to promote stability and prevent civil unrest. The effectiveness at the stated intention is a secondary concern.
We probably are better off with a basic income style system where people can opt-into public (or private) education, public (or private) medical services, and the balance is paid out to them in cash every month. The cost being recouped via cutting existing programs and raising tax revenues to recapture 100% of the value once you have a 6 figure household income.
"So is it fair to say 'most social programmes don’t work?'
I think this is a little ambiguous and potentially misleading. Individual projects mostly don’t work, but whole areas often do have a positive impact. So, if you pick an intervention at random, then on average your impact will be positive, because there’s a small but important chance of you picking one of the good ones."
Can we fix this to the proper, non clickbait title of "What fraction of social programmes don’t work?"
"4 Reasons Why The Answer To This Complicated Social Science Question is More Subtle Than You Think - The Conceptual Clarification in 3 Will Shock You"
[+] [-] schneems|8 years ago|reply
Their biggest clients are hospitals and insurers. There are people called "frequent flyers" who come to the emergency room frequently and while they might only be 2% of clients they account for a huge share of costs. It's actually cheaper to hire social workers to provide these people with the things they actually need (i.e. treating the causes) instead of the symptoms.
Here's an example. Guy is low income. Cannot afford food & insulin for diabetes. So he chooses to not buy insulin and the food he does buy is high in sugar (think cheap whitebread). Goes into diabetic shock and gets sent to the emergency room. Instead it's much cheaper for everyone involved if a social worker finds out the problem, helps him research some foodbanks to cover his nutritional needs or gets him on a discount insulin program. It's not free, but it's much cheaper than trying to clean up a mess after the fact.
[+] [-] protomyth|8 years ago|reply
I wish we had a separate drug assistance program. Something like if you are making less than 200 or 300% poverty[1] the government will pay the full cost[2] and then just pro-rate it from there. Since we are talking long term drugs, its by definition something that would be applicable to preexisting conditions. I worry that it will not take the steps necessary to control fraud or run out of money like IHS.
1) http://familiesusa.org/product/federal-poverty-guidelines
2) obviously some upper limits apply https://www.medicaid.gov/medicaid/prescription-drugs/pharmac...
[+] [-] auggierose|8 years ago|reply
[+] [-] 52-6F-62|8 years ago|reply
From what I understand, much of the medical world feels the same way. That's one thing I would certainly add to Canadian health care -- some kind of minimum of public mental health and social work. There are a lot of programs already that don't attach a fee for the patient/subject, but they're not necessarily common knowledge or accessible to people as a "walk in and book" sort of service.
edit: To wit, I think the reasons social programs don't work is inaccessibility to those who really need them, lack of familiarity with them, and taboo (but this last one is probably another discussion). I think right now we need more of them, and they need to have access to proper resources and that their end benefit to society would be substantial. I'd be interested in seeing some projections surrounding that to be sure.
[+] [-] lcamacho84|8 years ago|reply
[+] [-] protomyth|8 years ago|reply
There are consultants in DC that take all the data from a program and summarize it. They have a contract and aren't always the folks who have the assistance contract on the grant. People are sadly, people, and sometimes the summarizing folks have some pretty powerful incentives to put a spin on the numbers, drop certain grants, and well, other things. Life is incentive based after all.
When I look at the outcomes, I often wish there was a way to get some anonymized raw take because I am a bit mistrustful of third-party analysis particularly when they aren't up on simple things like geographical and economic differences. Plus the tendency to roll all the results in one set of numbers ignores how big the USA is and that San Francisco is a bit different than Salt Lake City or Rosebud.
PS: one thing, it is amazing how many children start life with hearing problems - fix that early and save the rest of us a lot of money. Plus, fatalogic has it right in my book, life is complicated, one axis of action doesn't really cut it.
[+] [-] jansho|8 years ago|reply
[1] https://www.hslda.org/strugglinglearner/CraftDocs/EarInfect....
[+] [-] fatalogic|8 years ago|reply
[+] [-] forgotmysn|8 years ago|reply
[+] [-] pillowkusis|8 years ago|reply
The tl;dr being in the mid 1900s when somebody actually DID try to measure the results of a mentor program for at-risk youth, they found the impact was slightly negative.
Actually fixing societal problems is a really hard problem and what actually works is often very counterintuitive. I think the whole field could do to let the data guide them better (also difficult because so many outcomes are "make people feel better", and most people are pretty bad at actually knowing how they are or will be feeling).
[+] [-] xxSparkleSxx|8 years ago|reply
On one hand it's great that children around the US can have life saving surgery because so many donated to their cause, on the other hand would we all be better off donating to a lobbying cause for single-payer healthcare?
If we could get that passed, every child in the US could have life saving surgery if needed and it wouldn't be a matter of whether or not their community is charitable enough.
[+] [-] maerF0x0|8 years ago|reply
The problem is that people hear "75% of programs fail" and think "Dont start programs". This is like saying "75% of businesses fail therefore do not start a business" .
[+] [-] kerkeslager|8 years ago|reply
[+] [-] BenjaminTodd|8 years ago|reply
[+] [-] twotwotwo|8 years ago|reply
Recognizing there's no silver bullet seems especially relevant to me because there's a folk interpretation of "most social programs don't work" that's more like "we spend tens of billions of dollars trying to alleviate poverty; why isn't it gone now?". And _that_ question only seems to make sense because people misjudge the size of the challenge relative to the amount spent on it and the returns you can expect from that money.
The U.S.'s yearly global health budget, if spent entirely on sub-Saharan Africa, would be under $20/person, and you don't, say, raise average income $10k with $20, no matter how wisely or creatively you invest it. Demanding that kind of return from charity is like expecting every investment to perform like early money in Google, except, perhaps, that the impact of your demand is less on the investor and more on the global poor.
Separately, it seems like a shame to me that in fields where we do have strong signs of a high multiplier, like bednets or deworming, the opportunities haven't already been fully exploited by governments and billionaires. Some of it is that large spenders are making their choices through a process much different from someone like GiveWell's, but some is also that governments, for example, spend a lot less of their budgets on non-strategic global-health aid than most think (<1% for the US). More about aid spending here: http://www.npr.org/sections/goatsandsoda/2015/02/10/38387558...
[+] [-] chrismealy|8 years ago|reply
[+] [-] therebedragons|8 years ago|reply
[deleted]
[+] [-] BartSaM|8 years ago|reply
[+] [-] schneems|8 years ago|reply
Many of the successful homeless programs are "housing first" programs. Search for "colorado homes first program" to read more about it. In many cases it's also the cheapest way to do it as well. If you want to organize a group of currently homeless people to do some task, you have to coordinate travel, pick them up, have a van or a truck, hire a coordinator to plan the events and oversee them, and then at then on top of it pay for housing. It's actually cheaper just to provide housing.
Anyway, just wanted to say the issue goes deeper than "why won't these lazybones work".
[+] [-] sedachv|8 years ago|reply
All of the things you listed are geared towards improving the looks and desirability of communities, which will not work to benefit the people receiving the benefits in the current US housing system because of gentrification. Forcing poor people to work to receive the equivalent of less-than-minimum-wage in benefits to drive gentrification and real estate speculation will price those people out of housing in their own neighborhoods and make their plight worse.
This is not hyperbole or hypothetical, it actually happens today in many places in the United States. There was an article in the Guardian yesterday about homeless people in Portland being jailed and then forced to work for $1 a day to destroy the homeless camps that they had lived in:
https://www.theguardian.com/us-news/2017/aug/09/portland-hom...
[+] [-] wmf|8 years ago|reply
[+] [-] yarrel|8 years ago|reply
You wouldn't have to do it, you just wouldn't get full payment unless you did.
[+] [-] robertwiblin|8 years ago|reply
[+] [-] confounded|8 years ago|reply
[+] [-] sharemywin|8 years ago|reply
[+] [-] vacri|8 years ago|reply
[+] [-] jake-low|8 years ago|reply
Genetic mutations seem to have similar properties (by analogy). Most don't 'work', but evolution still uses random mutation to build better organisms. Progress is sensitive to the degree of selection pressure: too small, and unfit organisms will reproduce; too great, and even fit organisms will frequently die.
I wonder if there's an empirical way to determine the optimal degree of selection pressure (where "optimal" means the degree which most rapidly improves the organisms) and whether the same logic could be applied to determine what fraction of underperforming social programs to cut and replace with new programs.
[+] [-] therebedragons|8 years ago|reply
[deleted]
[+] [-] nonbel|8 years ago|reply
This "approach" to research was debunked as pseudoscience long ago (eg here is one from 1967[1], and that wasn't the first)... So the answer is: no one knows what works or doesn't, (edit: at least not based on the usual claimed method of drawing conclusions from the data).
[1] http://www.fisme.science.uu.nl/staff/christianb/downloads/me...
[+] [-] cuchoi|8 years ago|reply
[+] [-] confounded|8 years ago|reply
[+] [-] BenjaminTodd|8 years ago|reply
[+] [-] zebraflask|8 years ago|reply
In less hyperbolic terms, many of these programs are attempting to tackle intractable societal problems when - even for ones that are relatively well-funded - the cost of organizational overhead, limited ability to cause lasting impact outside of their specific area of influence, etc., reduce what they do to a lot of good intentions and few meaningful results.
Depending on the "clientele," it amounts to attempting to provide assistance to people who really can't or don't want to be helped (hard drugs and the chronic homeless, for example).
[+] [-] VikingCoder|8 years ago|reply
Something like that might not give the wrong impression...?
[+] [-] fweespeech|8 years ago|reply
We probably are better off with a basic income style system where people can opt-into public (or private) education, public (or private) medical services, and the balance is paid out to them in cash every month. The cost being recouped via cutting existing programs and raising tax revenues to recapture 100% of the value once you have a 6 figure household income.
[+] [-] omegaworks|8 years ago|reply
"So is it fair to say 'most social programmes don’t work?'
I think this is a little ambiguous and potentially misleading. Individual projects mostly don’t work, but whole areas often do have a positive impact. So, if you pick an intervention at random, then on average your impact will be positive, because there’s a small but important chance of you picking one of the good ones."
Can we fix this to the proper, non clickbait title of "What fraction of social programmes don’t work?"
[+] [-] robertwiblin|8 years ago|reply
[+] [-] schneems|8 years ago|reply
[+] [-] BeStaXP|8 years ago|reply
[+] [-] AnimalMuppet|8 years ago|reply
[+] [-] AnimalMuppet|8 years ago|reply