Taking a sentient human being and throwing them in a cage is a profoundly violent act. I find it troubling that you guys so casually reach for it as a punitive tool, particularly when the subject has neither committed physical violence nor poses such a threat to others. Surely you clever people can think of forms of punishment/deterrence less destructive to both the individual and society as a whole.
According to them, they got approval for doing that:
> The alternative was to upload it to the cloud using tools such as Google Storage and use BigQuery to extract data from it. As PA has an existing relationship with Google, we pursued this route (with appropriate approval). This showed that it is possible to get even sensitive data in the cloud and apply proper safeguards.
PA purchased the commercially available Hospital Episode Statistics data set from the NHS Information Centre (now the Health and Social Care Information Centre). The data set does not contain information linked to specific individuals. The information is held securely in the cloud in accordance with conditions specified and approved by HSCIC.
This new approach to analytics can help the NHS improve patient care. We have been able to identify where services are needed most and to understand previously unseen side effects of drugs and treatments. Our approach protects patient confidentiality and allows insights to be derived at significantly lower cost, and a hundred times faster, than any traditional approach.
HSCIC's statement:
The NHS Information Centre (NHS IC) signed an agreement to share pseudonymised Hospital Episodes Statistics data with PA Consulting in November 2011.
This included Hospital Episode Statistics on Admitted Patient Care (1999/00 to Provisional 2011/12), Outpatient (2003/4 to Provisional 2011/12) and A&E (2007/8 to Provisional 2011/12). This agreement lasted to November 2012 and was amended in December 2012 to extend to November 2015.
The agreement obliged PA Consulting to abide by conditions to protect the confidentiality of the data, including restricting the data to a named list of individuals, a prohibition on sharing any information with risk of identifying individuals and a requirement to destroy the data after the agreement end date.
PA Consulting used a product called Google BigQuery to manipulate the datasets provided and the NHS IC was aware of this. The NHS IC had written confirmation from PA Consulting prior to the agreement being signed that no Google staff would be able to access the data; access continued to be restricted to the individuals named in the data sharing agreement.
Surely the people in power should have paid more attention to this before handing the data to a private third party. Those who ignored the problem before it emerged should not be allowed near position of decision for public office.
Rant apart, this seriously raise the question of do we not need to have an alternative (maybe a pan European subsidized, not for profit organization) to Skynet (sorry I meant Google)?
Given it is now run by the ex head of Barclays Bank and is the second largest recipient of government contracts in the UK, I wouldn't hold your breath.
If they have permission of government officials then what?
We can hold companies accountable but how do you hold government accountable? In a meaningful way? Certainly we can find a myriad of excuses not to fire an government worker for a mistake I am fine with doing the same for this as well.
The key is to learn from it and put into place processes that stop it from reoccurring. We need to weigh the penalties to the harm caused. Frankly, if no one lost their life or livelihood I don't think seeking the outcome you suggest is warranted.
PA Consulting definitely think they were in the right here, they attended a recruitment event at my university and told us about how they did this for the NHS using Google tools. I figured they had permission from the NHS or whatever, and they also seemed to have some relationship with Google. My first thought is that this is just an MP looking for attention, but if the NHS genuinely didn't know then I agree it's surely criminal.
It's hard to know from the article just what happened and what data was uploaded.
But, even though they got approval, they may have committed a criminal offence.
This data initiative is really important. They've got to do something to win back trust. Someone has to lose a job and someone has to go to jail (if a crime was committed).
PA Consulting are idiots and everybody who gave them this contract should be fired.
Saying "I didn't know" is no excuse as this is not the first time PA Consulting have lost data!
"The Home Secretary announced on 10 September that the government has terminated its contract with PA Consulting, following the recent high profile data loss
On 19 August PA Consulting formally notified the Home Office of the loss of a data stick containing sensitive information relating to the JTrack system which PA manage under contract to the Home Office
The data on JTrack relates to prisoners and other offenders in England and Wales."
It transpires that the publicly downloadable data was a mock dataset. How it took them 24 hours to work that out I've no idea, but I imagine there's a lot of headless chicken impressions going on behind the scenes at the moment.
[1] "We apply a strict statistical disclosure control in accordance with the HES protocol, to all published HES data. This suppresses small numbers to stop people identifying themselves and others, to ensure that patient confidentiality is maintained."
Not meaningfully, no. A UK postcode covers 20 households or less. If you have that plus gender and date of birth (as seems to be the case here), you almost always have a unique individual.
I trust Google more than I do "PA Consulting". Which begs the question, how did we get here? Who in their right mind sends out 27 DVDs with probably unencrypted, highly sensitive medical data? Even if the recipient is trustworthy, the transport isn't.
This data needs to be on a locked away government server that answers queries by 3rd party by throwing away half of the data and randomizing the remainder.
Basically. An alternative headline for this story could be "Contractor moves sensitive data from insecure, non-audited medium to secure, audited medium."
Sending things in the post may be the acceptable method of sending sensitive data. I'm guessing that the UK prosecutes people who mess with the post the same way that the US prosecutes people who mess with the mail.
EDIT I'm not sure what I'm talking about. Was the complaint about "27 DVDs" just the amount of data or the method?
I've worked with anonymized patient data in the U.S. at a small consulting firm nobody's ever heard of.
We received encrypted physical media via USPS and registered mail. It may seem byzantine, but we also worked exclusively on air-gapped servers.
Best practice in data security is pretty straightforward - you don't connect data to the internet if a single breach is catastrophic. We talk about things like the Target hack as catastrophic breaches, but they aren't. You can change your password or cancel your credit card. You can't change your medical history - once public, it is always public.
Actually, I wouldn't be surprised if there was a law stating that those DVDs be encrypted... I mean, the key might be in a text file on the DVD, but there are plenty of regs surrounding the transportation of patient data.
I don't understand why the data being on "Google servers" is generating such outrage. Google almost certainly has superior security to this "PA Consulting" or even the government itself.
The report in question (linked in the article) provides "exceptional" evidence for the performance of Cloud technology by comparing a Google BigQuery search against an on-premises SQL Server query.
So, cloud is good because map-reduce performs better than relational databases...
Why does the government repeatedly hire incompetent people?? They pay crazy amounts of money for it too.
I hope there is a very public investigation into this. We are losing privacy every day now and this is one area of our lives that needs to remain private at all costs. There is very little I can see to gain and lots to lose from losing privacy in health. Especially in a public system like the NHS.
I think there's 2 reasons:
Often times the services are contracted out and those contracts are often subject to "lowest bid wins" rules. It is very difficult to judge a quality like competence especially when there's differing interpretations of purchasing legislation involved.
The other reason is that it is often difficult to fire people in the public sector. I believe it to be due to the impact of unions, which will protect even the most egregious offender out of principle. This builds a culture around being hard to get rid of people, even if it is a non-union position the HR process of dismissal is often the same. For example public bodies often have a lot of employee supports (Employee Assistance programs,etc) and before you dismiss anyone you must allow them to avail of these supports.
Good. Imo, the fearmongering here is actually quite irrational. Google have more credibility (and money) to lose from a high publicity hack than government contractors who already act with impunity. If they'd invested in their own own map-reduce deployment we'd only be hearing another story about government contractors wasted millions of £ in taxpayers money on Big Brother data analysis.
> The extracted information will contain a person's NHS number, date of birth, postcode, ethnicity and gender.
Big woop? Your NHS# isn't used outside of the NHS or for anything of concern to most people, and your postcode (and address) is held on the unedited electoral roll by hundreds of organisations. Most people don't even opt-out of the edited register accessible for a small fee on 192.com
Why aren't us Brits worried about our credit histories and county court judgements being recorded and held by Equifax, an American company?
What specifically are people actually afraid of with regard to this data set sitting on Googles servers? I just don't get the regular public outcry about NHS data.
The shame of all of this noise is that resources going into medical research today ends up getting spent on data security and building expensive, custom solutions that avoid using servers of a certain type or location in the name of privacy.
Sure, it would be more secure to conduct medical research without using computers at all, but what about all those people dying of nasty diseases? If I had 6 months to live, I probably wouldn't mind these "criminals" trying to find me a cure.
Instead, we have a deafening din of screaming about data privacy and little or no mention of the benefits of the medical research itself. If people could calm down a little bit about Big Brother, these guys could spend more time doing their jobs, helping sick people.
Government privacy breaches are one of the things I despise most about current Western society. I am - day in, day out - one of those guys calling for ministerial blood.
However I have long thought that proper open access to health data could be as revolutionary as, say, antibiotics. The government can do whatever the hell they like with my data - on the condition that anyone else can too.
Can you imagine what insights could be gained with canonical graph schemas for individual (but (pseudo|a)nonymised) health records and a bit of statistics/ML? I think it would change the world, but it will never happen unless people like us get our hands on it. No amount of management consultants will innovate on the same scale as the tech community; saving lives through Github and AWS sounds like the only thing I'd do with my weekend.
On a side note, I think the same argument could be applied to a great many public services. I recently emailed my doctor a letter from one of my private doctors in PDF format for addition to my records. Can you guess what happened next? Yep, he printed it out and gave it to a secretary to scan it back in, because the Java app that manages this stuff has very tightly controlled boundaries. Shortly after that I overstayed on a trip to a different part of the UK and desperately needed a top-up of my meds. The solution? Print a prescription and mail it to the pharmacy by next-day post, because Scottish NHS and English NHS computers are incapable of communicating with each other. How long would it be after going open source before all this BS is obliterated? I'm thinking months.
Leaving a USB stick on a train is one thing. Spending weeks to upload 27 DVDs' worth of confidential health data to Google servers is quite another! One is negligent. The other is, imho, criminal.
I've very carefully opted out of every single program that the NHS has created for digital records going back years.
Whether that has done any good I have no idea but I do have signed letters from all relevant organisations (Doctors surgery) saying that I've opted out.
Don't the British have something like HIPAA in the US? If so PA Consulting would have had to follow those rules when using Google's infrastructure. Google's infrastructure passes many security levels and has just about every security certification (up to but _not_ including ITAR). There's nothing inherently insecure about doing this as long as they follow the rules. What are the rules about this over there?
A second scandal is now emerging out of this, as digital mapping firm Earthware are accused of posting HES data in Google maps form on its website for all to see.
Earthware's statement claims that they used mock data
HES Data Map Statement 3 March 2014 18:55 GMT. Earthware was contacted this morning by the HSCIC regarding a demo online map we had created to demonstrate how HES data might be displayed in a mapping environment.Earthware immediately withdrew this map from our website upon request from the HSCIC. Earthware would like to clarify the following: The map displayed mock data held by a third party who provided this data to Earthware via a web API. We do not hold nor have we ever held HES data on our servers. No patient identifiable data was ever displayed on the map. Earthware are confident that we have not breached any legal or regulatory rules regarding the licencing or publication of HES data. We will continue to co-operate fully with the HSCIC if required. http://www.earthware.co.uk/
Interestingly enough, the company behind this cluster#### has previous and proven record of similar behaviour.[1][2] Sure, it takes conscious effort to upload multiple DVD's worth of data, which already rules out accidents - but because this is not an isolated incident, I wouldn't rule out corporate policy of willful neglect either.
[+] [-] swombat|12 years ago|reply
I would strongly support throwing anyone involved in this into jail for a long time as a deterrent against future criminals.
This is just unbelievable.
[+] [-] ataggart|12 years ago|reply
[+] [-] ZoFreX|12 years ago|reply
> The alternative was to upload it to the cloud using tools such as Google Storage and use BigQuery to extract data from it. As PA has an existing relationship with Google, we pursued this route (with appropriate approval). This showed that it is possible to get even sensitive data in the cloud and apply proper safeguards.
[+] [-] crb|12 years ago|reply
PA purchased the commercially available Hospital Episode Statistics data set from the NHS Information Centre (now the Health and Social Care Information Centre). The data set does not contain information linked to specific individuals. The information is held securely in the cloud in accordance with conditions specified and approved by HSCIC.
This new approach to analytics can help the NHS improve patient care. We have been able to identify where services are needed most and to understand previously unseen side effects of drugs and treatments. Our approach protects patient confidentiality and allows insights to be derived at significantly lower cost, and a hundred times faster, than any traditional approach.
HSCIC's statement:
The NHS Information Centre (NHS IC) signed an agreement to share pseudonymised Hospital Episodes Statistics data with PA Consulting in November 2011.
This included Hospital Episode Statistics on Admitted Patient Care (1999/00 to Provisional 2011/12), Outpatient (2003/4 to Provisional 2011/12) and A&E (2007/8 to Provisional 2011/12). This agreement lasted to November 2012 and was amended in December 2012 to extend to November 2015.
The agreement obliged PA Consulting to abide by conditions to protect the confidentiality of the data, including restricting the data to a named list of individuals, a prohibition on sharing any information with risk of identifying individuals and a requirement to destroy the data after the agreement end date.
PA Consulting used a product called Google BigQuery to manipulate the datasets provided and the NHS IC was aware of this. The NHS IC had written confirmation from PA Consulting prior to the agreement being signed that no Google staff would be able to access the data; access continued to be restricted to the individuals named in the data sharing agreement.
http://www.paconsulting.com/introducing-pas-media-site/relea...
http://www.hscic.gov.uk/article/3948/Statement-Use-of-data-b...
[+] [-] stuaxo|12 years ago|reply
https://ico.org.uk/Global/contact_us
If anybody has details about how this may be a breach of the data protection act exactly, then please post below.
[+] [-] touristtam|12 years ago|reply
Rant apart, this seriously raise the question of do we not need to have an alternative (maybe a pan European subsidized, not for profit organization) to Skynet (sorry I meant Google)?
[+] [-] lotsofmangos|12 years ago|reply
[+] [-] Shivetya|12 years ago|reply
We can hold companies accountable but how do you hold government accountable? In a meaningful way? Certainly we can find a myriad of excuses not to fire an government worker for a mistake I am fine with doing the same for this as well.
The key is to learn from it and put into place processes that stop it from reoccurring. We need to weigh the penalties to the harm caused. Frankly, if no one lost their life or livelihood I don't think seeking the outcome you suggest is warranted.
[+] [-] mcintyre1994|12 years ago|reply
[+] [-] DanBC|12 years ago|reply
But, even though they got approval, they may have committed a criminal offence.
This data initiative is really important. They've got to do something to win back trust. Someone has to lose a job and someone has to go to jail (if a crime was committed).
[+] [-] dhoulb|12 years ago|reply
[+] [-] yapcguy|12 years ago|reply
Saying "I didn't know" is no excuse as this is not the first time PA Consulting have lost data!
"The Home Secretary announced on 10 September that the government has terminated its contract with PA Consulting, following the recent high profile data loss
On 19 August PA Consulting formally notified the Home Office of the loss of a data stick containing sensitive information relating to the JTrack system which PA manage under contract to the Home Office
The data on JTrack relates to prisoners and other offenders in England and Wales."
http://www.scl.org/site.aspx?i=ne9297
[+] [-] higherpurpose|12 years ago|reply
No Mr. Obama, neither NSA keeping the data nor 3rd parties is the solution. The solution is to stop spying on everyone.
[+] [-] pja|12 years ago|reply
This is beyond parody.
[+] [-] chillax|12 years ago|reply
"No individuals directly named records online. But a massive breach of the most basic information security policies to prevent jigsaw."
https://twitter.com/bengoldacre/status/440488463008550912
[+] [-] arethuza|12 years ago|reply
[+] [-] pja|12 years ago|reply
https://twitter.com/bengoldacre/status/440576479248662528
It's still the case that real NHS data was uploaded to Google's servers.
[+] [-] lern_too_spel|12 years ago|reply
http://www.earthware.co.uk/default.aspx
[+] [-] Sniperfish|12 years ago|reply
"Wait for details on story i’ve been tweeting today: twitter is first draft. Story is: small number rule breach, I believe, which is bad."
https://twitter.com/bengoldacre
[+] [-] crb|12 years ago|reply
It is anonymised [1], publicly licensable data.
Here are a list of users and uses. [2]
[1] "We apply a strict statistical disclosure control in accordance with the HES protocol, to all published HES data. This suppresses small numbers to stop people identifying themselves and others, to ensure that patient confidentiality is maintained."
[2] http://www.hscic.gov.uk/media/10495/Users-and-uses-of-HES/pd...
[+] [-] SideburnsOfDoom|12 years ago|reply
Not meaningfully, no. A UK postcode covers 20 households or less. If you have that plus gender and date of birth (as seems to be the case here), you almost always have a unique individual.
[+] [-] collyw|12 years ago|reply
So imagine you have one or two people in the country with certain symptoms. How anonymous is that?
[+] [-] revelation|12 years ago|reply
This data needs to be on a locked away government server that answers queries by 3rd party by throwing away half of the data and randomizing the remainder.
[+] [-] fixermark|12 years ago|reply
[+] [-] danielweber|12 years ago|reply
EDIT I'm not sure what I'm talking about. Was the complaint about "27 DVDs" just the amount of data or the method?
[+] [-] dangerlibrary|12 years ago|reply
Best practice in data security is pretty straightforward - you don't connect data to the internet if a single breach is catastrophic. We talk about things like the Target hack as catastrophic breaches, but they aren't. You can change your password or cancel your credit card. You can't change your medical history - once public, it is always public.
[+] [-] Pxtl|12 years ago|reply
[+] [-] morgante|12 years ago|reply
[+] [-] timthorn|12 years ago|reply
So, cloud is good because map-reduce performs better than relational databases...
[+] [-] k-mcgrady|12 years ago|reply
I hope there is a very public investigation into this. We are losing privacy every day now and this is one area of our lives that needs to remain private at all costs. There is very little I can see to gain and lots to lose from losing privacy in health. Especially in a public system like the NHS.
[+] [-] bananas|12 years ago|reply
[+] [-] easyfrag|12 years ago|reply
The other reason is that it is often difficult to fire people in the public sector. I believe it to be due to the impact of unions, which will protect even the most egregious offender out of principle. This builds a culture around being hard to get rid of people, even if it is a non-union position the HR process of dismissal is often the same. For example public bodies often have a lot of employee supports (Employee Assistance programs,etc) and before you dismiss anyone you must allow them to avail of these supports.
[+] [-] cjrp|12 years ago|reply
[+] [-] nly|12 years ago|reply
> The extracted information will contain a person's NHS number, date of birth, postcode, ethnicity and gender.
Big woop? Your NHS# isn't used outside of the NHS or for anything of concern to most people, and your postcode (and address) is held on the unedited electoral roll by hundreds of organisations. Most people don't even opt-out of the edited register accessible for a small fee on 192.com
Why aren't us Brits worried about our credit histories and county court judgements being recorded and held by Equifax, an American company?
What specifically are people actually afraid of with regard to this data set sitting on Googles servers? I just don't get the regular public outcry about NHS data.
[+] [-] adamrneary|12 years ago|reply
Sure, it would be more secure to conduct medical research without using computers at all, but what about all those people dying of nasty diseases? If I had 6 months to live, I probably wouldn't mind these "criminals" trying to find me a cure.
Instead, we have a deafening din of screaming about data privacy and little or no mention of the benefits of the medical research itself. If people could calm down a little bit about Big Brother, these guys could spend more time doing their jobs, helping sick people.
[+] [-] clienthunter|12 years ago|reply
However I have long thought that proper open access to health data could be as revolutionary as, say, antibiotics. The government can do whatever the hell they like with my data - on the condition that anyone else can too.
Can you imagine what insights could be gained with canonical graph schemas for individual (but (pseudo|a)nonymised) health records and a bit of statistics/ML? I think it would change the world, but it will never happen unless people like us get our hands on it. No amount of management consultants will innovate on the same scale as the tech community; saving lives through Github and AWS sounds like the only thing I'd do with my weekend.
On a side note, I think the same argument could be applied to a great many public services. I recently emailed my doctor a letter from one of my private doctors in PDF format for addition to my records. Can you guess what happened next? Yep, he printed it out and gave it to a secretary to scan it back in, because the Java app that manages this stuff has very tightly controlled boundaries. Shortly after that I overstayed on a trip to a different part of the UK and desperately needed a top-up of my meds. The solution? Print a prescription and mail it to the pharmacy by next-day post, because Scottish NHS and English NHS computers are incapable of communicating with each other. How long would it be after going open source before all this BS is obliterated? I'm thinking months.
[+] [-] samwillis|12 years ago|reply
http://news.bbc.co.uk/1/hi/7575989.stm
[+] [-] swombat|12 years ago|reply
[+] [-] shocks|12 years ago|reply
[+] [-] collyw|12 years ago|reply
[+] [-] anu_gupta|12 years ago|reply
[+] [-] noir_lord|12 years ago|reply
Whether that has done any good I have no idea but I do have signed letters from all relevant organisations (Doctors surgery) saying that I've opted out.
This does have legal battle written all over it.
[+] [-] nickbauman|12 years ago|reply
[+] [-] UVB-76|12 years ago|reply
[1] http://www.independent.co.uk/life-style/health-and-families/...
[2] http://www.hscic.gov.uk/article/3947/Statement-Use-of-data-b...
[+] [-] linlea|12 years ago|reply
HES Data Map Statement 3 March 2014 18:55 GMT. Earthware was contacted this morning by the HSCIC regarding a demo online map we had created to demonstrate how HES data might be displayed in a mapping environment.Earthware immediately withdrew this map from our website upon request from the HSCIC. Earthware would like to clarify the following: The map displayed mock data held by a third party who provided this data to Earthware via a web API. We do not hold nor have we ever held HES data on our servers. No patient identifiable data was ever displayed on the map. Earthware are confident that we have not breached any legal or regulatory rules regarding the licencing or publication of HES data. We will continue to co-operate fully with the HSCIC if required. http://www.earthware.co.uk/
[+] [-] bostik|12 years ago|reply
"Fined and fired" is not a sufficient deterrent.
1. http://www.theregister.co.uk/2008/09/11/pa_consulting_home_o...
2: http://www.scl.org/site.aspx?i=ne9297
[+] [-] kabdib|12 years ago|reply
[+] [-] binarymax|12 years ago|reply
[+] [-] unknown|12 years ago|reply
[deleted]
[+] [-] higherpurpose|12 years ago|reply
http://www.techdirt.com/articles/20140207/09552726132/uk-pol...