top | item 30754961

Automating cookie consent and GDPR violation detection

109 points| tomgp | 4 years ago |usenix.org | reply

247 comments

order
[+] M2Ys4U|4 years ago|reply
The GDPR does not require websites to inform users that a website sets cookies. There is nothing in the GDPR about cookies.

It's the ePrivacy Directive[0] that deals with cookies (or, rather, "[storing] information or to gain[ing] access to information stored in the terminal equipment of a subscriber or user"). This is a law that pre-dates the GDPR.

If you can't get that right, frankly I question whether anything you write on the subject is correct.

[0] Directive 2002/58/processing of personal data and the protection of privacy in the electronic communications sector - https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A...

[+] atoav|4 years ago|reply

   (25) However, such devices, for instance so-called "cookies", can be a legitimate and useful tool, for example, in analysing the effectiveness of website design and advertising, and in verifying the identity of users engaged in on-line transactions. Where such devices, for instance cookies, are intended for a legitimate purpose, such as to facilitate the provision of information society services, their use should be allowed on condition that users are provided with clear and precise information in accordance with Directive 95/46/EC about the purposes of cookies or similar devices so as to ensure that users are made aware of information being placed on the terminal equipment they are using. Users should have the opportunity to refuse to have a cookie or similar device stored on their terminal equipment. This is particularly important where users other than the original user have access to the terminal equipment and thereby to any data containing privacy-sensitive information stored on such equipment. Information and the right to refuse may be offered once for the use of various devices to be installed on the user's terminal equipment during the same connection and also covering any further use that may be made of those devices during subsequent connections. The methods for giving information, offering a right to refuse or requesting consent should be made as user-friendly as possible. Access to specific website content may still be made conditional on the well-informed acceptance of a cookie or similar device, if it is used for a legitimate purpose.
The rest of the GDPR makes it extremely clear that the goal of the whole thing is not to mandate some specific solution but to force people who run services to allow tracking only with informed consent and to offer options that do not track.

If you are not storing data on your users machines or just do so for legitimate purposes, you should not have a need to ask for a users consent and thus don't have any need a cookie banner.

The issue here is, that many people running websites just don't know what they are storing and how. Just slapping a cookie banner on that bad boy and calling it a day won't work either, because you have to list the purposes of these cookies. If you don't know why your weird wordpress template loads a cookie, maybe it is time to change it (or alternatively: change your profession).

[+] privacylawthrow|4 years ago|reply
You're wrong. The ePrivacy Directive does require that a website get consent before storing information on the end-user's device. Prior to GDPR, the local country implementations of the ePD allowed for implicit consent in some EU countries, and opt-out consent in other EU countries. GDPR redefined what constitutes legitimate consent to process personal data. Consent that was previously valid under the ePD was no longer valid under GDPR, which is why GDPR is about cookies, and every other processing of personal data.
[+] 1vuio0pswjnm7|4 years ago|reply
"There is nothing in the GDPR about cookies."

(30) Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them.

[+] tinus_hn|4 years ago|reply
It would be ineffective to have a law against cookies because everyone would just switch to localstorage or another persistent storage.
[+] deugtniet|4 years ago|reply
It's pretty well known that cookie-walls are rife with anti-consumer patterns. Going to something like formula1.com requires me to click more than a 100 times to object to the 'legitimate interests' of as many companies. Which is a pretty terrible anti-pattern when I don't want to be tracked at all...

After reading the abstract, it seems the authors try to classify cookies using a special browser extension called "CookieBlock" [1]. I hope they are successful, because I hate being tracked on the internet.

[1]https://github.com/dibollinger/CookieBlock

[+] jjoonathan|4 years ago|reply
Right, as with the cookie laws companies seem to have collectively come to the idea that "they can't catch us all!"

So far they seem to be correct. I would really like to see the courts deal a few black eyes over this, I hope this tool can help.

[+] shadowgovt|4 years ago|reply
I'm not sure what lessons the rest of the world should have taken from the US's "war on drugs" (or, for that matter, the US's prohibition before it).

... but "If you pass the law that outlaws a wildly-popular behavior, most people will stop that behavior" probably wasn't it. Law can bend behavior on the margins. It just encourages rule-breaking when you try to drive it like a spike through the middle.

[+] karaterobot|4 years ago|reply
Handy guide to GDPR for web developers:

* You can't set all your cookies first, then ask permission.

* You can't set all your cookies whether the user accepts them or not.

* You can't tell users to stop using the website if they don't want cookies.

* You can't convince any business owner to follow the above rules.

[+] PragmaticPulp|4 years ago|reply
GDPR is about far more than just cookies.

Once you get into it, the GDPR is extraordinarily vague. It obviously wasn't written by engineers or even people with domain experience. You can easily interpret common server-side logging operations as GDPR violations if you're not careful.

[+] legitster|4 years ago|reply
Part of my job is to maintain GDPR compliance for corporate websites. Even for companies that legitimately want to exceed compliance, you would not believe how much of a pain in the ass it is.

The first company wanted to do it "right". So we enabled opt-out by default for all cookies. Which requires setting an anonymized master cookie to check everytime we load a webpage to see if we are allowed to set other cookies. And since IP-detection was not allowed, we did it for all website visitors. And because we have to remember your settings, we had to create a seperate anonymized database outside of our normal website.

And the website broke ALL THE TIME. Product configurators, shopping carts, forms, downtime detection - all this stuff relied on cookies. And for several months the web team had a constant nightmare of customer complaints about broken stuff.

In the first year we ended up spending close to $250k on legal advice from European lawyers, and most of the advice boiled down to "you're not going to get in trouble if you just do what everyone else is doing". Seriously.

Since then it's gotten better - most third party vendors have done a better job of offering anonymized cookie versions of their products. Or there is just more industry guidance available on what kind of cookies can be considered sufficiently anonymous.

For people who claim GDPR compliance is clear and straightforward - I can't believe they actually have much experience working in Privacy. Actual implementation gets... very opaque. Especially when the law says it's illegal to deny service based on their cookie preference, but some services are literally impossible to provide without a cookie of some form.

[+] bryanrasmussen|4 years ago|reply
> Especially when the law says it's illegal to deny service based on their cookie preference, but some services are literally impossible to provide without a cookie of some form.

To clarify what others are saying here - it is illegal under GDPR to deny service based on people opting out of providing PII in the cases where that PII is not needed for providing the service, not for refusing to accept cookies (although, sure, there can be some relation between these things).

If for example you were providing a service where you sent someone emails on their birthday with autogenerated Love from your AI Momma messages it would not be illegal for you to refuse to provide them access to your service if they opted out of you storing their email and birthday, because those two pieces of PII are needed for the service to work.

That said, most services do not need to store any PII for any length of time to work. Thus if a service says you can't read our medical advice column unless you allow us to store all this stuff we just hoovered up from your browser forever, that would be illegal. Because they don't need any of that stuff to show you the article they already have written and ready to go.

[+] privacylawthrow|4 years ago|reply
I'm a privacy lawyer that has worked on cookie consents for a number of commercial websites. Everything you said here is all too true. The real legal answer in a lot of cases is "Do what everyone else is doing. Don't be an outlier. Use industry tools because if there's a problem with an industry tool, they'll go after the tool and not its users."

The comments about cookies not being part of GDPR are grossly wrong. One of the early discussions in the privacy law community was how to handle the collision of the new consent requirements under GDPR with the fact that the ePrivacy Directive requires consent for cookies. Prior to GDPR, a large number of EU jurisdictions allowed for implicit consent through a variety of actions, like scrolling a page, or non-actions, like seeing a banner and not clicking "no". GDPR redefined consent and that's why cookie banners pop up.

[+] LinAGKar|4 years ago|reply
The GDPR isn't about cookies, it's about personal data. You can still use cookies for functional stuff, like keeping track of the shopping cart on the client.

The problem here is that companies have an ingrained culture of taking the easy route and just grabbing all the data they can without regard to privacy, which now comes back to bite them.

[+] andyjansson|4 years ago|reply
> some services are literally impossible to provide without a cookie of some form.

You seem to be under a misapprehension about what GDPR is about. It is not about cookies, it's about PII.

[+] dmitriid|4 years ago|reply
> Product configurators, shopping carts, forms, downtime detection - all this stuff relied on cookies.

You don't have to ask for consent or permissions for data that is strictly required for the functionality of your website. You're still responsible for keeping PII data safe etc., of course.

basically, you created those problems for yourslef, and now blame the law.

> For people who claim GDPR compliance is clear and straightforward - I can't believe they actually have much experience working in Privacy.

It is very straightforward when it comes to the use cases you described.

> Actual implementation gets... very opaque.

That's your problem, not the law's problem. Ask only for data you strictly need. Keep safe. Do not share with/sell to third parties.

This was true for years before GDPR, and the only reason everyone found it "so hard" is that everyone, including you and your corporate sites, didn't give two craps about users' privacy.

[+] oblio|4 years ago|reply
Growing pains.

Like Neo being unplugged out of the Matrix.

It takes a while to learn to respect privacy when all you knew was information = ads = $$$.

[+] alkonaut|4 years ago|reply
> Product configurators, shopping carts, forms, downtime detection - all this stuff relied on cookies.

Yes? Are you saying that when people reject your cookie consent, you block the cookies that are fundamental to your product? Why would you do that?

[+] tempnow987|4 years ago|reply
Yeah, anyone who says GDPR is "easy" is just lying through their teeth. It really is folks who have not actually had to implement or try to implement anything.

The best is they claim (falsely) that you don't actually have to pop-up the consent dialogs. Not really true on almost any actual website that does anything anyone wants.

[+] elygre|4 years ago|reply
Whenever people go "it's been four years, this law is too complicated", I am reminded that every now and again the US Supreme Court has to deal with issues that relate to the constitution.
[+] akersten|4 years ago|reply
Oh the irony of this site itself having a "we use cookies, got it?" banner while lamenting this exact perceived lack of choice. I always laugh a little when I see those anyway, knowing that my browser's settings and privacy extensions are blocking the cookies and tracking connections either way.

Did we consider that if everyone is breaking the law, the law itself might need a rework?

[+] skaul|4 years ago|reply
Brave has an option to block cookie notices - you need to enable the "Filter obtrusive cookie notices" list in brave://adblock. https://twitter.com/shivan_kaul/status/1488989740690853888

We're experimenting with blocking cookie notices by default in Nightly. There's webcompat risk - some websites just break if you block the cookie notice. "Works on 90% of websites" is just not good enough when deploying to 50 million Web users.

[+] olalonde|4 years ago|reply
Given the amount of confusion and conflicting interpretations of GDPR we get on HN, I'm not really surprised. Then there's always the vocal minority that is fully convinced that GDPR is very simple and clear.
[+] Nextgrid|4 years ago|reply
There's a huge amount of misinformation spread around it, and not to mention existing online information about the earlier and completely stupid "cookie law" is sometimes mistaken for the GDPR.

It doesn't help that the GDPR is only really simple if you don't abuse personal data. It will obviously become very complex when you're hoping to find loopholes do something that the GDPR was fundamentally designed to outlaw, and it just so happens that a large chunk of this site makes their money from this.

[+] spiderfarmer|4 years ago|reply
Isn't every webserver that uses the standard access.log format (thus including IP address) already non-compliant?
[+] layer8|4 years ago|reply
No. You are allowed to keep such logs for a limited time in order to be able to analyze attacks on your web server.
[+] gyulai|4 years ago|reply
This is a very poor default, and I think it's a good thing that the legal environment challenges that default.

It's not automatically non-compliant, of course, but you might have to clear some legal hurdles to make it so.

[+] globalise83|4 years ago|reply
What about a wiki system + workflow tool for documenting all GDPR infringements on every website of interest with auto-submission of a complaint to the regulatory agencies?
[+] tacone|4 years ago|reply
I really think we should reject the law and make another one that requires the browser vendors to provide the appropriate notices (think of what currently happens with non-https connections) and (browser enforced) choices.

No added work for website developers, no lawyers required, no dark patterns. Common icons and warnings the user can recognize easily because they would be the same for every website.

[+] dmitriid|4 years ago|reply
> I really think we should reject the law and make another one that requires the browser vendors

1. GDPR isn't just about browsers

2. Those "consent" popups are mostly illegal under GDPR. They are often provided by companies whose entire business is dark patterns. Thankfully, the EU is going after them, too: https://techcrunch.com/2021/11/05/iab-europe-tcf-gdpr-breach...

[+] tobr|4 years ago|reply
That makes no sense. How is the browser supposed to inform the user what they are consenting to? The point of the law is, among other things, that you need to have informed consent when you process personal information. That’s not a technical problem that you can solve with a new API. It requires organizations to work differently. Unfortunately it seems that very few orgs have been willing to put the necessary thought and care into this, instead they just slap these cargo cult consent dialogs across everything.
[+] endisneigh|4 years ago|reply
Fine them all! Europe will collect billions.
[+] ars|4 years ago|reply
And people will just stop providing service to Europe. It's already started, there are tons of sites that refuse service to Europe.
[+] mariusor|4 years ago|reply
I doubt that very much. A lot of the indieweb sites don't bother collecting information about their users so they don't need to show information pop-ups nor worry about GDPR. I know I don't.
[+] PragmaticPulp|4 years ago|reply
> A lot of the indieweb sites don't bother collecting information about their users so they don't need to show information pop-ups nor worry about GDPR.

Not true.

I've spent far too much time with expensive lawyers going through the painful details of GDPR compliance and edge cases. If you keep logs at all, anywhere, then technically you could be at risk of crossing the GDPR. Don't assume that you're free and clear because you haven't gone out of your way to add any analytics.

[+] shadowgovt|4 years ago|reply
if your site is running on apache with default logging, or a shared host like DreamHost, you are probably not fully in compliance with the letter of the GDPR since you're logging IP addresses and aren't using them for necessary site operations.

... especially if the log just grows and grows and never rotates. The GDPR is a very wide-reaching law.

Of course, there's no real need to worry since, practically speaking, it was intended as a cudgel to beat FAANG with and not a dagger to stab indies with. If you're comfortable with the safety of your operations being "The folks with legal power to enforce won't wield it on you", you have nothing to worry about.

[+] trh0awayman|4 years ago|reply
The cookie consent stuff has always seemed straight forward to me, but maybe I've had it wrong this whole time. It does really say a lot that 95% of websites had a violation. I wish that we could make the GDPR entirely client-side.

Semi-related: my understanding is that it's impossible for American hosting companies to comply with GDPR (due to the CLOUD act).

If that's the case, and you're American/using an American host, is there any point in even trying to comply?

[+] Loeffelmann|4 years ago|reply
Isn't there insane money to make just suing everybody in breach of gdpr? I always thought there were laywers scouring the internet in search of a quick buck.
[+] delusional|4 years ago|reply
I don't think you really "sue" anyone for breaching GDPR. I think you report it to the local authorities, and then they pursue a case.

Basically I don't think there's any money for the lawyers to pick up here.

[+] Pungsnigel|4 years ago|reply
Wouldn't that just end up in the hands of whatever government is relevant? I believe the fines you pay for GDPR violations are paid to governments, not users or suers.
[+] FreeHugs|4 years ago|reply
I run a website with a few hundred thousand monthly active users. I get tons of mails from users telling me how much they love it. One unintrusive, smallish Adsense banner pays for everything. For years now, everyone was happy.

Now Google sent me an email that they want me to gather user consent before showing Adsense. They offer an automatic consent modal. But the problem with that one is that it not only displays the consent modal but also injects a smaller widget into the site. It looks like the widget only pops up when the user scrolls down to the bottom of the page. Unfortunately, that also makes it pop up when the page is not longer than the screen. So pages where the content fits on the screen behave really really shitty. Maybe that is the reason why I have never seen it used anywhere.

And of course loading the consent script from Google before getting consent is not in line with GDPR in the first place.

Other consent solutions I see around the web are heavy third party widgets that do a lot of complicated stuff. And because they are third party scripts, they are also not in line with the GDPR.

I have not found any indie developers who have implemented their own consent solution. And as far as I understand it, Google has no communication channel. They just threaten to kick you off Adsense. So all I can do is implement my own solution and wait if it happens or not.

I started to implement my own consent banner now. Not sure if I will get it right so that it pleases Google.

I fear that this whole GDPR thing might be the end of my website.

[+] ElDomingo|4 years ago|reply
Honestly why can't browsers just implement a option in there settings? Let the users decide in one place if the want to consent to extra none essential cookies. And add a extra field to exclude certain sites in case you have a domain that you want to grant permission.
[+] ______-_-______|4 years ago|reply
You mean the "do not track" header? Advertisers won't respect any setting that makes it that easy to opt out.
[+] bjt2n3904|4 years ago|reply
That's the end result of extremely complicated legislation. Everyone breaks it, but you only get caught if you stick out enough.

Uncharitably, it's a way for the government to arbitrarily prosecute anyone they please.

[+] tschellenbach|4 years ago|reply
Government regulation that outsources/hides the cost on consumers and businesses needs additional scrutiny. Did anyone analyze the full cost of these regulations? It must be insanely high.
[+] zelphirkalt|4 years ago|reply
If those businesses had thought of actual consent to their practices before and had acted accordingly, they would not sit on a mountain of tech debt now and their costs of becoming conform with GDPR would be minimal.