Handling cookies is a minefield

[maxwellg]

Cookies are filled with weird gotchas and uncomfortable behavior that works 99.95% of the time. My favorite cookie minefield is cookie shadowing - if you set cookies with the same name but different key properties (domain, path, etc.) you can get multiple near-identical cookies set at once - with no ability for the backend or JS to tell which is which.

Try going to https://example.com/somepath and entering the following into the browser console:

  document.cookie = "foo=a"; 
  document.cookie = "foo=b; domain=.example.com";
  document.cookie = "foo=c; path=/somepath";
  document.cookie

I get

  'foo=c; foo=a; foo=b'

[kibwen]

The article mentions Rust's approach, but note that (unlike the other mentioned languages) Rust doesn't ship any cookie handling facilities in the standard library, so it's actually looking at the behavior of the third-party "cookie" crate (which includes the option to percent-encode as Ruby does): https://docs.rs/cookie/0.18.1/cookie/

[marumari]

Thanks for pointing that out -- I've updated the article and given you credit down at the bottom. Let me know if you'd prefer something other than "kibwen."

[juped]

De facto standardization by snapping up good names early!

[0xbadcafebee]

Did anyone else notice that the HTTP protocol embeds within it ten-thousand different protocols? Browsers and web servers both "add-on" a ton of functionality, which all have specifications and de-facto specifications, and all of it is delivered through the umbrella of basically one generic "HTTP" protocol. You can't have the client specify what version of these ten-thousand non-specifications it is compatible with, and the server can't either. We can't upgrade the "specs" because none of the rest of the clients will understand, and there won't be backwards-compatibility. So we just have this morass of random shit that nobody can agree on and can't fix. And there is no planned obsolescence, so we have to carry forward whatever bad decisions we made in the past.

[Analemma_]

This is also the fault of shit-tastic middleware boxes which block any protocol they don't understand-- because, hey, it's "more secure" to default-fail, right?-- so every new type of application traffic until the end of time has to be tunneled over HTTP if it wants to work over the real Internet.

[tyleo]

Tbh I’ve made peace with this world and I might even enjoy it more than the planned obsolescence one.

[pjc50]

Anarchy is the price to pay for not having a monopoly dictate a nice clean spec which they can force-deprecate whenever they want.

[jeffreyrogers]

About 10 years ago I implemented cookie based sessions for a project I was working on. I had a terrible time debugging why auth was working in Safari but not Chrome (or vice-versa, can't remember). Turned out that one of the browsers just wouldn't set cookies if they didn't have the right format, and I wasn't doing anything particularly weird, it was a difference of '-' vs '_' if I recall correctly.

[hombre_fatal]

IIRC there is (or was?) a difference in case-sensitivity between Safari and Chrome, maybe with the Set-Cookie header? I've run into something before which stopped me from using camelCase as cookie keys.

Can't seem to find the exact issue from googling it.

[gweinberg]

I got the impression that almost as soon as they were introduced people thought the only sensible use of cookies is to set an opaque token so the server can recognize the client when it sees it again, and store everything else server side.

I don;t understand why it's a problem that the client (in principle) can handle values that the server will never send. Just don't send them, and you don;t have to worry about perplexing riddles like "but what would happen if I did?"

[mu53]

Cookies are an antiquated technology. One of the first introduced while the web was still young in the 90s, and they have had a few iterations of bad ideas.

They are the only place to store opaque tokens, so you gotta use them for auth.

[paol]

Cookie header parsing is a shitshow. The "standards" don't represent what actually exists in the wild, each back-end server and/or library and/or framework accepts something different, and browsers do something else yet.

If you are in complete control of front-end and back-end it's not a big problem, but as soon as you have to get different stuff to interoperate it gets very stupid very fast.

[AshleysBrain]

Cookies seem to be a big complicated mess, and meanwhile are almost impossible to change for backwards-compatibility reasons. Is this a case to create a new separate mechanism? For example a NewCookie mechanism could be specified instead, and redesigned from the ground-up to work consistently. It could have all the modern security measures built-in, a stricter specification, proper support for unicode, etc.

[flotwig]

It's funny that you mention NewCookie, there is actually a deprecated Set-Cookie2 header already: https://stackoverflow.com/q/9462180/3474615

[RadiozRadioz]

NewCookie is, roughly, what browser Local Storage is.

At least for some use cases. Of course, it doesn't directly integrate with headers.

[notatoad]

i think the main problem there is that cookies are so intractibly tied up with tracking, any attempt to create better cookies now will get shut down by privacy advocates who simply don't want the whole concept to exist.

we're stuck with cookies because they exist.

[bob1029]

The DOM & URL are the safest places to store client-side state. This doesn't cover all use cases, but it does cover the space of clicking pre-authorized links in emails, etc.

I spend a solid month chasing ghosts around iOS Safari arbitrarily eating cookies from domains controlled by our customers. I've never seen Google/Twitter/Facebook/etc domains lose session state like this.

[cruffle_duffle]

Needs a better name than NewCookie though. Suggestions include SuperCookie, UltraCookie or BetterCookie

Or to be slightly more serious avoid calling it a cookie and call it something else. Too much baggage surrounding the word cookie.

[nox101]

the new thing should be called "cupcakes" or "candies" or "snacks" or "munchies"

[pavel_lishin]

That feels like that XKCD comic about now there being 15 standards.

[solatic]

Author started with throwing the results of JSON.stringify into a cookie, and I was surprised that his issue wasn't just that someone had thrown a semicolon into the JSON that was being stringified.

Most of the headaches around cookies seem to be around people trying to get them to work with arbitrary user input. Don't do that. Stick with fixed-length alphanumeric ASCII strings (the kind you use for auth tokens) and you'll be fine.

[deathanatos]

That is a bit of a minefield, I agree…

The way around this, as a developer, is URL-safe-base64 encode the value. Then you have a bytes primitive & you can use whatever inner representation your heart desires. But the article does also note that you're not 100% in control, either. (Nor should you be, it is a user agent, after all.)

I do wish more UAs opted for "obey the standard" over "bytes and an prayer on the wire". Those 400 responses in the screenshots … they're a conforming response. This would have been better if headers had been either UTF-8 from the start (but there are causality problems with that) or ASCII and then permitted to be UTF-8 later (but that could still cause issues since you're making values that were illegal, legal).

[johnp_]

> URL-safe-base64

And make sure to specify what exactly you mean by that. base64url-encoding is incompatible with base64+urlencoding in ~3% of cases, which is easily missed during development, but will surely happen in production.

[jeffrallen]

The article mocks Postel's law, but if the setter of the cookie had been conservative in what they sent, there would have been no need for the article...

[Sohcahtoa82]

> The article mocks Postel's law

As they should. Postel's Law was a terrible idea and has created minefields all over the place.

Sometimes, those mines aren't just bugs, but create gaping security holes.

If your client is sending data that doesn't conform to spec, you have a bug, and you need to fix it. It should never be up to the server to figure out what you meant and accept it.

[marcosdumay]

The problem with Postel's law is exactly that the sender is never conservative, and will tend to use any detail that most receivers accept.

[IgorPartola]

Cookies need to die. Their only legitimate use is with for which we have the Authentication header. Having a standard way to authenticate into a website in a browser would be amazing, just too bad that Basic and Digest auth wasn’t good enough at the time.

As a bonus we could get Persona-style passwordless future.

[j16sdiz]

How about user preference without logging in? Are you suggesting create a trillion throwaway accounts?

[wdr1]

Why are first-party cookies bad?

[burntcaramel]

Re Safari’s networking code being closed source, a good substitute might be the Swift port of Foundation. You can see checks for control and delete characters here: https://github.com/swiftlang/swift-corelibs-foundation/blob/...

[AlienRobot]

>everything behaves differently, and it's a miracle that [it] work at all.

The web in a nutshell.

[jerf]

And the article isn't even about the proliferation of attributes cookies have, that browsers honor, and in some cases are just mandatory. I was trying to explain SameSite to a coworker, and scrolled down a bit... https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#co... wait, cookie prefixes? What the heck are those? The draft appears to date to 2016, but I've been trying to write secure cookie code for longer than that, hadn't heard of it until recently, and I can't really find when they went in to browsers (because there's a lot more drafts than there are implemented drafts and the date doesn't mean much necessarily), replies explaining that welcome.

Seems like every time I look at cookies they've grown a new wrinkle. They're just a nightmare to keep up with.

[marcosdumay]

Well, prefixes are opt-in. You don't have to keep-up with them.

The only recent large problem with cookies were to changes to avoid CSRF, those were opt-out, but they were also extremely overdue.

All of the web standards are always gaining new random features. You don't have to keep-up with most of them. They do look like bad abstractions, but maybe it's just the problem that is hard.

[minitech]

> https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#co... wait, cookie prefixes? What the heck are those?

https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#co...

> For more information about cookie prefixes and the current state of browser support, see the Prefixes section of the Set-Cookie reference article.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se...

(Cookie prefixes have been widely supported since 2016 and more or less globally supported since 2019.)

They’re backwards-compatible, so if your cookie need meets the requirements for the `__Host-` prefix, you should use `__Host-`.

[0x073]

IT IS a mess, but I never saw json inside a cookie. For json I use local storage or indexeddb.

[robgibbons]

In both cases (cookie vs localStorage) you're really just storing your data as a string value, not truly a JSON object, so whether you use a cookie or localStorage is more dependent on the use case.

If you only ever need the stored data on the client, localStorage is your pick. If you need to pass it back to the server with each request, cookies.

[lambdaone]

You're really going to hate it when you learn about JSON Web Tokens, which exist exactly to hack past this sort of problem.

[hinkley]

Good way to hit max header length issues. Ask me how I know.

[ricardo81]

Are they ubiquitous? I'm no client side guru, I know I could look at makeuseof etc, but why not ask some professionals instead.

[trevor-e]

I came across a similar issue when experimenting with the Crystal language. I thought it would be fun to build a simple web scraper to test it out, only to find the default HTTP client fails to parse many cookies set by the response and aborts.

[jmull]

> ...tragedy of following Postel's Law.

The "law" is: "Be liberal in what you accept, and conservative in what you send."

But here the problem is caused by being liberal in what is sent while being more conservative in what is accepted. It's using invalid characters in the cookie value, which not everything can handle.

Following Postel's law would have avoided the problem.

[rcxdude]

Postel's law is the main reason why there are so many cases where something is being liberal in what it sends. It's a natural approach when trying to enter into an existing ecosystem, but when the whole ecosystem follows it you get a gigantic ball of slightly different interpretations if the protocol, because something that is non-compliant but happens to work with some portion of the ecosystem won't get discovered until it's already prevalent enough it now needs to be accounted for by everyone, complexifying the 'real' spec and increasing the likelihood someone else messes up what they send.

[hinkley]

    Firefox accepts five characters which RFC recommends that servers not send:

    0x09 (horizontal tab)
    0x20 (spaces)
    0x22 (double quotes)
    0x2C (commas)
    0x5C (backslashes)

I agree with at least some of these. Cookies without commas? Quotes?

[remram]

Quotes in the value when quotes delimit the value? Yeah that seems dangerous to me.

[hinkley]

One of the things I’ve always found frustrating about cookies is that you have to do your own encoding instead of the API doing it for you. I’m sure someone somewhere does but too often I’m doing my own urlencode calls.

[matsemann]

Go and failing to parse http headers correctly should become a meme at some point.

One issue we had was the reverse proxy inserting headers about the origin of the request to the server behind. Like ip, ip city lookup etc. And that parsed through a service written in go that just crashed whenever the city had a Norwegians letter in it, took ages to understand why some of our (luckily only internal) services didn't work for coworkers working from Røros for instance. And that was again not the fault of the Go software, but how the stdlib handled it.