top | item 24905920

Twitter preconnects to the wrong domains

221 points| weinzierl | 5 years ago |ctrl.blog

149 comments

order
[+] matsemann|5 years ago|reply
Twitter is so full of weird connectivity bugs.

For the last weeks until a few days ago I've been unable to access twitter unless doing a hard refresh on every click. Because they installed a faulty service worker of some kind that would break most of the requests. (Both on desktop and android)

And on Android, every time I follow a link to Twitter in an app that opens in a web view, it gives me a faulty page that I have to refresh a few times before I'm able to view it. It loads the page fine, but some rest call or whatever to fetch the tweet crashes.

Edit: Not heard many others complain about this before, so mainly thought it was something about my setup. But the huge amount of upvotes this got suggests some Twitter engineers better look into this..

[+] the_only_law|5 years ago|reply
> And on Android, every time I follow a link to Twitter in an app that opens in a web view, it gives me a faulty page that I have to refresh a few times before I'm able to view it. It loads the page fine, but some rest call or whatever to fetch the tweet crashes.

I get this issue too, I figured it was part of some trick to make me want to use the app.

[+] jerf|5 years ago|reply
Thank you for mentioning this. I've been getting frustrated with my Firefox install for a while because everything is so slow and it eats so much memory. After deleting the 200 service workers [1] I didn't even realize were installed, it is taking wildly less memory and seems generally snappier. I wouldn't call it "night and day", but it's faster.

I had been under the impression that service workers required approval from the user to be installed. I had service workers from websites that I haven't visited in years and don't even exist anymore, sitting there chewing on my RAM every time I started Firefox.

Also Twitter has also been broken for me this way for months.

Service workers have some nice features, but I submit that they need to be something to be whitelisted by users, not something websites can just toss in there whenever they feel like it. The odds of one of them screwing up and eating far more resources than they have any right to approach 1 as the number of them increases. I don't know exactly how they managed to collectively eat 2GB of RAM and I don't care; there is no way they were bringing that much value to me, especially as I permit no website to simply push me notifications.

[1]: A suspiciously round number; it was exactly 200. Is that a limit? I don't see it in about:config.

[+] raxxorrax|5 years ago|reply
So it wasn't just my browser or my internet connection. I used CTRL + F5 in FF and it always worked, but on a normal refresh it would either display an error or the page didn't load at all.

Thought that my IP had landed on some list where Twitter degraded the connection and I wasn't even logged in.

[+] saagarjha|5 years ago|reply
In Safari it will just drop you to an error page 70% of the time you navigate to it through a link, I assume because it doesn’t get its tasty tracking data. You have to reload the page to get anything to show up.
[+] Macha|5 years ago|reply
This has been my experience on mobile twitter for years, and in the last year on the desktop one as well. I thought it was a passive aggressive push for users to log in or install the official apps.
[+] oauea|5 years ago|reply
This has been happening for at least a year for me in Firefox. 100% of the time. It's rather odd.
[+] userbinator|5 years ago|reply
I don't use Twitter except when I'm linked to it and for occasional searches, and it's been a huge step backwards with the new UI that was forced on everyone a few months ago. It even has a loading screen because it's much slower, and somehow also manages to show less content than the previous design.

If I remember correctly there was a brief period a while ago when they did use a JS-only "web app", but then switched back to simple HTML with JS enhancements (and I seem to remember they announced that change with much pride).

Of course, there is always mobile.twitter.com that is still static-only and quite usable without JS, but IMHO Twitter is the perfect example of how the "modern web" is doing less with more.

[+] miffe|5 years ago|reply
Go to about:serviceworkers and ctrl-f twitter and remove every instance. That will fix it.
[+] abstractbarista|5 years ago|reply
Wow, I guess this is common. This has happened to me for months using Chrome on Ubuntu with UBlock & Privacy Badger. I always assumed it was the extensions but I guess not.

Twitter is literally so buggy. Most of the time I have to refresh a page several times. Maybe it's because I don't have an account and refuse to use their apps?

[+] Vvector|5 years ago|reply
I've seen similar issues. Who knew it was so hard to display 280 characters?
[+] djhworld|5 years ago|reply
I've been experiencing something similar, every time I click on a twitter link from outside of Twitter, Firefox errors with "There has been a protocol violation" error and a hard refresh fixes it.

I'm not sure whether it's because I've setup FF containers to open all twitter.com links in a separate container.

[+] httpsterio|5 years ago|reply
Service workers are hard to get right and if they've for example updated the service worker, it might be damn near impossible to unregister the old one and installing the newer one before the old worker is set to be cleared from the cache.
[+] wazoox|5 years ago|reply
I switched to using nitter.net for all links to twitter. At least this trick works.
[+] imperialdrive|5 years ago|reply
No freakin way... this whole month I thought twitter was broken... I figured it was some censorship related filter. Every visit (for me at least) requires a reload to work properly.
[+] slickrick216|5 years ago|reply
I’ve noticed something weird moving between hacker news and Twitter safari on my iPhone 7. If I first visit Twitter then in the same tab go to hacker news. The favicon is the Twitter one but the tab says hacker news. This doesn’t work with other sites like google that I tested. It’s quite odd. It’s like the favicon for Twitter overrides the hacker news one for the first page load
[+] Semaphor|5 years ago|reply
Oh, so it’s not my browser being weird, thanks :) I have to refresh every tweet I load unless it’s opened within a few seconds of the first.
[+] Kye|5 years ago|reply
I see this a lot in Safari on iOS. I don't think it's happened in Firefox yet. Which is weird because it uses Safari on iOS.
[+] floatingatoll|5 years ago|reply
It's been like that for me for the past full year on iOS Safari. Surprised to hear it's reached Android!
[+] Machado117|5 years ago|reply
This also happens to me. I just assumed that it was a dark pattern because I don’t have an account.
[+] TimSchumann|5 years ago|reply
Been having this exact same experience for weeks, maybe months. Desktop on two OS’s and mobile.
[+] curiousllama|5 years ago|reply
There are some websites that I just accept only work 60-70% of the time for me; Twitter is one
[+] driverdan|5 years ago|reply
I have the same issue with Firefox on MacOS. 3rd party cookies disabled, privacy protection on, uBlock Origin, and PiHole. I assume it's a poorly handled call to a tracking or ad domain that fails, blowing it up.
[+] GhostVII|5 years ago|reply
Oh I assumed that was intentional, to make you sign up for an account or something. I get it almost every time, opening in incogneto helps.
[+] dragonsngoblins|5 years ago|reply
The issue on Android also happens for me. I honestly assumed it was a deliberate dark pattern to encourage installing the app
[+] AlwaysRock|5 years ago|reply
I get the same thing on my android. I always wonder how an issue like that can go on for years with no fix.
[+] nukemandan|5 years ago|reply
This, for me, was due to an expired cookie. You need to clear the cookies and local storage and log ba in to fix.

Not ideal

[+] chowned|5 years ago|reply
I have to refresh the page once or twice when attempting to load a tweet in Safari or Chrome on iOS, otherwise I usually get a cryptic "this tweet is not available to you" message or something similar. I also just assumed it was a dark pattern to get me to use the app.
[+] saagarjha|5 years ago|reply
> It strips out the www. prefix to make a ”display version“ of the URL. I have no problem with this, as the prefix is entirely meaningless to humans. It does serve important technical functions, however.

I know most people don’t know the difference, and it would generally be a bad idea to have your www not redirect to the bare domain (or vice versa), but personally I prefer when we don’t hide these things. Just a bit of pedantic correctness, I guess.

> I can’t look at older versions of Twitter, as its pages don’t work well in the Internet Archive’s Wayback Machine.

Now this really gets to me.

[+] chrisweekly|5 years ago|reply
All of this gets to me. A www subdomain is _not_ necessarily equivalent and interchangeable with the apex domain; treating them as equivalent is presumptuous in the extreme.
[+] megous|5 years ago|reply
So to reveal someone's IP address do I just send a DM with URL to my server and log connections? Is it that easy? No clicking needed?
[+] AstralStorm|5 years ago|reply
It also bypasses browser preloading settings and messes with people who have limited bandwidth or transfer quota. It should be optional.
[+] dtech|5 years ago|reply
You'd need to use a unique domain for it though.

That might be why they don't add the subdomain, because adding a unique subdomain to track a user. is free and a domain isn't.

[+] chinathrow|5 years ago|reply
I think at least a link hover is required.

"Technically, it only preconnects when you hover over the link."

[+] lgats|5 years ago|reply
Is there any easy way to monitor preconnect / dns prefetch and others [in chrome / firefox] ?
[+] arusahni|5 years ago|reply
> Twitter redirects links through its t.co link-shortening service. It was once a useful addition to its service as it helped people stay underneath the strict character limits. The link shortener reduced all links to 23 characters. Twitter gains some more data-insights about its users in the form of click-stream data and insight into popular links.

The t.co link also helps them block URLs that they deem problematic on their platform - in the event of spam, attack, or abuse, the redirect can instead be a black hole.

[+] encom|5 years ago|reply
They don't need `t.co` to censor messages. `t.co` was never anything but click tracking, since the character limit is completely arbitrary.
[+] pbhjpbhj|5 years ago|reply
Well I learned something that's not in the article.

I thought browsers requested a domain lookup (gethostbyname()) and basically got back a "zone file" which would have the cnames in. So, I was confused, when people complained about Twitter forcing a domain lookup "on the wrong domain" as I was assuming this would at least cache the domain lookup: It's the right domain, of course, but the lookup for an address on a subdomain includes the subdomain and then gets the cname directing wherever.

It always confused me that dig/nslookup didn't seem to provide all the info. They can, using nslookup 'ls example.com' or 'dig example.com -t AXFR' but the server in general refuses to serve the zone file (seemingly for security by obscurity reasons).

So, for example, if the browser looks up example.com it doesn't get that there is a cname from www->example.com . It only gets that relationship from looking up "www.example.com".

So, TIL, and now results provided by dig/nslookup on the command line make more sense!

[+] d2wa|5 years ago|reply
Almost everyone uses a “recursive DNS resolver“ provided by their ISP, or one of the big ones from Google/Cloudflare/Cisco. The recursive resolver does all the hard work of resolving the root, top-leevl-domain, the apex domain, the subdomain, any CNAMEs, and finally find the right IP addresses to respond to the DNS client. Recursive resolvers benefit a lot from caching responses at each stage of the chain, the same way your browser/OS/router DNS client benefit from caching the final responses from the recursive resolvers. If you run a full DNS resolver, you have do to all of these steps locally.
[+] apples_oranges|5 years ago|reply
How does the preconnect work with the t.co redirect in between? t.co will return a 301 right? Then we see the real domain, then the browser can preconnect to the server, not earlier, or can it?
[+] jzwinck|5 years ago|reply
Twitter run t.co so they know where the redirect goes without actually asking t.co. So they can preconnect to the target domain as well. And that's where they goofed up (I mean, apart from preconnecting to all these domains in the first place).
[+] daodedickinson|5 years ago|reply
So it's not just me that has to reload basically every time I try to view a Twitter page?
[+] outloudvi|5 years ago|reply
Now we have at least three tech giants that don't know www.example.com is not example.com:

* Twitter

* Google (maybe deliberately)

* "The almighty WHATWG" who accepts Google's revision on whatwg/url about this

[+] deepstack|5 years ago|reply
Perfect! Please hire more qualified diligent software developer twitter.
[+] deepstack|5 years ago|reply
I don't know why that gets down voted it really a test to see what kind of dev are on Hacker News now days.
[+] netsharc|5 years ago|reply
> It strips out the www. prefix to make a ”display version“ of the URL. I have no problem with this, as the prefix is entirely meaningless to humans.

Hah, what a dumb comment. Let's just go back to AOL keywords... but we can call it Google keywords.

I propose a new URL scheme: "web:nytimes/some/article/". Sadly I don't work for the Chrome team, so I can't just force it down the web's throat.

[+] hk__2|5 years ago|reply
> I propose a new URL scheme: "web:nytimes/some/article/"

This is already what we have, except that "web" is "http" and we have TLDs to namespace domains.