1. For things like remote work, it only became a trend because of the pandemic. You would have needed to predict the pandemic to predict that management would permit remote work. It was not really a trend, but rather an emergency measure that morphed into a desirable benefit.
2. This only looks at successful things. We must examine unsuccessful things to see if there is any real predictive ability.
As someone who has worked on exclusively remote teams for around 7 years, the pandemic accelerated the trend. It was already very much headed in that direction.
Point 2 notwithstanding, I wonder what's really the threshold of "predicting" for point 1. I've been reading and participating in conversations about remote work on HN for many years before the pandemic; hell, HN is how I discovered this concept in the first place, and what helped me become a remote worker some 3 years before COVID-19 hit the news. REMOTE tag in the monthly "Who is hiring?" / "Who wants to be hired?" threads has also been established before the pandemic.
As someone who has been remote, hybrid remote, and almost every other version distributed teams and remote/asynchronous working over the past 13 years, I assure you there was and still is a remote working trend well before COVID. COVID only poured gasoline on an otherwise very long-term "how we work" trend. COVID probably condensed 10 years of where remote work probably would have ended up in the span of 18 months.
To point 2., This article seems like a great example of selection bias. Hacker news discusses many new ideas in the tech world. I'd also like to see things HN didn't like and see the success rate on those services (the famous Dropbox comment[0] comes to mind)
It's rather short of that of course, but after hearing about an outbreak of flu in China on Canadian television in a hotel lobby on holiday from the UK, very early January 2020, I think the 2nd [*] - I heard approximately nothing about except via HN, which included 'this will be bad/global', until March.
[*] (For a while I remembered, because it was notably earlier than UK newspapers kept claiming as a start date, or 'detected in China', in their charts from March. I didn't claim to know the correct date, but obviously that was an upperbound!)
> It was not really a trend, but rather an emergency measure that morphed into a desirable benefit.
Do you have data to support this? Anecdotally (from own experience and tech workers I know) I feel that remote work has been growing steadily over the past decade, with a an increase in job ads mentioning remote (even if they went out their way to say "office only") but I don't have data either way to back this up. It feels that long-term changes in tech as well as management culture were making remote more common in the long term, and the pandemic accelerated a trend (much like the shift in the movie industry from cinemas to home streaming).
Pandemics have worked as an accelerator or even as a catalyst for technological change since at least the Black Death and this one certainly is no exception.
However, remote work didn't become a trend because of the pandemic, it became the default scenario for many jobs that don't actually require on-site presence but that due to cultural inertia nevertheless often still happened in a colocated office pre-2020.
Remote work has been a trend since long before 2020. With the pandemic the proposition in many cases now simply was "Either work remotely or see your company go out of business.", which had many companies reevaluate their ways rather quickly.
That was not difficult to do, even in Late December. Most discussions were taking it really casually, and believed that none of that would happen, which is exactly why it happened and how I predicted it. People are be filled with unfounded optimism, overestimate their skills, and underestimate their bad lack.
Combining the three with the belief that most humans are incapable of understanding exponential and autoregressive processes, it became clear.
But all of this is anecdotal evidence and could very well be an instance of a bear predicting a depression, after all, bears predicted 10 out of the last 3 economic crises.
Not sure if it is true in general. But from the perspective of the software industry, our company has been remote-first since its inception. We do encourage coming to the offices, but it has never been mandatory. I think there is a fine balance in this. Personal time saved vs. the social aspect. And of course some problems just get solved faster in front of a whiteboard.
This would have been a better analysis if it ignored headlines and used comments, perhaps in combination with some sentiment analysis. Also, nfts, Tesla, etc aren’t really tech trends but rather consumer and valuation trends. They should have tested for specific technologies like programming languages, paradigms, frameworks, and libraries. Or specific hard tech
for 1.. I'd say it was a trend...just a much smaller one.
I've been 100% WFH in tech for 20 years, and it got easier every year even before covid. What you had to predict, wasn't the pandemic but what tech would be hot enough employers wouldn't care where you worked.
I'm pretty sure overlaying Google Trends onto keywords in HN is more of an informal analysis than a rigorous decision process. True predictive value needs a more rigorous model, but these sorts of data explorations are a great way to motivate folks to dig deeper.
3. Hacker News comments have been vehemently anti-cryptocurrency for a decade. I remember being shocked how out of touch with actual hackers this place is.
Feels like a great case study in confirmation bias: you only looked at the topics that ended up being successful. How many tech trends did Hacker News hype up only to have them go nowhere? Without that info we don’t really know what HN’s success rate is.
That was my first instinct but I'm not sure it's actually true.
If my priority is to know about interesting topics so I can be informed on them when they take off then this is the right methodology. HN might be a bit inefficient in time, but this suggests that new tech trends will generally not arrive without having had some advance warning on HN. This is what I need - as a consultant it's useful to know what the client is talking about when they bring up something cutting-edge, and useful to know about cutting edge things to recommend as relevant.
> How many tech trends did Hacker News hype up only to have them go nowhere?
Do all of these graphs taken into account actual "hype" or just the existence, and number, of posts on hacker news? I would like to see the graphs adjusted for discussion. Take number of comments into account, analysis of sentiment in the discussion, etc. Are they actually being hyped?
For instance, the description above the Bitcoin graph gives a link to the very first mention of Bitcoin[1] but the link goes to a post with no comments.
I guess number of posts and timeline of posts could indicate trend but should we call it "hyping" those trends? I would imagine that would come from the averaged response sentiment of the discussion on those posts.
Exactly. Or at last a comparative analysis with similar news feeds. Would reddit tech posts not yield similar results? If not then perhaps one has an edge in term of early discovery future trends.
The variety of tech subjects submitted by hacker newsers is very large, and mostly about the most innovative and financially rewarding domain: tech.
So sure, hacker news does predict every upcoming trend since someone will be posting something about each and every plausibly promising thing.
HN talks about Bitcoin/crypto frequently, but the overwhelming opinion for years has been that it's pointless.
That will certainly be a huge miss, similar to the dismissal of easy file syncing. DeFi/crypto/NFTs seem to be gaining real traction and HN hasn't accepted it.
Not only confirmation bias, but also they're essentially polling a very specific subset of the population.
This is a segment of the population that would be replete with model train enthusiasts, RC plane hobbyists, and HAM radio operators back in the day. So, yeah, things of interest in the tech space would be of even more interest to people on HN.
But yeah, they also picked the "winners", they really have to filter for all topics discussed by HN and then find out how many of those became "future tech" and how many fell by the wayside. Because if it's 1 in 100, it's not a really good predictor.
And if something becomes mainstream, then of course it'll be talked about. That's kind of how it works.
Understanding what will go nowhere would be so much more useful! Following trends can be valuable, but old ways still work. Chasing wild geese is catastrophic.
This! Since people have limited time, if HN leads more more often then not on some tech goose chase, then it doesn't matter if some tech trend is mentioned early on HN and has interest.
But ultimately what makes something successful is investment in that thing in both time and money. HN is very VC heavy tech crowd and of course it should lead in trends where VC tech crowd creates out of pure effort and money.
Telsa and 3D printing is an interesting example. If it isn't strictly a digital good, HN might not be ahead given the audience.
HN is pretty good at identifying trends. I generally read some comment here way before I hear about it somewhere else. Does HN accurately predict where the trend goes? Not necessarily. But if something actually does trend, I will have heard low-level chatter about it on HN for a long time.
Remote work trended because of the pandemic, for example, but there was constant chatter about remote work anytime the topic of employment came up, and the monthly job match-making posts even post whether the job / worker is open to remote. Functional programming has had low-level chatter from the beginning (given that PG viewed it as one of the reasons for the success of his company), and indeed, functional programming has grown more mainstream. I started exploring Python for my Perl-type tasks because it kept getting good mentions on HN.
I don't think any of the above ever got to the point where it got so many mentions it would hit a strong 100 in his methodology compared to Google trends. I'm also not sure you could identify successful trends early on, but I think you hear chatter on HN long before other places.
* Dolphin emulator progress reports (I love it when they show up)
* Random posts on x86-64 emulation or emulating a gameboy or something like that. Fabrice Bellard is a hero.
* People who post their very first side project ever made either via no-code solutions or some React/Bootstrap type of thing. In many cases, these things are job boards for a specific niche.
* HN praising Apple for standing up to the FBI (or CIA?). HN hating Apple because of recent events with scanning pictures on iCloud and all that.
* All the things the blog post talks about
* Hardcore learning resources on many topics, one that I actually used: http://neuralnetworksanddeeplearning.com/ (tons of books on Calculus as well, I remember a 1900 textbook called Calculus Made Easy)
* The random physics or biological discovery
* And so much more
The way I see it: HN talks about tech (and interesting stuff outside tech), it's bound to cover tech that eventually becomes popular. But will the Dolphin progress reports become as popular as Bitcoin? I don't think so. It'd be interesting times if they would ;-)
Another Apple perennial is blog posts about switching away from Apple to Linux/Windows/ChromeOS. This successfully predicted that 300% of Apple's user base is gone now.
This only looked at submissions it seems. I gather a lot more useful information lurking the comments. I think discussions in the comments here are far ahead of the mainstream.
HN is fantastic for surfacing up and coming technologies. Reading HN daily is almost like a superpower.
The HN community is...not so great...at predicting which things are exciting and the industry should adopt. If it were the industry would have settled on a LISP dialect long ago, the web would have adopted good readability standards (based on thousands of meta-complaints on virtually every top rated post on any topic), and mobile app development would still be a growing and profitable business (and Apple would finally be catering to niche HN reader needs).
Hacker News, like any news aggregator, pops up new stuff all the time.
The hardest part is keeping an open mind for jumping on early. Hard because 99% turn out to be dud anyway, not to mention our crowd are skeptical bunch.
My biggest "regret" is ignoring Bitcoin/Ether.
Saw Bitcoin posts when it was $5.. nah it's a scam.
Oh it's gone up to $10.. nah it won't go up further.
Went up further.. nah won't buy bcoz already too expensive.
Ether announced.. lolz nobody going to buy another coin since there's already Bitcoin..
> Remote Work [...] I thought [in contradistinction to the graph] that Hacker News would have been more ahead of the trend on this one.
> For each topic I counted how many times it had been mentioned in a post title
I suspect that's the problem. A lot of trendy discussion happens in comments; I think that's going to be particularly true for a topic like remote work.
People are going to extol its virtues when it's tangentially related to a submission, but especially pre-pandemic there's only so many stories you could submit about it? 'Company goes full-remote'? 'How we handle remote working at Company'?
It'd be interesting to search comments too I think. hn.algolia.com supports it.
I find that when Hacker News really likes a piece of highly technical SaaS technology it's usually pretty good. The best example I can think of is Datadog which got rave reviews here years ago and has continued to be enormously successful. Particularly for enterprise tech as there tends to be less herd behavior up-market. If I'm diligencing a technology I'll often actually look at HN to see the opinions.
The mood on Hacker News (and reddit/twitter) has become too pessimistic to be a good predictor of the future.
The long-term trend is that the world is becoming a better place. Pessimistic people are biased against that fundamental trend that other trends will develop on. Groups of pessimistic people will not be good predictors of the future.
During the infamous leap second fiasco of Java in 2012 which brought down all JVM servers by saturating the CPU, I saved the day because I was reading about it here on HN while my colleagues were struggling to undestrand why all our environments stopped working.
It took me a few minutes to realize that we were indeed hit by what I just read, look at ten of my colleagues each closely inspecting logs at different ssh sessions, and gleefully give them a precise description of the problem, an approach to verify it and a solution, adding “it’s all over the news guys, you should keep yourself updated”.
One thing worth nothing is that HN didn't launch publicly until 2007 February. There is some older content in the DB, but it looks mostly like very low volume manual testing. So the data set definitely shouldn't be extended that far back, and is affecting a few of the be graphs.
The author didn't control for what was on the front page, how many points the article received, or the amount of discussion on the topic. Not to mention the myriad of technologies that are on Hacker News that fizzle.
They missed crypto and NFTs. For some reason HN doesn't appreciate the potential of these tech. Even when some prominent Silicon Valley investors do. I wouldn't have expected this from a technical and freedom minded audience.
I'm not aware of HN doing any predicting at large. If you mean 'will the average person later act like a given segment of HN people do now?' - then it depends on how good you are at selecting your segment, which makes it no better or worse than using any other sample of people for prediction.
Less abstractly, productisation is what takes things from obscurity (e.g. HN) to mainstream audiences, so future trends might grow out of stuff that you see here, but the people who can figure out whether certain tech can become a product (would-be predictors) -hopefully- also have the means to turn it into a product themselves.
Two things I learned on HN before they became mainstream:
* Rust
* Covid
For the latter I remember thinking that it was going to become a big deal soon but no one seemed to be paying attention except for those paranoid HNers.
"My conclusion: Hacker News is typically ahead of the mainstream, often by a few years, but you would need to be paying very close attention to catch the early mentions of a new tech trend. Most of the linked posts and comments have very few upvotes and probably wouldn't even make it to the front page."
[+] [-] MattGaiser|4 years ago|reply
2. This only looks at successful things. We must examine unsuccessful things to see if there is any real predictive ability.
[+] [-] em500|4 years ago|reply
[+] [-] devin|4 years ago|reply
[+] [-] TeMPOraL|4 years ago|reply
[+] [-] speby|4 years ago|reply
[+] [-] rPlayer6554|4 years ago|reply
[0] https://news.ycombinator.com/item?id=9224
[+] [-] OJFord|4 years ago|reply
It's rather short of that of course, but after hearing about an outbreak of flu in China on Canadian television in a hotel lobby on holiday from the UK, very early January 2020, I think the 2nd [*] - I heard approximately nothing about except via HN, which included 'this will be bad/global', until March.
[*] (For a while I remembered, because it was notably earlier than UK newspapers kept claiming as a start date, or 'detected in China', in their charts from March. I didn't claim to know the correct date, but obviously that was an upperbound!)
[+] [-] danjac|4 years ago|reply
Do you have data to support this? Anecdotally (from own experience and tech workers I know) I feel that remote work has been growing steadily over the past decade, with a an increase in job ads mentioning remote (even if they went out their way to say "office only") but I don't have data either way to back this up. It feels that long-term changes in tech as well as management culture were making remote more common in the long term, and the pandemic accelerated a trend (much like the shift in the movie industry from cinemas to home streaming).
[+] [-] Naac|4 years ago|reply
For example, the infamous Dropbox comment: https://news.ycombinator.com/item?id=9224
[+] [-] BjoernKW|4 years ago|reply
However, remote work didn't become a trend because of the pandemic, it became the default scenario for many jobs that don't actually require on-site presence but that due to cultural inertia nevertheless often still happened in a colocated office pre-2020.
Remote work has been a trend since long before 2020. With the pandemic the proposition in many cases now simply was "Either work remotely or see your company go out of business.", which had many companies reevaluate their ways rather quickly.
[+] [-] PartiallyTyped|4 years ago|reply
That was not difficult to do, even in Late December. Most discussions were taking it really casually, and believed that none of that would happen, which is exactly why it happened and how I predicted it. People are be filled with unfounded optimism, overestimate their skills, and underestimate their bad lack.
Combining the three with the belief that most humans are incapable of understanding exponential and autoregressive processes, it became clear.
But all of this is anecdotal evidence and could very well be an instance of a bear predicting a depression, after all, bears predicted 10 out of the last 3 economic crises.
[+] [-] eerikkivistik|4 years ago|reply
[+] [-] Rd6n6|4 years ago|reply
[+] [-] mech422|4 years ago|reply
I've been 100% WFH in tech for 20 years, and it got easier every year even before covid. What you had to predict, wasn't the pandemic but what tech would be hot enough employers wouldn't care where you worked.
[+] [-] Karrot_Kream|4 years ago|reply
[+] [-] rkagerer|4 years ago|reply
I think it was more of a catalyst.
[+] [-] luffapi|4 years ago|reply
Edit: for those downvoting, here’s the data:
https://slidemodel.com/remote-work-key-trends/
[+] [-] staplers|4 years ago|reply
Reminds me of punks who'd shop at Hot Topic.
[+] [-] afavour|4 years ago|reply
[+] [-] dredmorbius|4 years ago|reply
[+] [-] rkangel|4 years ago|reply
If my priority is to know about interesting topics so I can be informed on them when they take off then this is the right methodology. HN might be a bit inefficient in time, but this suggests that new tech trends will generally not arrive without having had some advance warning on HN. This is what I need - as a consultant it's useful to know what the client is talking about when they bring up something cutting-edge, and useful to know about cutting edge things to recommend as relevant.
[+] [-] flanbiscuit|4 years ago|reply
Do all of these graphs taken into account actual "hype" or just the existence, and number, of posts on hacker news? I would like to see the graphs adjusted for discussion. Take number of comments into account, analysis of sentiment in the discussion, etc. Are they actually being hyped?
For instance, the description above the Bitcoin graph gives a link to the very first mention of Bitcoin[1] but the link goes to a post with no comments.
I guess number of posts and timeline of posts could indicate trend but should we call it "hyping" those trends? I would imagine that would come from the averaged response sentiment of the discussion on those posts.
1. https://news.ycombinator.com/item?id=463793
[+] [-] hirako2000|4 years ago|reply
The variety of tech subjects submitted by hacker newsers is very large, and mostly about the most innovative and financially rewarding domain: tech.
So sure, hacker news does predict every upcoming trend since someone will be posting something about each and every plausibly promising thing.
[+] [-] rewtraw|4 years ago|reply
HN talks about Bitcoin/crypto frequently, but the overwhelming opinion for years has been that it's pointless.
That will certainly be a huge miss, similar to the dismissal of easy file syncing. DeFi/crypto/NFTs seem to be gaining real traction and HN hasn't accepted it.
[+] [-] bena|4 years ago|reply
This is a segment of the population that would be replete with model train enthusiasts, RC plane hobbyists, and HAM radio operators back in the day. So, yeah, things of interest in the tech space would be of even more interest to people on HN.
But yeah, they also picked the "winners", they really have to filter for all topics discussed by HN and then find out how many of those became "future tech" and how many fell by the wayside. Because if it's 1 in 100, it's not a really good predictor.
And if something becomes mainstream, then of course it'll be talked about. That's kind of how it works.
[+] [-] beardyw|4 years ago|reply
[+] [-] mempko|4 years ago|reply
But ultimately what makes something successful is investment in that thing in both time and money. HN is very VC heavy tech crowd and of course it should lead in trends where VC tech crowd creates out of pure effort and money.
Telsa and 3D printing is an interesting example. If it isn't strictly a digital good, HN might not be ahead given the audience.
[+] [-] prewett|4 years ago|reply
Remote work trended because of the pandemic, for example, but there was constant chatter about remote work anytime the topic of employment came up, and the monthly job match-making posts even post whether the job / worker is open to remote. Functional programming has had low-level chatter from the beginning (given that PG viewed it as one of the reasons for the success of his company), and indeed, functional programming has grown more mainstream. I started exploring Python for my Perl-type tasks because it kept getting good mentions on HN.
I don't think any of the above ever got to the point where it got so many mentions it would hit a strong 100 in his methodology compared to Google trends. I'm also not sure you could identify successful trends early on, but I think you hear chatter on HN long before other places.
[+] [-] mettamage|4 years ago|reply
* Dolphin emulator progress reports (I love it when they show up)
* Random posts on x86-64 emulation or emulating a gameboy or something like that. Fabrice Bellard is a hero.
* People who post their very first side project ever made either via no-code solutions or some React/Bootstrap type of thing. In many cases, these things are job boards for a specific niche.
* HN praising Apple for standing up to the FBI (or CIA?). HN hating Apple because of recent events with scanning pictures on iCloud and all that.
* All the things the blog post talks about
* Hardcore learning resources on many topics, one that I actually used: http://neuralnetworksanddeeplearning.com/ (tons of books on Calculus as well, I remember a 1900 textbook called Calculus Made Easy)
* The random physics or biological discovery
* And so much more
The way I see it: HN talks about tech (and interesting stuff outside tech), it's bound to cover tech that eventually becomes popular. But will the Dolphin progress reports become as popular as Bitcoin? I don't think so. It'd be interesting times if they would ;-)
[+] [-] microtherion|4 years ago|reply
[+] [-] shotta|4 years ago|reply
[+] [-] bane|4 years ago|reply
The HN community is...not so great...at predicting which things are exciting and the industry should adopt. If it were the industry would have settled on a LISP dialect long ago, the web would have adopted good readability standards (based on thousands of meta-complaints on virtually every top rated post on any topic), and mobile app development would still be a growing and profitable business (and Apple would finally be catering to niche HN reader needs).
[+] [-] wiradikusuma|4 years ago|reply
The hardest part is keeping an open mind for jumping on early. Hard because 99% turn out to be dud anyway, not to mention our crowd are skeptical bunch.
My biggest "regret" is ignoring Bitcoin/Ether.
Saw Bitcoin posts when it was $5.. nah it's a scam.
Oh it's gone up to $10.. nah it won't go up further.
Went up further.. nah won't buy bcoz already too expensive.
Ether announced.. lolz nobody going to buy another coin since there's already Bitcoin..
Ether goes up.. WTF.
[+] [-] OJFord|4 years ago|reply
> For each topic I counted how many times it had been mentioned in a post title
I suspect that's the problem. A lot of trendy discussion happens in comments; I think that's going to be particularly true for a topic like remote work.
People are going to extol its virtues when it's tangentially related to a submission, but especially pre-pandemic there's only so many stories you could submit about it? 'Company goes full-remote'? 'How we handle remote working at Company'?
It'd be interesting to search comments too I think. hn.algolia.com supports it.
[+] [-] hackitup7|4 years ago|reply
[+] [-] keewee7|4 years ago|reply
The long-term trend is that the world is becoming a better place. Pessimistic people are biased against that fundamental trend that other trends will develop on. Groups of pessimistic people will not be good predictors of the future.
[+] [-] leugim|4 years ago|reply
And that is okey.
[+] [-] mywittyname|4 years ago|reply
Hacker News is good about surfacing new technology; Hacker News is bad about figuring out how that technology will be leveraged.
[+] [-] davemp|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] New_California|4 years ago|reply
Only recently the SV/HN community is catching up and waking up to cryptocurrencies - and still hesitantly.
[+] [-] yuchi|4 years ago|reply
During the infamous leap second fiasco of Java in 2012 which brought down all JVM servers by saturating the CPU, I saved the day because I was reading about it here on HN while my colleagues were struggling to undestrand why all our environments stopped working.
It took me a few minutes to realize that we were indeed hit by what I just read, look at ten of my colleagues each closely inspecting logs at different ssh sessions, and gleefully give them a precise description of the problem, an approach to verify it and a solution, adding “it’s all over the news guys, you should keep yourself updated”.
[+] [-] jsnell|4 years ago|reply
[+] [-] kkdaemas|4 years ago|reply
[+] [-] asciimov|4 years ago|reply
The author didn't control for what was on the front page, how many points the article received, or the amount of discussion on the topic. Not to mention the myriad of technologies that are on Hacker News that fizzle.
[+] [-] rafale|4 years ago|reply
[+] [-] username91|4 years ago|reply
Less abstractly, productisation is what takes things from obscurity (e.g. HN) to mainstream audiences, so future trends might grow out of stuff that you see here, but the people who can figure out whether certain tech can become a product (would-be predictors) -hopefully- also have the means to turn it into a product themselves.
[+] [-] f154hfds|4 years ago|reply
* Rust
* Covid
For the latter I remember thinking that it was going to become a big deal soon but no one seemed to be paying attention except for those paranoid HNers.
[+] [-] hnbad|4 years ago|reply
That's a long way of saying "no".
[+] [-] AnimalMuppet|4 years ago|reply