Hacker News “Who is Hiring?” top-level comments over time

[tjwds]

Thanks for hugging my site to death! It's hosted on a $5 DO droplet and I'm honored to have this problem. https://archive.ph/AVbPV

[paxys]

A $5 DO droplet will easily handle peak HN traffic with a bit of optimization. Set cache headers, cache static assets on the server side (or even better, with an external CDN), or even cache the entire page if there's no dynamic content.

[eatonphil]

Do you serve static files? Static file server on a $5 DO droplet should handle the HN front page.

There's also free tiers on many CDNs like Fastly/Cloudflare. Github Pages is also free.

[DeathArrow]

Now you can write another article about how you optimized the site to handle HN traffic. :)

[mountaineer]

I run https://www.hntrends.com on a $5/month box and never had a problem with spikes. Generated, static resources all the way.

[mobilio]

Another link: https://web.archive.org/web/20220608222735/https://blog.joew...

[jasfi]

Then you need a faster web framework or CMS. Nexus is made with Nim and is designed to be fast (https://github.com/jfilby/nexus).

But most problems like that can be solved with caching no matter which tech you're using.

[andrewxdiamond]

For simple static sites like this, sticking a default Cloudflare stack in front helps a lot

[anonimou]

Hey. Nice post.

I was thinking about this same problem for a couple of days. Another question I have is: do we have a higher number of top-level comments on a given number of months or not?

[lamontcg]

So the "pullback" seems similar to the one that was seen in pre-pandemic 2019.

That about correlates to my impression that this is not the recession.

[soheil]

It's a static site, no reason for the server not to be able to handle thousands of connections at once with almost no configuration changes with some like Nginx or Apache. Or even a domain which points directly to a S3 bucket. Hope you're not looking at devop roles on who is hiring posts.

[minimaxir]

A reminder that all Hacker News posts and comments are available on BigQuery and can be queried for free: https://console.cloud.google.com/marketplace/details/y-combi... (the `full` table is up-to-date; ignore the others)

Here's a query for a rough reproduction of what's asked in the title:

    WITH whoishiring_threads AS (
      SELECT id FROM `bigquery-public-data.hacker_news.full`
      WHERE `by` = "whoishiring" 
      AND REGEXP_CONTAINS(title, "Ask HN: Who is hiring?")
    )

    SELECT FORMAT_TIMESTAMP("%Y-%m", `timestamp`) as year_month,
    COUNT(*) as num_toplevel_posts
    FROM `bigquery-public-data.hacker_news.full`
    WHERE parent IN (SELECT id FROM whoishiring_threads)
    GROUP BY 1
    ORDER BY 1

Which results in something like this: https://docs.google.com/spreadsheets/d/13yGlJzFpVzZ-WNHAOsdo...

Still a bit of room to clean up the query, though, and there are some differences from the chart in the post.

[pm90]

I just want to say, in a comment that furthers the discussion in no way, that having something like this available for free is just incredible.

[tedmiston]

I suppose that massive spike is the beginning of the pandemic.

It's interesting to see March 2020 (the highest month of all time) and April 2020 (the lowest month since early 2016) back to back.

[colmanhumphrey]

Hey this is nice! I created a little app here that's quite similar but can be run without knowing SQL: https://slight.run/apps/colman/hacker_news_monthly_top_level..., and a graph to go with it: https://slight.run/graphs/colman/hacker_news_who_is_hiring_c...

One thing that's interesting: this is a sneaky case where the line hides two missing months in 2015.

[psnehanshu]

Google is infamous for breaking backward compatibility, and sunsetting things, so I will not advice anyone to use this to build any products. However it is fine for one-off tasks, like fetching data for this blog post.

[hrpnk]

Thanks for formatting the timestamp, which I missed in the original post.

[tobr]

> all Hacker News posts and comments are available on BigQuery and can be queried for free

Good thing there aren’t any laws that regulate copyright or privacy, so datasets like this can be published without asking users for consent.

[ryandrake]

I started and abandoned a little side project a while ago where I wanted to see which employers tended to have the exact same (or fuzzy-the-same) "Who's Hiring" post month after month. Goal would be to find some signal about which posts were actually resulting in hiring (in other words, the company didn't come back with the same posting) and which ones were just trawling for the same job month after month without actually hiring anyone. Was going to call it "Who's Not Hiring?"

[illwrks]

I know some of the recruiters at my employer, various teams have started testing and hiring multiple vacancies off of a single job ad which is kept open for longer. To an external person it's odd because the assumption is one ad = one vacancy, but company policies and processing speed makes them try odd things to be able to hire what they need.

So while your idea is nice it wouldn't work because not every company lhas the same recruitment process, frontend or backend (visible to the candidate vs workable for the recruitment teams)

[thedufer]

Companies posting every month could just as easily be continuously growing (or, I suppose, have consistent attrition problems). Most of the places I've worked at have had a perpetually-open listings for SWEs, despite regularly hiring them.

[jeffbee]

You know I've wondered that about companies that post hiring announcement here constantly, in particular Tesorio ( https://news.ycombinator.com/from?site=tesorio.com ) which as far as I can tell post every time the rate limit permits them to do so. After 7 years are they a healthy successful company or is it just endless spam?

[bredren]

In my experience, long-standing unexplained openings in small companies, including early stage startups, are signal that something is wrong.

I did see this in one example in the “who’s hiring” thread here, but also in following the newsletter for a startup I applied to.

Two reasons I saw in my last job hunt:

1. The company is lowballing wages.

2. The hiring manager for the position does not want to hire or is not aligned with the startup team‘s decision that they should hire for the role.

You can get good at qualifying for these scenarios early, but if you don’t have experience looking for and actively screening these out, a list of repeat posts might save some people time.

At least it might add weight to a possible dismissal.

[cwkoss]

We tend to hire multiple developers in bursts, and recycle our text between months. We only remove the role-specific blurb if we hit capacity for that role.

So, not sure this methodology identify quite what you're looking for.

[snowwrestler]

The US Digital Service posts in pretty much every hiring thread, because many folks who work for USDS are on a temporary “tour” that only lasts a year or two.

I’m sure there are other reasons companies would post each month: churn, or always on the lookout for strong talent, etc.

[dekervin]

Since I spent a lot of time looking at "Who is hiring post" for fun anyway, I was looking for a way to make it productive and came up with a different idea [0].

I was wondering if you could detect the slow rise of a new technology and profit from it. Either by retraining or more directly, in the case of crypto, by trading on that info. I put together something, focused on a couple of crypto technologies and as luck would have it, it's still online !

[0] https://datum.alwaysdata.net/static/data-hnjob-trends/index....

[kodah]

I like this. I'd also love to see the venn diagram of "Who is Hiring?" top-level comments and a mix of the best and worst feedback about those companies and their hiring processes during that time. I want to see if companies are getting better or worse. I just wasted over a month with a company only to get feedback like, "The CTO wanted you to know the Bitcoin whitepaper more". Companies, as the market gets more competitive, are only doing shadier and shadier things with their time and communication.

[mountaineer]

Had a similar thought a few years ago and added a total posts chart to go along with the technology trends, https://www.hntrends.com/2022/may.html

Looks like you have June 2021 as the all time high as well. Doesn't look like June 2022 is going to get even close, which makes sense based on current freezes/layoffs.

[swyx]

the real number you want to see is the one you cant get - number of applicants per job posting is surely going up

[jpablo]

Can we overlay this with the nasdaq or some other stock index/other economic indicators?

[larwent]

Ask and you shall receive https://i.imgur.com/fN5EoIB.png

[gombosg]

Haha, I had fun making this a week ago, to learn more about the inverse: "Number of 'layoff' posts on HN vs NASDAQ" https://imgur.com/a/v4qPut5

[speakspokespok]

Your blog post starts by discussing posts being down this month (June 2022) but the graph provided starts in 2011. For the sake of the discussion, a second graph that starts June 2021 to current, and in monthly increments would be more enlightening. Because ultimately I think that's what we all want to know. In your copious free time would you mind adding that?

Thank you for writing this.

[pcthrowaway]

Here's a bar chart comparing "Who is hiring" with "Who wants to be hired" over the last year: https://jsfiddle.net/9d1f534b/

[tjwds]

Sure: https://dev.joewoods.dev/img/june-to-present.png

But my website seems to have disappeared, so I'll have to update the post later!

[endymi0n]

Anecdotally, after two years of literally just looking at tumbleweed in the application inbox, as a still-hiring company I have rarely felt as swamped with really interesting candidates and annoying recruiters as in the past four weeks.

The tide has turned for sure.

[gopher_space]

Senior dev with my own anecdote. There are more open positions with interesting companies than there is time in the day to apply for them. I’m able to filter them by turn around time on hiring and interview process because I’m spoiled for choice.

[itsdrewmiller]

Very cool! Would be interesting to try to control for hacker news popularity - something like total upvotes or total comments as a denominator might make this look even worse.

[ianai]

Forgetting which bit of macroeconomics this comes from, but the labor market is one of the slowest to settle (i.e. find equilibrium). Capital (real estate/manufacturing investments, stuff used to make stuff) are the slowest. So the capital unemployment rate should generally be greater than the unemployment rate, for instance.

The anecdote of Tesla posting a bunch of jobs just before the 10% reduction was announced comes to mind. Lagging indicator stuff.

[LanceH]

I find it matters a lot with what sort of tech work you're doing. If you are operations, a big company needs to keep their current stuff up and running. Exploratory new programs shrivel up when things get tight, though.

[jqbx_jason]

Front page! Way to go Joe!

[garrtt]

woah- small world. didn't expect to see a familiar username on this random HN thread. thanks for making JQBX!

[jbigelow76]

Meta-side question I've had since being part of this community for awhile, how come when a YC staked company posts they are hiring comments are disabled? As I post this "Generally Intelligent" (YC S17) is hiring for ML people (spot 15 per my last refresh.)

[dwaltrip]

Those are ads. It’s a perk that YC companies get. Otherwise, HN and YC are fairly independent, from what I understand.

[NickRandom]

OG link https://news.ycombinator.com/item?id=22767253 by Dan G

" If job ads were threads, each thread would be a generic referendum on the company. Worse, it would be the same referendum over and over.

Job ads are boring, so there wouldn't be anything to discuss other than how one feels about the company, and that's boring except to people who have strong feelings on the topic, and strong feelings on the internet tend to be negative, so the threads would fill up with negative generic comments.

I believe that the hivemind resents boring things, such as submissions where the only new information is "X is hiring", so it gets cranky and fills the vacuum with indignation, basically as the only way to amuse itself in the absence of anything interesting to discuss. It doesn't want to, it just doesn't know any better way to have fun in a vacuum. In practice, what this would look like in a job thread is "I applied in 2015 and never heard back", plus—if there has ever been a negative story X about the company—every variation of X, X, X, repeated increasingly snarkily.

Actually, it's worse. Building a business is a long hard slog. One needs to hire far more often than one has scintillating new information for the community to have fun discussing. Therefore, each successive job ad would be even more boring than the previous one, leading to monotonically increasing resentment. Repetition is the enemy of curiosity.

Launch posts don't suffer from this dynamic because by definition, the startup is new, so there's something new to discuss. "

HTH

[mcemilg]

From another point I would like to check who wants to be hired posts by comparing who is hiring. I can see that hundreds of ML engineers and Data Scientists looking for a job but on the other side there are not that many jobs for ML folks.

[dryd]

Another way to interpret these results is that HN has gotten more popular as a place to source talent. It would be interesting to see top-level comments by number of total users on the site (as a proxy for popularity).

[wcedmisten]

Very cool to see this data analyzed! I wonder if the number of posts could be normalized against some other measure of HN popularity. Maybe total number of posts?

[langsoul-com]

I wonder if you could find out how many positions from HN get filled up.

Say by comparing with an employees official job site and the HN post role.

[seizethecheese]

In economics data, they tend to do seasonal adjustments. I would be curious to see if there’s seasonality here.

[soared]

Good thought, though fiscal year v calendar year probably messes that up. Oracle did budgeting in FYQ1 and so tended to not hire much during that quarter, but then hired a lot in Q2, then slightly less Q3/4.

[bella22231]

Hacking your spouse is not exactly fair nor is it legal. This is why you have to get yourself an experienced hacker like I did when I could not take my husband's infidelity no more. Now we are seperated due to all i found out, could not stomach it all. The hacker was so discreet, calm and collected,no leaks. Reach him via wisetechacker @gmail com Thanks

[IshKebab]

Am I supposed to see a dip at the end? Hmm maybe if you squint quite a lot!

[svara]

Uh, so, HN predicted the pandemic? :) Curious what's going on there.

[Mountain_Skies]

Before anyone in the West was aware of the virus, much less taking it seriously, information was leaking out around the East. Given how much connection there is between Silicon Valley and China, it's possible some tech companies were hearing non-specific dangers from their partners in the East. Or it could be entirely coincidental. Probably will never know for sure.

[classified]

Ouch. White diagrams on a black background. Someone needs a lesson in design.