Tell HN: The German job market is crashing

[Sladik]

If we looked at the German Job Market as if it were the stock market, we would say that it's crashing! On the following link you can see my pet project where I have been scrapping the major job offer portal in Germany for over one year. In the last two weeks it has lost 33% percent of all posted job offers and it keeps dropping as a rock :-(

Dashboard: https://jobmarketanalytics.com/#months=%2212%22&technology=%...

Source Code: https://github.com/petracarrion/job-market-analytics

Slide Deck: https://petra.carrion.io/job-market-analytics/slides/

Important! If it is offline, please refer to this image: https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1

[snowpid]

For me, the OP's interpretation is very questionable or at least hasty. The first reason that comes to mind is that there is less recruitment at the end of the year. Because in the above statistics (due to lack of data?) only the data within the year is possible, such rhythms are not recorded.

I talk to HR people from different companies (in Berlin) from time to time and they still complain about the lack of skilled workers. Lastly, there is an armada of economic institutes in Germany that do not foresee a "job market crash" either. Jobs are a lagging indicator and the German economy is still growing. Only next year will there be a slight recession.

P.S.: For a good statistical analysis, the data source should be mentioned (which website), different sources (e.g. different portals, different source). A data lake otherwise does not make much for an alarmist analysis.

[vernon99]

Have you checked the chart linked by OP? It has data for the same period of the last year and there’s no such drop so it disproves the seasonality thesis.

There could be something else of course. But also possible that some companies are being hit by energy crisis.

[sdfgdfghj]

> Only next year will there be a slight recession.

How do you know that?

[uberman]

0 the y-axis otherwise the chart is misleading.

[Sladik]

Good point, now the y-axis starts at 0 but even now we can appreciate how abrupt the fall is.

[rognjen]

Since the jobs are scraped and not fetched through an API, I'd also check whether the structure of the site changed in some way.

I'm very suspicious of the data, because a 33% decrease in less than a month is unreasonable. Even at the height of Covid, there wasn't that much of a drop-off.

My money is on the script being broken.

[Varqu]

Yeah, our observations at GermanTechJobs.de also don't align with the OP's data.

November looks more like a recovery month compared to a very bad September and bad October.

[sendfoods]

This is very, very disheartening. Finishing my degree in exactly 2 weeks.

[dont__panic]

Don't wait until you get the degree to start applying! Send out those applications in droves until you start to get tired of it. Then study for tests. When you get sick of studying, get back to applying. It's honestly just a numbers game.

[BadBadJellyBean]

My experience is that the degree doesn't matter that much if you have experience. So if you have no experience yet it might be tougher.

Hiring will probably pick up again next year, though. I hope you find something.

[Sladik]

Then good luck with that. I remember 2009 which was also a hard year but it wasn't impossible. Maybe start right after you are done.

[Sladik]

Important! If it is offline, please refer to this image until I get it back online: https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1

[Sladik]

More images on https://petra.carrion.io/static_dashboard/

- https://petra.carrion.io/static_dashboard/imgs/overview.png

- https://petra.carrion.io/static_dashboard/imgs/top_5_cities....

- https://petra.carrion.io/static_dashboard/imgs/top_5_technol...

[xbar]

Thank you. I'd hate to make a joke-in-poor-taste about the German Job Market Dashboard crashing.

[Sladik]

As the Dash frontend wasn't able to handle the traffic I have replaced the dynamic dashboard with static HTML, sorry for that. I will replace it with the real dashboard when I am able to handle the traffic.

[Sladik]

OP's disclaimer: I started this hobby project one year ago. The point of this project was and still is to learn about new technologies. Maybe I am lacking the statistical background, or the data since it goes back only 13 months, to make such an statement but when I starting seeing these trends in the number of job posts, I thought that it would be interested to share it in the Hacker News community and get some feedback.

About whether these trends that we see in the graphs are something real or just a data issue like seasonality or a problem with the data sources, I think we will soon know from the mainstream media. As always the time will tell.

[tkiolp4]

Regardless of the final (statistical) outcome, I think the architecture of your project is impressive. Kudos!

[_lth]

The presentation is "nice", but not even close to what it needs to be for making any assumptions on the result. Questions that are necessary to be clarified:

1) What is your source? - the slides only state it's the biggest platform. 2) How do you compensate, that your dataset only represents x% of the real offers? 3) If you think you have a reasonable share of the job offers, how do you justify this assumption?

These are the main questions, that I would ask if you tell me to use it as business information - all the technical stuff is secondary. But there are more:

4) How do you compensate a change in recruitment strategy? You are just looking at one platform. 5) What makes you sure, that every item in your dataset is interpreted correctly? Can you somehow make sure, that your parsing and reformating is always correct? Can you verify that? Can you detect if not? 6) Can you detect a change on the website itself, e.g. new categories, differences in job description?

Also my experience is different, with getting a ton of job offers currently, directly via business network, I'm willing to look at a statistics based bigger picture that changes my mind. But the interpretation and the harsh change, together with these open questions make me believe that you more likely have a technical problem. So my advise would be, spend a lot more slides on the data and not the acquisition, because all acquisition effort is useless if you can not verify your data and results.

[_aavaa_]

> https://petra.carrion.io/job-market-analytics/slides/#/6/0/0

Is there some German/EU law that requires job postings to be available for at minimum one calendar month? Cause that spike at ~30-31 days is real conspicuous.

Or at HR people just defaulting to 30/60/90 days for the posting being automatically removed?

[hobofan]

I would assume that it's not due to HR choosing to take them offline, but because that's the interval you pay for them on the platform that was scraped.

[HeikeMaria]

Not sure which sources she crawled ... Many of the German job boards require companies to purchase job ads which expire after 30 days.

[Sladik]

I cannot say if it is a law or the pricing plan but 30 days seems to be the most recurrent job post length.

That means that the crash probably started in October and we started seeing it on the data at the beginning of November.

[sdevonoes]

Interesting! Are you planning to keep recollecting data for the year 2023?

> the major job offer portal in Germany for over one year

I hope that's Linkedin. Because if it's another site like Monster, StepStone or similars, I'm afraid the data is probably garbage. I don't know a single engineer in Germany who uses anything but Linkedin (well, many engineers use niche job boards like remoteok, or directly use the company's career section... but no single one uses Monster/StepStone/Glassdoor, because they are total crap)

Edit: the data source seems to be stepstone.de. That's unfortunate. Is there any chance to run this against Linkedin for 2023? That would be awesome!

(https://github.com/petracarrion/job-market-analytics/commit/...)

[malteg]

seems stepstone is the only source... perhaps try to compare against "agentur für arbeit", see https://github.com/bundesAPI/jobsuche-api (there are multiple issues with the api - some parts/parameters are broken....)

[Sladik]

I am working on integrating additional sources, and Linkedin will be definitely one of them.

[chrispeel]

In my area, COVID stats are updated by the local public health agency daily, but often deaths are only included in the death data a month after the death occurs. So the COVID death data is good only for dates that are about a month old. Is there any thing similar going on here? Is the data for the past few weeks of the plot [1] as reliable as the data in July?

[1] https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1

[Sladik]

After some research, my hypothesis might be wrong. Comparing the different data sources I have found out that there is a drop only in one them, so my best guest is that an increase in the job post price or a change in the business model (e.g. shortening the length of the time a job is posted) is the most likely reason for the drop we can see in November.

[jcadam]

Exploding energy costs will do that.

[TheAlchemist]

Very nice project and presentation deck - well done !

That being said, with the data you have available here, you can’t conclude anything at all - you would need several years of data to rule out seasonalities, and also take into account some external factors influencing it.

Ps. Are you sure you’re not using something like rolling average and you just don’t have the data points (if it’s centered around current date) ?

[xrayarx]

This is a link to the latest statement from the labor agency in germany. Due to seasonal effects, employment increased, of course this is only a short statement and does not include details. Statement is in german.

https://www.arbeitsagentur.de/news/arbeitsmarkt-2022

[nmridul]

Is this the year end factor ? Generally there is less recruitment towards end of the year (On top of the generally gloomy market)

[Sladik]

I don't think so, note how last year, there was still a small job post increase from middle of November until end of December.

[djangpy]

My assumption is that we are moving back to a pre-covid buyers market for the tech industry. It did feel like childs play to get a job during covid, from anywhere. I am certainly happy that I found (German) clients and a job during the covid period, and don't have to go looking for them now.

[eeemacs]

If everyone who found out the "German job market is crashing", "German economy is crashing" or "the EU economy is crashing" would have sent me 1 EUR, I wouldn't need the German job market any longer. Guess we're safe.

[Phelinofist]

I can also see this by the messages from recruiters I get, really dropped off in the last weeks

[rsj_hn]

This is a great project! But I would caution against being quick to interpret these numbers until you actually accumulated many years of data and are able to compare it to other macro economic variables such as the unemployment rate or changes in wages. Right now, you only have a few months of data, and it might be the case that this just happens every few months and doesn't mean much.

That's the point of time series -- you need to accumulate them over long periods of time and then you start processing them to understand them better to see if they are leading/contemporary/trailing predictors of what you really care about -- in this case, the tech job market.

So keep it up! But also wait a bit before using it :)

[the-printer]

In the future I wonder what other types of news like this will spread by way of data collected via someone’s pet/side projects.

[beardedman]

I guess a recession will do that? Would love to see other countries' breakdowns.

[netsharc]

I'm going to flag this for IMO sensationalist interpretation of data...

[mir3la]

It’s currently offline

[Sladik]

It seems that Dash cannot take the Hacker News traffic, here is a screenshot until it is online again: https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1

[ushakov]

are there any other sources for this?