Three Findings We Killed Before Publishing Them

The job-market content you read is written from data that survived. Almost nobody publishes the findings that died, which makes the surviving ones look more solid than they are. Here are three we computed this month, wanted to publish, and killed.

1. Where the jobs are, by country

We have country codes on 87.3% of active postings. The ranking looked immediately publishable: United States 50.4%, India 6.9%, United Kingdom 6.2%, Canada 5.2%, Germany 3.6%.

Then we split each country by which job board it came from.

That single board is a remote-focused one. "Jobs in Poland" was overwhelmingly one board's remote listings tagged with Poland as an eligible location. A ranking like that is a map of our ingestion contracts wearing the costume of a labour market.

We had made this exact error before, in a metro-level ranking that put Columbus, Ohio ahead of Seattle. That one shipped, and correcting it is what taught us to run this check first.

2. Month-over-month market trends

The most requested thing in job-market content is a trend: what changed since last month. We have a corpus with timestamps, so it should be easy.

It is not, and the reason is worth being specific about. Here is how the source mix of newly-ingested postings shifted between July and August:

Nothing in the labour market did that. Seven of our eight budgeted API sources exhausted their monthly quota in the first days of August and went dark, so what changed month over month is which feeds still had budget. Any trend computed across that boundary would be reporting our procurement, not hiring.

We would need either paced ingestion that holds source mix stable, or a within-source trend that only compares a feed to itself. Until one of those exists, we are not publishing month-over-month movement, and we would treat anyone else's with the same suspicion unless they say how they held source mix constant.

3. The remote work rate

Our corpus is 59.0% remote. That is a headline, and it is meaningless.

Three of our sources are remote-only job boards: 100.0% of their postings are remote by construction, and one of them alone is 44% of the entire corpus. Drop those three and the remote share falls to 25.1%.

Neither number is the market's remote rate. The 59% measures our board mix. The 25.1% measures what is left after removing the remote-focused boards, which is a different bias, not a corrected one. We do not have a defensible remote rate and we are not going to invent one. What we can honestly say is a supply-side fact: a large share of the remote listings a job seeker encounters come from boards that carry nothing else.

There is a second, structural problem: our remote field is a boolean. Hybrid is invisible to us. A "remote rate" from a system that cannot represent hybrid is answering a different question from the one being asked.

Why publish the graveyard

Two reasons, one principled and one selfish.

The principled one: a control that kills your finding is doing its job. All three of these died the same way, by splitting the result by data source and watching it dissolve. That single check is the highest-yield thing we do, and it is the check most missing from published job-market statistics.

The selfish one: we would rather you trust the numbers we do publish. Every insight we ship now names its denominator, states its control, and says what it does not claim. Knowing which findings did not clear that bar is the only way to judge whether the bar is real.

If a job-market statistic does not tell you how many sources it drew from and what happens when you split by them, that is the first question to ask. It is usually the last one it survives.