How We Classify 1B+ Jobs: Inside Our NLP Enrichment System

Raw Postings Are a Commodity. Enrichment Is the Product.

Every day, tens of thousands of new job postings appear across Indeed, LinkedIn, Greenhouse, Lever, and hundreds of thousands of employer career pages. The titles alone are a mess. "SW Eng II," "Software Developer (Remote)," "Sr. Software Engineer - Backend," and "Programmer Analyst" might all describe the same role. Multiply that by 433 million unique jobs, each needing standardized titles, occupation codes, seniority levels, and predicted salaries, and the problem becomes clear quickly.

We solve this with our NLP enrichment system: a suite of independent, purpose-built ML models that each handle one enrichment task across our entire corpus of 1,011,115,180 job postings. This post walks through each model, what it does, and how it performs at scale.

Why Independent Models, Not a Monolith

Each model is independently trained, deployed, and versioned. When we improve our SOC classifier, we do not risk regressing our seniority model. When we add a new capability (NAICS prediction is next), we add a new model without touching existing ones.

This matters operationally. A monolithic enrichment pipeline means one bad deployment can degrade every downstream field. With independent models, a regression in title normalization does not corrupt skills extraction or salary prediction. Each model has its own accuracy metrics, its own rollback path, and its own retraining schedule.

The enrichment layer runs retroactively over the whole archive rather than only on new intake, so a 2022 posting carries the same fields as one ingested today: 99.64% coverage on seniority, employment type, and work mode, 99.88% on SOC. Together with the extraction layer, they populate 100+ fields per record.

Title Normalization

Job titles are the noisiest field in any job market dataset. Employers invent titles freely: "Ninja Developer," "Customer Happiness Hero," "VP of Vibes." Our title normalization model maps every raw title to a standardized canonical form.

Across all normalized titles, the confidence distribution tells the story:

The result is 99.71% title normalization coverage across 1,011,115,180 postings. About 2.9 million records lack a normalized title entirely.

That distribution looks pessimistic next to a naive model's, and that is the point. These scores are isotonic-calibrated against held-out truth, so a 0.7 means the mapping is correct roughly 70% of the time rather than "the model felt good about it." An uncalibrated confidence that clusters at 0.9 tells you nothing; a calibrated one lets you trade coverage for precision with a number you can actually reason about.

If you are working with normalized titles downstream, that makes the threshold yours to set. A labor economist might filter to 0.7+ and accept 28.6% of postings at high precision; a search engine might accept 0.5+ and take 72.98%. See our glossary for details on how normalization relates to SOC classification.

SOC Classification

The Standard Occupational Classification system is the lingua franca of labor economics. Every government statistic and compensation benchmark uses 6-digit SOC codes. Our classifier assigns codes using both the job title and description as context, not title alone.

Across 493.9 million NLP-processed records, we achieve a 0% null rate on SOC codes. The top codes span software developers (15-1252), registered nurses (29-1141), retail salespersons (41-2031), general managers (11-1021), and truck drivers (53-3032).

We publish our exact numbers rather than ranges: on a held-out gold set spanning all 867 BLS 2018 detailed occupation codes, our classifier achieves 94% top-5 accuracy and 73% top-1. Description length matters enormously: short postings under 100 words produce lower-confidence classifications, while detailed postings with bulleted requirements give the model rich signal. This is one reason why ATS feeds tend to produce better enrichment than aggregator sources, since their descriptions are longer and more structured.

Seniority and Employment Type

Our seniority model classifies every job into Entry, Mid, Senior, Lead, and Executive levels. What distinguishes it: it always returns a classification. Our current enrichment system achieves 0% null on seniority across all 69.2 million records processed. The legacy enrichment pipeline shows 15.6% null on its 424.7 million records, a gap we are actively backfilling.

Employment type classification (full-time, part-time, contract, temporary, internship) follows the same pattern. The current model achieves 0% null. The legacy pipeline has a 16.9% null rate. The difference between 0% and 15.6% illustrates why model versioning matters: you can measure improvement precisely and track regressions at the source level.

Remote Work Detection

Remote work classification is our most historically complex enrichment. The field did not exist before 2020. Coverage by vintage:

• Pre-2020: Null is expected and correct. Virtually all jobs were implicitly on-site.
• 2020 to 2022: Structured fields emerged. Coverage improved from near-zero to 40 to 70%.
• 2023 and beyond: Work mode is standard. Our current NLP pipeline shows only 4 null values out of 493.9 million processed records.

This vintage sensitivity applies across the pipeline. Fields that look broken in 2018 data are perfectly normal. Fields that are null in 2024 data represent real bugs. We document these patterns in detail in our methodology.

Salary Prediction

Stated salary coverage has improved since US transparency laws took effect, but even in 2023+ data, only 20 to 30% of postings include a stated salary. Our prediction model fills the gap. Trained on 50+ million salary observations from Glassdoor and Indeed, it predicts annual compensation given a job's state, zip code, and SOC code.

The model returns -1 when prerequisites are missing. We would rather return nothing than return noise.

The prerequisite requirement (valid state, zip code, and SOC code) means salary prediction coverage varies by how well upstream location parsing and SOC classification perform. This is another argument for the modular architecture: improvements in SOC classification directly improve salary prediction coverage. For a full breakdown of salary coverage by year and state, see our datasets page.

The NER Layer: Skills, Certifications, and Benefits

Alongside classification models, we run Named Entity Recognition in two steps: a high-recall dictionary scan, followed by a title-aware relevance filter. From 599.3 million processed records:

• 93.3% have at least one extracted skill
• Coverage scales with description length: 47.5% for postings under 50 words, 99.5% for 500+ words
• Average word count across processed descriptions is 486 words
• Our taxonomy covers 40,000+ technical skills, 3,000+ certifications, and 250+ soft skills

The relevance filter is critical. Without it, a Barista job mentioning "experience with Java" (the coffee) would incorrectly receive "Java" (the programming language) as an extracted skill, and a delivery driver posting could pick up "Ruby on Rails" from boilerplate. The title-aware filter reviews each role and skill pairing and removes these off-role mentions, which matters most for high-frequency terms that appear in non-technical contexts. It is tuned conservatively to keep legitimate skills: an independent multi-judge audit confirms about 99% of removed tags are genuine false positives, and fewer than 1 in 500 legitimate skills are affected. In production this concentrates the average from roughly 10 to roughly 8 skill tags per posting.

For more on how skills extraction performs across different source types and description lengths, see our comparison tool.

How It All Comes Together

Our enrichment pipeline's output culminates in 1,011,115,180 job postings with 100+ fields each, resolving to 433,666,433 unique jobs after cross-source deduplication. Every record carries a normalized title, a 6-digit SOC code, seniority, employment type, remote work status, predicted salary (when prerequisites are available), and extracted skills.

We are currently developing NAICS (industry classification) prediction as the next addition to our enrichment system. The modular architecture makes this straightforward: each model is independent, stateless, and horizontally scalable.

To see how enriched records compare to raw postings, try our comparison tool. If you want to evaluate the data for a specific use case, request a sample filtered by source, region, or occupation.