Why Our AI Resilience Scores Just Got a Reality Check

Methodology update · August 19, 2026 · 7 min read

Until now, every occupation in the AI Resilience Report had two related but separate pieces of information: its AI resilience score and its confidence level.

The resilience score answered our main question: how well-positioned is this occupation as AI changes the labor market?

The confidence level answered a different question: how much evidence do we have behind that estimate?

That distinction is useful, but it created an important limitation. Two occupations could receive equally extreme resilience scores even if one was supported by many agreeing data sources and the other was based on much thinner or less consistent evidence.

We wanted the topline score itself to acknowledge that difference.

Applying a confidence adjustment to AI Resilience results

We now apply a confidence adjustment after the standard AI resilience score is calculated.

The underlying AI Resilience model has not changed and neither has the calculation of confidence itself; the score still begins with the same three dimensions, Meaningful Human Contribution, Long-term Employer Demand, and Sustained Economic Opportunity, using the same underlying data sources and weighting structure.

The new step happens afterward. When confidence is high, the original score changes very little. When confidence is lower, we move the score modestly toward 50, the neutral midpoint of the AI resilience scale.

Importantly, this works in both directions. A low-confidence score of 80 moves somewhat downward toward 50. A low-confidence score of 20 moves somewhat upward toward 50.

That symmetry is important since low confidence is not the same thing as low resilience; instead, it means we should be less certain that the occupation belongs as far from the middle as the available data currently suggests.

Simply multiplying the AI resilience score by confidence would have treated uncertainty as evidence of vulnerability. We deliberately avoided that.

How the adjustment works

The strength of the adjustment depends on the occupation's confidence score.

At the adjustment strength we selected:

A confidence score of 100 leaves the original score unchanged. As confidence decreases, the score moves progressively closer to 50.

The same rule applies below the midpoint. For example, an AI resilience score of 20 with confidence of 50 becomes 27.5 rather than being pushed even lower.

This adjustment is applied as the final step in calculating the AIR score, so the score users see reflects both the underlying resilience estimate and the confidence behind it.

What confidence actually measures

Our confidence indicator is based on two things: how many relevant sources are available and how much those sources agree with one another.

We evaluate those separately for our AI Exposure and Demand data and combine them into an overall confidence score.

That also means a very low confidence score should not automatically be read as “the sources strongly disagree.”

In some cases, confidence is low because there simply are not enough sources available to calculate agreement reliably. The adjustment treats both situations cautiously: when the evidence supporting an extreme score is limited, the final estimate moves closer to neutral.

What actually moved

We tested the adjustment across all 1,597 occupations in the report.

The overall effect was modest:

• The average occupation's score changed by 2.3 percentage points.
• The adjusted scores maintained a 0.992 correlation with the original AI resilience scores.
• 201 occupations, or 12.6%, changed resilience labels.
• Every label change moved an occupation toward the middle of the resilience scale.
• No occupation jumped across multiple resilience categories.

The relationship with confidence also followed the expected pattern.

In other words, most of the movement is concentrated where the evidence is weakest. High-confidence occupations remain comparatively stable.

Why 50?

Any method that pulls uncertain estimates toward a reference point raises an obvious question: why choose 50?

The simplest reason is that 50 is the midpoint of the AI resilience scale. Moving toward it does not inherently favor either higher or lower resilience.

But we also tested that assumption.

We compared neutral anchors of 45, 50, and 55. An anchor of 45 caused substantially more scores to move downward. An anchor of 55 caused substantially more to move upward.

The 50-point anchor produced the fewest label changes, the smallest average adjustment, and the most balanced number of upward and downward score movements of the three options.

That does not prove that 50 is the only possible choice. It does give us confidence that it is the least directionally biased starting point for this adjustment.

Why not adjust more aggressively?

We tested several adjustment strengths.

A stronger adjustment would make confidence matter more, but it would also pull many more occupations toward the middle and create substantially more label changes.

At half the selected adjustment strength, 6.8% of occupations changed labels. At our selected strength, 12.6% changed. At the maximum strength we tested, 23.4% changed.

We chose the middle option because it allows confidence to meaningfully affect uncertain estimates without overwhelming the information already contained in the underlying AI resilience score.

The goal is not to erase differences between occupations. It is to be appropriately cautious about differences that are supported by weaker evidence.

What this means if you're using the report

The biggest change is conceptual: the topline AI resilience score now reflects not only what our data suggests, but also how strongly the available evidence supports that estimate.

For most occupations, particularly those with strong data coverage and agreement, the score will look very similar to how it did before. If you are viewing a score for the first time, you can feel confident that source availability and agreement are taken into account.

Ultimately, occupations with limited data or lower agreement may move somewhat closer to the middle of the scale, and some occupations near a category boundary may receive a different resilience label.

The underlying evidence has not disappeared. Confidence remains useful context for understanding why an occupation's score was adjusted and how much weight to place on the result. We are continuing to display the confidence level separately because we believe it remains an important transparency signal.

What's next

Confidence adjustment does not eliminate uncertainty, and it does not turn AI resilience scores into probabilities.

We'll continue testing the approach against recognizable occupations, edge cases, and future updates to the underlying data. We also plan to evaluate whether occupations with particularly limited source coverage should receive different treatment as the model evolves.

As with the rest of the AI Resilience Report, the goal is not false precision. It is to give students, job seekers, educators, and career advisors the most useful picture we can of how careers may hold up as AI transforms work, while being transparent about the uncertainty behind those estimates.