Rapid7 Insight VM CVSS Ratings rapidly changing / disrupting established evaluations and workflow

InsightVM
1

Hello,

starting with the content updates of Tue, 2025-02-18, we see extensive changes in the CVSS ratings of vulnerabilites (especially those w/o CVE-entries @ NIST), having a very negative impact on the usability of Insight VM.

To illustrate: A simple, established scope defining query (checking for fulfilled patching SLAs) like (sample):

asset.groups IN [‘asset group’] && asset.os.family CONTAINS ‘windows’
vulnerability.cvssScore >= 7
finding.firstFound <= /NOW - PXD/

where X is 30, 60, 90, 120

delivered

  1. a stabled result of 0 until 2025-02-18 morning (simply because the Win Srv env here always kept up-to-date).

  2. In the afternoon of the same day the result was 7 vulns on almost all assets.

  3. In the afternoon of 2025-02-19 it was 20 vulns on almost all assets.

All of that w/o any changes on the assets of course, only caused by R7 changing the CVSS ratings of quite a substantial amount of vulnerabilities.

A massive change like that w/o the chance to coordinate and perhaps opt-out for a certain time should not happen.

And there is no documentation anywhere regarding those changes, the content update documentation contains nothing regarding the entries that changed. No reasons, nothing.

This not massively disrupts established workflows, the newly introduced CVSS ratings are partly erratic and simply obviously wrong.

Sample:

X.509 Server Certificate Will Expire Within 30 Days receives 9.1 (vector: CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N)
X.509 Server Certificate Is Invalid/Expired receives 7.5 (vector: CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N)

No need to explain why this is obviously wrong.

  • Are those changes related to the very brief email from R7 of 2025-01-28 (“Upcoming changes to your risk scores”)? Don’t want to jump to anything, but those partly erratic CVSS values look a bit like something following a halluziation.
  • Anybody else experiencing this as disruptive as we are?

BR
Robert F.

2

Robert,

I noticed this on my end as well starting on February 18th and then noticed the scores reverting back to their original status a couple of days later. From looking through the vulnerabilities that changed it appear that some older items, mostly those related to SSL/TLS vulnerabilities were updated to have a CVSS v3 score, which did not exist for them previously. I was wondering if it had something to do with the announcement of Rapid7 using AI to assist with the vulnerability scoring.

I was going to open a case with support to ask about this change and if anything else like this is expected in the near future, but it would be nice for someone from Rapid7 to comment on here or perhaps they could publish a blog post or email regarding this issue.

3

For more information on the AI-Generated CVSS scores, we have a public blog post Rapid7 Enhances CVE Assessment with AI-Powered Vulnerability Scoring | Rapid7 Blog and also updated documentation here Working with vulnerabilities | InsightVM Documentation

These AI-Generated scores are only provided where there is no human provided score, and ensures risk prioritisation is more accurate, and if a human generated score becomes available, then we will update it.

As part of rolling out this feature, and updating thousands of historical, underscored vulnerabilities, there was an edge case where the model picked up some of our older, security best practice checks, and added scores to these. As you have seen, we quickly reverted this change for this very small number of checks.

We are confident that going forward, these AI-Generated scores will provide great value in providing more accurate risk prioritisation despite the growing problem of slower analysis by agencies such as NIST.