Hi everyone,
I'm sharing this in case others have hit the same issue after upgrading to 8.60 or 8.61. We have an open case with Rapid7 support and are working with their team, but wanted to put this out here for visibility. Again this is what we have seen so far, and we might be wrong but I would let Rapid7 to look into this on priority to see and address this issue.
Environment
-
Security Console upgraded from 8.57 to 8.60 (now on 8.61)
-
30 distributed scan engines across multiple data centres, including Rapid7-hosted engines
-
Scan engine pools in use
-
All engines show as online, active and on the correct version
The problem
After upgrading from 8.57 to 8.60, scans stop being dispatched to engines. The UI shows "Scan Engine: N/A" with 0 assets indefinitely. All 30 engines remain online and reachable. A console restart temporarily restores scanning, but the problem returns within hours, sometimes after just one scan completes. Upgrading to 8.61 did not resolve it.
What we found
We installed a JDK on the console and captured a thread dump while a scan was stuck. It showed a deadlock between two threads inside the console:
-
The engine updater thread (NSEUpdater) holds Lock A (a ReentrantLock inside NSEScanManager) and is waiting for Lock B (a HashSet lock inside NSEManager).
-
The scan dispatch thread (ExecutiveScanManager) holds Lock B and is waiting for Lock A.
Neither thread can proceed, so no scan can be dispatched to any engine. The two code paths acquire the same two locks in opposite order, which is a classic lock-ordering issue. Java's built-in deadlock detector does not flag it because the two locks are different types (a ReentrantLock and a synchronized monitor).
We also found intermittent TLS handshake failures between the console and engines. Engine logs show the console occasionally connecting with SSL_NULL_WITH_NULL_NULL (null encryption), which the engine rejects with SSLPeerUnverifiedException: peer not authenticated. The console logs no error for these failures. On the same engine, within the same hour, we see successful TLSv1.3 connections alongside the failures. These silent TLS hangs likely contribute to the lock contention by keeping threads stuck on unresponsive connections.
An interesting configuration detail
We have a second console on the same version (8.60) with 19 engines that does not have this issue. The two consoles have identical configuration except for one setting: coreScanStatusThreadPoolSize. Our affected console has this set to 19, while the healthy console has 3. This setting was 19 on our console since before the upgrade and worked fine on 8.57.
We believe the higher thread count increases the likelihood of the two code paths hitting the locks at the same time, which is why the deadlock triggers within hours on our console but never on the smaller one. However, the setting itself isn't the root cause — it was the same on 8.57 where scanning worked fine. Something in the 8.58–8.60 upgrade appears to have changed the lock ordering.
We've asked the Rapid7 engineering team to advise whether reducing coreScanStatusThreadPoolSize to 3 is a safe interim workaround for a large environment with 30 engines and pools, and what the expected impact on scan result processing would be.
What we've provided to Rapid7
-
Full thread dump showing the deadlock with exact lock addresses and stack traces
-
Engine logs showing the intermittent TLS failures
-
Console comparison (affected vs healthy)
-
TRACE-level engine communication logs
Current status: OPEN
This issue is still unresolved. We are working with Rapid7 support and have requested engineering involvement. A console restart is our only workaround, and it's temporary. If anyone else has experienced scan dispatch failures after upgrading to 8.60 or 8.61, especially with scan engine pools and a larger number of engines, please share your experience.
Has anyone else seen this, or has anyone been advised by Rapid7 to change coreScanStatusThreadPoolSize?
Thanks