Agent-Based Policy Problems

InsightVM
1

Hi All,

We’ve been Rapid7 IVM users for roughly four years, and we’re still experiencing significant issues with Agent-Based Policies. I couldn’t find any existing posts discussing this, so I figured I’d share our experience and ask for potential solutions.

Our number one complaint is the tool’s inability to correctly and consistently report registry values across our environment, which is the primary use case for us. Is anyone else experiencing similar issues (more info below)?

Issues:

  • Consistently reporting on the true configuration of the device.
    • The tool will report that a rule is failing, but when we check the associated Group Policy and registry key for that specific rule, the device is actually configured correctly.
      • This is an issue I’ve discussed with Support and my Customer Advisor many times. A device may show compliance with 96% of rules one day, then drop to 35% the next, without any configuration changes. The tool is clearly capable of reporting the correct state at times; however, getting it to do so consistently has recently become impossible.
        • This behavior is impacting roughly 20% of our environment at any given time. As a result, using this tool to troubleshoot enforcement or configuration issues isn’t practical or reliable.
      • Most often, when this occurs, the Proof section will simply state the expected registry value and mark the asset as failing, without displaying the actual value currently set on the system. When this happens, it’s been our strongest indicator that the device is configured correctly and is instead being evaluated incorrectly by the tool.
  • User Based rules or rules which reference User rights assignments.
    • We’ve had support cases open regarding Rapid7’s ability to correctly report on user-based rules since day one. Ultimately, we’ve had to resort to overriding any rules that check HKEY_USERS, due to the presence of the correctly configured local administrator account (SID-500).
      • Using HKEY_CURRENT_USER would be the logical next step to address this; however, through multiple support cases, we’ve been informed that there are limitations with the agent’s ability to query that registry location.
  • Poper Communication regarding new version releases
    • When a new CIS policy is released within Rapid7 the tool just enables the latest version, doesn’t migrate any overrides, and doesn’t provide any notice the new version is released or anything.
    • I’ve chatted with my CA regarding this and they shared there were a lot of complaints and open tickets on this

Also, are any of you who use the agent based policies tool in a heavily regulated environment (Healthcare, FI, etc.)?
I’ve asked a prior Customer Advisor to help identify peer organizations that are using Agent-Based Policies in a similar compliance-focused capacity, and we were told that we’re essentially alone in this use case, which I find hard to believe. That said, there also don’t seem to be many community discussions or shared experiences around this particular tool.

Any feedback on your issues or experiences would be appreciated.

2 Likes
2

Please let me know what you find out. I am about to start implementing the agent-based policies.

3

I think everyone has pretty much had the same experience. Unreliable, convoluted, and not worth my time or anyone on my team’s time to babysit support cases and manual troubleshooting on whole policy sets. We have gone elsewhere for Policy enforcement and reporting.

4 Likes
4

Mind sharing where you went?

1 Like
5

We’ve been working through some CIS controls via console and agent. Although i find struggles with the tool for compliance benchmarks, i (thankfully) haven’t experienced your level of strange behaviors.
IME, reporting isn’t great with either engine or agent.
Agent reporting via the cloud platform will fall on its face sometimes and not render pages correctly.
Engine reporting via SQL queries is cumbersome until you get a decent query written out and formatted and for us, i’ve been able to make edits and apply to the few other benchmarks we’re using.

Rapid7 has had issues with supporting devices that they claim would work, but it didn’t really work and i got the same feeling that you did - we’re in the minority of people trying to use this function of IVM. However, for one type of device that we were trying to do this on, Rapid7 did eventually get it to work but i was left in the dark with the support case until i checked back and they had me try it again and surprisingly it worked. I have no idea what they did to resolve the issue or when it was resolved.

I do hope that the agent based policy checks improve and that they can be incorporated into dashboards because leadership will eat that up, especially if Rapid7 will create the pretty pictures for me that i can copy and paste for review.

From my point of view, Rapid7 has focused it’s cloud platform on what they want and not what their customers actually want.

4 Likes
6

Hi @jrainey1 ,

It seems to be a similar issue to mine. I have an open case with Rapid7 Support. The first response I received was on February 20th, stating:

"This is classified as a defect with reference DCA-30793 and is obviously a top priority."

This defect has been partially resolved; they are now working on the fix with reference SI-33339.

Perhaps if you try to share this information with Support, they can help us more quickly.

Let's keep our fingers crossed.

7

I used to work in healthcare IT and found Tenable.io much better for compliance reporting than R7. We have frequent odd issues with R7 reporting too.

8

Anyone using WDAC and having issue with compliance scanning?

We too have had a very, frustrating experience with compliance scanning (Financial Institution Australia)