Deleting ghosts

InsightVM
1

Recently our firewall guy implemented a new feature, which has been causing two types of false positives

  1. Assets with no name and no risk, but with an OS detection, which is a false positive (OpenBSD, FreeBSD, MacOS, various kinds of Linux, none of which we use.)
  2. Assets with no name, no risk and no detected OS

Between these to categories I’m seeing over 150k false detections, and it’s incredibly annoying to deal with.

I’ve done a fair amount of searching, and I can’t see a way to get rid of them other than by going through hundreds of pages of the dynamic asset pages and deleting them 500 at a whack.

Is there some better way of paring these down?

Thanks,
Kurt

2

in InsightVM: Best Practices to Improve Your Console | Rapid7 Blog
look at section ‘Do not treat TCP reset responses as live assets’. To cleanup old the easiest way to to have a retention period and let them fall off automatically. If you cannot wait, there use to be a Tag filter that you could look for and tag then delete, but I forget what that was because with those settings it was no longer needed.

3

“Do not treat TCP reset responses as live assets” has been checked for a very long time, so I don’t believe that is the problem - the firewall seems to be analyzing the traffic coming across several ports to detect malicious traffic (Check Point Autonomous Threat Protection), but I haven’t been able to get a packet capture yet, so I’m not sure if it’s sending resets or doing something else.

I do have a retention period set as well, but I believe that would depend on the asset reporting with an agent, or aging it out from the last scan of the address, and since the retention period for stale assets is longer than our scan intervals, these false detects don’t get a chance to age out.

I did create an asset tag to match addresses with no OS and zero score, but it looks like the process for deleting the matches is the same as for deleting from the dynamic asset groups - selecting the visible assets, max 500/page, and deleting them.

Once we figure out how to exempt the scans from the scan engines, that should help, but we’re still working on getting that done in the firewalls.

Thanks,
Kurt

4

you can loop through them with a script to the api.
Call the tag first with the assets, then loop through them and delete. You will just have to code for paging

5

This can be done by using the API.

https://help.rapid7.com/insightvm/en-us/api/index.html#operation/deleteAsset
Delete InsightVM Assets.snpt (3.0 KB)

If you use InsightConnect I have a Snippet that does this. You just need to get the assets you want to delete.

6

I have been having same issue, with insightVM where ghosts assets appear with similar results and it only does it for the VPN subnet, for one site and not for others. Unsure as to why these ghost assets appear.
Any suggestions?

7

Thanks. I haven’t taken the time to do more than look through the docs on the API. I’ll bear down on that a bit and see what I come up with.

Kurt

8

Unfortunately, I don’t have InsightConnect. Thanks for the link, though. I’ll starting looking at the docs. on this.

Kurt

9

I encountered this when firewalls are set to send reset packet when denying a connection instead of silently dropping. Ask your firewall person what the firewall is set to do. Hope this helps.

Sorry, this is just repeating what was said before. Didn’t see that reply. But what is the requirement for the firewall to send reset packets. If there isn’t one, why is it set?

10

The example for deleting an asset shows this:
https://localhost:3780/api/3/assets/{id}/databases

where {id} is integer Required

I presume this is either the Windows UUID or the Rapid7 Agent ID. If that’s true, then that’s unfortunate, because the ghosts don’t have either.

If it’s something else, where might I find that?

Thanks,
Kurt

11

no, that is the Asset ID, you get that by querying https://localhost:3780/api/3/assets

Every asset has an Asset ID in IVM

What you want to call is this where {id} is the ID of the Tag you created
/api/3/tags/{id}/assets
that will give you the list of Asset IDs to call the delete on

12

Sheesh. I should know better to ask silly questions before reading through the docs.

I’m gathering my thoughts now, and planning my approach.

Thanks for your patience

Kurt

13

are you seeing any ports or services on these hosts by chance, for instance, 5060/tcp and 2000/tcp ??

14

Yes, indeed I am. Before this implementation, there were detections on port 1720, so I added that to the “do not scan” ports list in IVM. Now, I’m seeing ports 80,3128 and 8080. I don’t think it’s reasonable to exempt so many ports from scanning, so I’m working with our firewall admin to see how we can work around this. Since all of the IVM scan engines come from a specific address at each location, (10.nnn.150.10), I’m hoping that we can have the firewalls ignore traffic coming from that set of addresses.

Thanks,
Kurt

15

so this may or may not be one in the same, but we have had several FW’s after recent updates act this way, one of them being Fortinet and a few others, now this is directly in regards to fortinets but it could be worth looking at similar configs.

For the Fortinets, one or both of these should be helpful -
 
SIP/ALG service = 5060/tcp and 2000/tcp
Ident service possible causing 113/tcp
FortiOS : Closing TCP port 113
 
From <https://kb.fortinet.com/kb/viewContent.do?externalId=11763&sliceId=2>
 
 
By default, SIP-ALG is enabled, and only by the following command which can be verified with “show full”
 
   # config system settings
       set default-voip-alg-mode proxy-based
   end
 
By running the following command, we tell the FortiGate to disable SIP-ALG (proxy-based) and use SIP-helper (kernel-helper-based):
 
   # config system settings
       set default-voip-alg-mode kernel-helper-based
   end
16

OK - after getting sidetracked on other tasks, I had a few moments, and did some more research I found this, which works, but aborts after only a few deletions, and my hypothesis right now is that I need to do some sleeping between deletions:

I’m manually exporting the data from the dynamic asset group, then do the following PowerShell command (Windows PowerShell, not Core)

$ghosts = import-csv C:\temp\Ghosts1.csv | select -expand address
foreach ( $ghost in $ghosts ) { $assetid = get-nexposeasset -ip $ghost | select -expand id ; remove-NexposeAsset -id $assetid ; start-sleep -seconds 1}

The insertion of the start-sleep command seems to be helping, but with over 83,000 assets in the group it’s going to take a while. I’ll try to remember to come back here to give an udpdate.

Kurt

17

Pull a list of the asset IDs to IP first as a named array based on IP, then you can save half of your calls by doing
remove-NexposeAsset -id $mapping[$ghost]

I had to do something similar with tagging because searching for the asset ID one by one was killing performance

18

I put this together - is this what you have in mind?

$ghosts = import-csv C:\temp\ghosts.csv | select -expand address
$assets = @{}
$assets = ForEach ( $ghost in $ghosts ) { Get-NexposeAsset -ipaddress $ghost | select ip,id }
foreach ( $asset in $assets ) { Remove-NexposeAsset -id $asset.id }

I just started a run of the above - we’ll see if it bombs out.

Thanks,
Kurt

19

you’re still making the same number of calls. My suggestion would be to reduce the calls in the first loop to pull multiple assets per call and page through them. Your first loop could be reduced from 83,000 calls to166 with the max page of 500.
The idea is your script is making 166,000 API calls and probably getting throttled. I don’t think there is a bulk delete, but there is a bulk pull that could reduce these calls