Deleting Duplicate Assets in Nexpose

Does anyone have a decent way of filtering for and deleting duplicate assets by IP? Right now, I’m exporting my full list of assets, opening in Excel, and using conditional formatting to find the duplicate IP entries. That’s an OK way of doing it, but i haven’t found a great way to take that list and mass delete only one of the two (or three) duplicates within Nexpose. I could enter those IPs into the query builder, but how would i go about making that list deletable?

Thanks for any help.

@alan_sarz A couple quick questions so I can get you going in the right direction. First, how are you identifying which assets to be deleted out of the duplicates by IP? Second, are you trying to do this with an InsightConnect workflow?

We do have the ability to delete assets by ID with the following API endpoint: https://help.rapid7.com/insightvm/en-us/api/index.html#operation/deleteAsset. Making sure we have the list of IDs programmatically (or from your Excel file if you do manual filtering) would be a good place to start.

I am searching for an IP, and then deleting all entries except for the one with the most current scan date.

I am not using insight connect. Is that something only for InsightVM? We are using onsite Nexpose, not any cloud-based solution.

I would make a dynamic asset group, and use it to limit scans to not most recent (greater than last seven days or whatever), then simply review and delete. However, if the rule is that it MUST be a duplicate as well… that’ll be harder… For that type of stuff we use the InsightVM API and have it run a script overnight. Specifically, we do this with short-lived instances that use the InsightVM so they don’t keep cluttering our asset counts, but really can use for any type of filtering you can do in code.

1 Like

I use the API to do the same thing for our systems with hostnames. I pull all of the systems that have a hostname, find all of the duplicates, and then delete the duplicate that was missing a MAC address, wrong OS, etc… There may be 1 or 2 that I have to do manually but easier than doing 20+ individually through Excel and the GUI. I use Powershell for the code but it will work with any language. Hope that helps

1 Like

I ran across an article a while ago on a good way to do this in Nexpose.
I think this might be it https://blog.rapid7.com/2017/07/11/cleaning-house-maintaining-an-accurate-and-relevant-vulnerability-management-program/
The big takeaway is making sure you have asset linking on. It also talks about removing stale assets and “Ghost” assets (Asset risk score is 0 & OS is empty & Asset name is empty) by creating asset groups to tag these. This won’t get all the duplicates, but it will get quite a few making the rest of the cleanup easier because I think you’ll find a lot of the duplicates will fall under the Ghost category.

yes thank you. I did read that before and I’ve added ghost asset criteria to a deletion asset group. Without moving everything to InsightVM (which i’m not allowed to do), I’m afraid it’ll be a largely manual process for me.

Here’s the easiest way I’ve found to do this:

*export entire asset group to csv
*Open excel and highlight, filter, and sort duplicates with the IP column
*create a user made tag and import with file. Might have to copy and paste the excel list into notepad and upload the txt file
*create an asset group and make the only criteria assets with the tag you created earlier

This way, you have a list of duplicates and you can select and delete whichever ones you want.

Can you use the API with on-premises Nexpose? I dont have licenses for InsightVM.

yes, documentation is at https://NexposeServer.domain.com:3780/api/3/html
It supports basic auth. I’ve done Tag management, ad-hoc scans, and software inventory with it and it works well.
The InsightVM Plug-in in InsightConnects works against this. They recently added the ability to limit a scan to one host in a site and as soon as my test box gets moved to its new building I’ll be doing a PR request to add the options to define the other scan parameters.

The InsightVM v3 API is compatible with InsightVM and Nexpose. Public documentation for it can be found here: https://help.rapid7.com/insightvm/en-us/api/index.html. And as Brandon mentioned it is also available from the Nexpose host itself.

Out of this process, it would be possible to automate most of the steps with the API out side of the filtering by IP. If you add any “human” decisions as to which asset by IP you keep versus get rid of then yes this would need to be reviewed; however, if you simply take the latest scan ID (or some other criteria) this might also be possible with the use of the API.