Hi, I’m trying use the insightVM plugins with insightConnect and it’s just so slow, so I’m assuming what I’m trying to do is not what it’s intended for or I’m doing something wrong?
I’ve created an asset group in insightVM that applies a tag to all systems with one or more Critical Vulnerabilities (CVSS 9+). The workflow then does an Asset Search for assets with this tag. It then loops through each asset to retrieve the vulnerabilities on that asset, then it loops through those vulns to find the critical ones. When it finds one it adds the title and description to a Global Artifact (GA) and when it’s finished with that asset it sends an email with the GA data, clears the GA and then moves onto the next asset.
This all works really well, but it takes a huge amount of time to run. I’ve just run it against a site that only has 6 devices with Critical assets and it took 15 minutes. Given the number of assets we have that unfortunately meet the criteria it would take literally days to run.
Is this just a limitation of the insightConnect system I’m finding or am I being hugely inefficient? Or is there another way to do this?
It would use two workflows, but you could have one workflow get a list of assets and loop through them and kick off another workflow that does the checks and has an input of the asset. The first workflow would loop quickly and trigger parallel runs of the ones doing work. This could trigger many jobs that would hurt performance though (don’t ask me how I know), so have a batching process to protect it
I have a Get Assets insightVM step, that then goes into a Loop around the assets discovered. To simplify this, how would I create an array of all the asset hostnames though the loops that I could then retrieve once the loop had completed to then add into an artifact? I’ve always used GA for this kind of thing (even though I don’t like that method) as I’ve never worked out to store data that is persistent through loops and then outside of the loop at the end.
Oh - as nothing in that page of the loop setup is configurable I’ve always just skipped over it. I never realised that once you add a step and go back inside the loop you get the option to create output. I’ll read the manual again!
InsightConnect is known to take a performance hit when looping over many items. Not only are you making many API calls each loop, but the longer a loop runs the slower they get. The solution would be to use an API that can get more data all at once(think SQL query export) or put some of your solution in a python step since it’s not bound by the same performance issues as an ICON loop. Using plugins like JQ can also help to digest large amounts of JSON without looping over each item to find what you are looking for.