While it’s still fairly common for InsightVM scanning infrastructure to be static, i.e deployed once with new scan engines provisioned in an ad-hoc, manual fashion, we also hear from plenty of customers that they would like to orchestrate this process. Orchestration of scan engine infrastructure can reduce the administrative overhead of deploying scan engines, and even enable workflows such as temporary engine provisioning prior to a scan and termination after the scan completes. This is especially handy if you’re trying to scan cloud infrastructure while keeping costs low.
For this post, I’ll cover the steps involved with orchestrating the engine deployment life cycle; however, we won’t provide a concrete example. Everyone is using different technology for managing infrastructure orchestration, so the goal of this post is to outline this workflow and provide a starting point if this is something that you’re interested in doing. Some of the concepts will be specific to Linux installation, so I’ll call that out where applicable.
There are two pairing methods, Engine to Console and Console to Engine. We’ll cover both and try to note when a step is specific to one or the other. For the sake of brevity, they’ll be abbreviated
E -> C and
C -> E. Below are the steps that we’ll need to manage the entire engine deployment lifecycle, from installation to termination (if desired).
- Get the installer
E -> C: Generate a Shared Secret
- Configure a varfile for the install
- This allows for passing options to the installer
- Run the installer
C -> E: Add the scan engine to the console
C -> E: Approve the console on the engine
- Validate the engine is available in the security console
- Delete the engine from the console (when scanning is complete or it is no longer needed)
Let’s get started.
Possibly the easiest part of this process, all we have to do to get the installer is download the correct binary for our operating system and validate the checksum. Binaries and checksums for each supported platform are available here.
You could use
wget or Powershell to download these, or use whatever capabilities are provided by the toolset that you’re using.
Only required for E → C communication
To generate a shared secret, the InsightVM API (v3) provides the following endpoint:
Which will return the current shared secret or generate a new one.
The public documentation at InsightVM API (v3) isn’t fully up to date, so this endpoint isn’t documented there. However, if you go to the documentation within the Security Console via help menu in the top menu bar, you will find full documentation for this endpoint and others associated with scan engine shared secrets.
Additionally, the API user will need to be a
Global Admin in order to generate the secret.
The installer allows for a variable file to be passed so that the installation can be performed in an unattended fashion. The parameters that need to be added to a variable file named as follows for each platform:
Below is an example of this file with comments detailing changes required for each scenario:
Example Engine varfile
# Rapid7 Engine Varfile # 0 for E -> C, 1 for C -> E communicationDirectionChoice$Integer=0 # nse for engines, nsc for consoles component=nse createDesktopLinkAction$Boolean=false dbPort=5432 company=<your company> firstname=<support contact name> lastname=<support contact name> port=5432 proxyHost= proxyPort= registrationRequired$Integer=1 requirePasswordChange$Boolean=False sys.adminRights$Boolean=true # True for engines, False for consoles sys.component.engine$Boolean=True # False for engines, True for consoles sys.component.typical$Boolean=False # C:\\Program\ Files\\rapid7\\nexpose on Windows sys.installationDir=/opt/rapid7/nexpose sys.languageId=en # False on Windows to create the application menu sys.programGroupDisabled$Boolean=True initService$Boolean=True # omit for C -> E consoleAddress=<console IP/hostname> consoleDetailPort=40815 sharedSecret=<shared secret generated by the API>
<some_value> need to be replaced with the correct values.
The names of the files are important, as without the appropriate names the installer will not read from the file.
As an alternative to using the sample file provided here, you can manually run the installer once. Once the installer is run, it will create a
<installation_directory>/.install4j/response.varfile file containing all of the answers from the manual installation that could be used as the template for your engine installations.
varfile ready, the installer can be run. On Linux, this looks like the following:
<path_to_installer>/Rapid7Setup-Linux64.bin -q -varfile <path_to_file>/Rapid7Setup-Linux64.varfile
On Windows it’s similar:
<path_to_installer>\Rapid7Setup-Windows64.bin -q -varfile <path_to_file>\Rapid7Setup-Windows64.varfile
As part of the installation, there is quite a lot of output. It’s recommended to review this output as part of your orchestration process to validate that the installation was successful.
If you’re using the
E -> C pairing direction, the scan engine should be ready to go once the service initializes (you can skip to Verify The Engine is Paired).
The V3 API provides the following endpoint to add a scan engine:
This will add the engine to the console; however, you’ll still need to approve the console on the engine, which we’ll cover next.
After you have added the engine to the console, there are two ways to go about approving the console on the engine: modifying the
consoles.xml file or sending a command to the engine’s screen session (if running Linux).
The engine should now have an XML file that will contain the details for the console, provided that the engine service is started and ready for pairing.
The file is located at the following path
<installation_directory>/nse/conf/consoles.xml and will look similar to the following:
<?xml version="1.0" encoding="utf-8" ?> <!-- NeXpose consoles authorized to access the scan engine --> <Consoles> <console id="1" enabled="0" connectTo="0" name="O=Rapid7, CN=Rapid7 Security Console" lastAddress="10.0.0.1" port="40815" plaintext_sharedSecret=""> <cert>redacted</cert> </console> </Consoles>
To enable the console on the engine, there are two things that need to be done:
- Change the value on the
- Restart the engine service.
You can also perform this process without a restart by using the
screen session on a Linux installation. To do so, the following command can be run:
screen -S nexposeconsole -X stuff 'enable console 1^M'
The name of the
screen session for engines is the same as that for consoles. In this command, you’ll note
stuff which may seem weird if you aren’t very familiar with
screen, but it is in fact, a command for screen which essentially pastes the input into our running session.
We’re assuming that the console ID in this case is
1. To be more thorough, we could also parse the output of the
show console screen session console command to enumerate all consoles and IDs available, but unless you have multiple console connecting to the engine, there should only be one with an ID of
Verifying that the engine is successfully paired is fairly straightforward, though we need to look at different information depending on the communication direction. To start we can make a request to the following endpoint:
E -> C communication scenario, validate that an engine with an
address matching our newly paired engine exists.
C -> E, validate that the engine was successfully created during the creation process (
201 response from the API), and that details for the engine in the API response for the above endpoint include a
lastRefreshedDate key and value.
Finally, if your workflow necessitates deleting the scan engine, the following API endpoint can be used, provided that you have the
id for the engine:
If you don’t have the
id, it can be retrieved using the
GET /api/3/scan_engines endpoint.
There’s a lot here; but if you’ve made it this far, you should have everything that you need to get started orchestrating scan engine deployments in your environment!