Orchestrating InsightVM Scan Engine Deployment

While it’s still fairly common for InsightVM scanning infrastructure to be static, i.e deployed once with new scan engines provisioned in an ad-hoc, manual fashion, we also hear from plenty of customers that they would like to orchestrate this process. Orchestration of scan engine infrastructure can reduce the administrative overhead of deploying scan engines, and even enable workflows such as temporary engine provisioning prior to a scan and termination after the scan completes. This is especially handy if you’re trying to scan cloud infrastructure while keeping costs low.

For this post, I’ll cover the steps involved with orchestrating the engine deployment life cycle; however, we won’t provide a concrete example. Everyone is using different technology for managing infrastructure orchestration, so the goal of this post is to outline this workflow and provide a starting point if this is something that you’re interested in doing. Some of the concepts will be specific to Linux installation, so I’ll call that out where applicable.

Workflow

There are two pairing methods, Engine to Console and Console to Engine. We’ll cover both and try to note when a step is specific to one or the other. For the sake of brevity, they’ll be abbreviated E -> C and C -> E. Below are the steps that we’ll need to manage the entire engine deployment lifecycle, from installation to termination (if desired).

  1. Get the installer
  2. E -> C: Generate a Shared Secret
  3. Configure a varfile for the install
    1. This allows for passing options to the installer
  4. Run the installer
  5. C -> E: Add the scan engine to the console
  6. C -> E: Approve the console on the engine
  7. Validate the engine is available in the security console
  8. Delete the engine from the console (when scanning is complete or it is no longer needed)

Let’s get started.

Get the Installer

Possibly the easiest part of this process, all we have to do to get the installer is download the correct binary for our operating system and validate the checksum. Binaries and checksums for each supported platform are available here.

You could use wget or Powershell to download these, or use whatever capabilities are provided by the toolset that you’re using.

Generate a Shared Secret

Only required for E → C communication

To generate a shared secret, the InsightVM API (v3) provides the following endpoint:

  • POST /api/3/scan_engines/shared_secret

Which will return the current shared secret or generate a new one.

Note:

The public documentation at InsightVM API (v3) isn’t fully up to date, so this endpoint isn’t documented there. However, if you go to the documentation within the Security Console via help menu in the top menu bar, you will find full documentation for this endpoint and others associated with scan engine shared secrets.

Additionally, the API user will need to be a Global Admin in order to generate the secret.

Configure the Varfile for Install

The installer allows for a variable file to be passed so that the installation can be performed in an unattended fashion. The parameters that need to be added to a variable file named as follows for each platform:

  • Linux: Rapid7Setup-Linux64.varfile
  • Windows: Rapid7Setup-Windows64.varfile

Below is an example of this file with comments detailing changes required for each scenario:

Example Engine varfile
# Rapid7 Engine Varfile
# 0 for E -> C, 1 for C -> E
communicationDirectionChoice$Integer=0

# nse for engines, nsc for consoles
component=nse

createDesktopLinkAction$Boolean=false
dbPort=5432
company=<your company>
firstname=<support contact name>
lastname=<support contact name>
port=5432
proxyHost=
proxyPort=
registrationRequired$Integer=1
requirePasswordChange$Boolean=False
sys.adminRights$Boolean=true

# True for engines, False for consoles
sys.component.engine$Boolean=True

# False for engines, True for consoles
sys.component.typical$Boolean=False

# C:\\Program\ Files\\rapid7\\nexpose on Windows
sys.installationDir=/opt/rapid7/nexpose

sys.languageId=en

# False on Windows to create the application menu
sys.programGroupDisabled$Boolean=True

initService$Boolean=True

# omit for C -> E
consoleAddress=<console IP/hostname>
consoleDetailPort=40815
sharedSecret=<shared secret generated by the API>

Note:

Fields with <some_value> need to be replaced with the correct values.

The names of the files are important, as without the appropriate names the installer will not read from the file.

As an alternative to using the sample file provided here, you can manually run the installer once. Once the installer is run, it will create a <installation_directory>/.install4j/response.varfile file containing all of the answers from the manual installation that could be used as the template for your engine installations.

Run the Installer

With the varfile ready, the installer can be run. On Linux, this looks like the following:

<path_to_installer>/Rapid7Setup-Linux64.bin -q -varfile <path_to_file>/Rapid7Setup-Linux64.varfile

On Windows it’s similar:

<path_to_installer>\Rapid7Setup-Windows64.bin -q -varfile <path_to_file>\Rapid7Setup-Windows64.varfile

As part of the installation, there is quite a lot of output. It’s recommended to review this output as part of your orchestration process to validate that the installation was successful.

Add the Engine to the Console

If you’re using the E -> C pairing direction, the scan engine should be ready to go once the service initializes (you can skip to Verify The Engine is Paired).

The V3 API provides the following endpoint to add a scan engine:

  • POST /api/3/scan_engines

This will add the engine to the console; however, you’ll still need to approve the console on the engine, which we’ll cover next.

Approve the Console on the Engine

After you have added the engine to the console, there are two ways to go about approving the console on the engine: modifying the consoles.xml file or sending a command to the engine’s screen session (if running Linux).

Modifying the consoles.xml File

The engine should now have an XML file that will contain the details for the console, provided that the engine service is started and ready for pairing.

The file is located at the following path <installation_directory>/nse/conf/consoles.xml and will look similar to the following:

Example consoles.xml
<?xml version="1.0" encoding="utf-8" ?>
<!-- NeXpose consoles authorized to access the scan engine -->
<Consoles>
  <console id="1" enabled="0" connectTo="0" name="O=Rapid7, CN=Rapid7 Security Console" lastAddress="10.0.0.1" port="40815" plaintext_sharedSecret="">
    <cert>redacted</cert>
  </console>
</Consoles>

To enable the console on the engine, there are two things that need to be done:

  1. Change the value on the enabled attribute to ”1”
  2. Restart the engine service.

Approving via the Engine’s screen Session

You can also perform this process without a restart by using the screen session on a Linux installation. To do so, the following command can be run:

screen -S nexposeconsole -X stuff 'enable console 1^M'

The name of the screen session for engines is the same as that for consoles. In this command, you’ll note stuff which may seem weird if you aren’t very familiar with screen, but it is in fact, a command for screen which essentially pastes the input into our running session.

We’re assuming that the console ID in this case is 1. To be more thorough, we could also parse the output of the show console screen session console command to enumerate all consoles and IDs available, but unless you have multiple console connecting to the engine, there should only be one with an ID of 1.

Verify the Engine is Paired

Verifying that the engine is successfully paired is fairly straightforward, though we need to look at different information depending on the communication direction. To start we can make a request to the following endpoint:

  • GET /api/3/scan_engines

In the E -> C communication scenario, validate that an engine with an address matching our newly paired engine exists.

For C -> E, validate that the engine was successfully created during the creation process (201 response from the API), and that details for the engine in the API response for the above endpoint include a lastRefreshedDate key and value.

Delete the Engine from the Console

Finally, if your workflow necessitates deleting the scan engine, the following API endpoint can be used, provided that you have the id for the engine:

  • DELETE /api/3/scan_engines/{id}

If you don’t have the id, it can be retrieved using the GET /api/3/scan_engines endpoint.


There’s a lot here; but if you’ve made it this far, you should have everything that you need to get started orchestrating scan engine deployments in your environment!

8 Likes

Are all of the available options for the varfile documented anywhere?

1 Like

Did you find the documentation for the varfile? :slight_smile: