InsightVM - SCCM Integration Fails

samuel_hernandez · October 27, 2020, 3:18pm

Hello,
Our team is new to using InsightVM and one of the selling points for our team was to ease the automation of patch management. We are currently experiencing and issue with the integration of SCCM and InsightVM.

Things we have done:

Created and activated the orchestration server using the OVF file provided by R7
Added an AD service account with local admin privileges to the primary SCCM server
Added the service account to the SCCM console and granted the following permissions

Collection
Create
Modify
Modify Resource
Read
Read Resource
Remote Control
Software Update Group
Create
Modify
Read
Verified WinRM is listening on ports 5896 and 5895
Tested WinRM with PSEXEC.exe

Excerpt from the error message is:

Connect: Connecting... rapid7/Microsoft SCCM:2.0.4. Step name: add_devices_to_collection Connection test failed! There is likely an issue with the connection details, or the plugin can not communicate via WinRM on the SCCM host. An error occurred while fetching the SCCM site based on site path: <SITE NAME>, please review the error for additional details: the specified credentials were rejected by the server Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/winrm/transport.py", line 278, in _send_message_request response.raise_for_status() File "/usr/local/lib/python3.7/site-packages/requests-2.22.0-py3.7.egg/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: for url: https://<IP>:5986/wsman During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/connection/connection.py", line 26, in test run_script = util.powershell(self, script, self.logger, False) File "/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/util/util.py", line 37, in powershell run_script = powershell_session.run_ps(script) File "/usr/local/lib/python3.7/site-packages/winrm/__init__.py", line 50, in run_ps rs = self.run_cmd('powershell -encodedcommand {0}'.format(encoded_ps)) File "/usr/local/lib/python3.7/site-packages/winrm/__init__.py", line 37, in run_cmd shell_id = self.protocol.open_shell() File "/usr/local/lib/python3.7/site-packages/winrm/protocol.py", line 157, in open_shell res =

We opened a ticket with R7, but they simply said there is a permissions issue. I don’t how we can grant the service account more permissions than local admin and the SCCM permissions in the documentation. We haven’t given the service account domain admin, and I am hopeful that is not required since that would be a major security risk.

Any help would be greatly appreciated.

zyoutz · October 28, 2020, 3:46am

Hey @samuel_hernandez, it has been a while since I’ve set this up from scratch but did you make sure the service account you created is added to the Remote Management Users group on the SCCM host in question? Perhaps even the WinRMRemoteWMIUsers_ might work.

Given the 401 and that the user is not a domain admin (understandable), it’s possible the account can’t connect remotely to the host from the orchestrator since the account isn’t a member of one of those groups. This MS documentation provides a bit more details comparing the two groups: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/dn579255(v=ws.11)?redirectedfrom=MSDN#remote-management-users

samuel_hernandez · October 28, 2020, 2:35pm

Thank you @zyoutz. I’ll try that today and post the results. BRB.

samuel_hernandez · October 28, 2020, 3:28pm

Added the service account to the WinRMRemoteWMIUsers__ but no luck. Same error.

evan_nichols · October 30, 2020, 10:49pm

I ran into this before. Can you try to sign into the site server as your service account and launch powershell from the console? There is a prompt that you must first bypass, otherwise the connection over WinRM will fail.

samuel_hernandez · November 3, 2020, 3:05pm

Thanks for the response Evan. I logged in with the service account and opened powershell but there was no prompt. So I set the powershell execution policy to unrestricted for the service account and the local machine as bypass. Same error

evan_nichols · November 3, 2020, 8:51pm

Did you launch it from within the SCCM console using the “Connect via Windows PowerShell” button in the top left corner? I believe that will be the easiest way to load the PSSnapIn and trigger the prompt.

samuel_hernandez · November 3, 2020, 9:20pm

I just tried it but received the same results. Also verified the service account is in the local administrator group and Remote Management Users. The full error is below:

rapid7/Microsoft SCCM:2.0.4. Step name: add_devices_to_collection Connection test failed! There is likely an issue with the connection details, or the plugin can not communicate via WinRM on the SCCM host. An error occurred while fetching the SCCM site based on site path: <SITE CODE>, please review the error for additional details: the specified credentials were rejected by the server Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/winrm/transport.py", line 278, in _send_message_request response.raise_for_status() File "/usr/local/lib/python3.7/site-packages/requests-2.22.0-py3.7.egg/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: for url: https://<IP>:5986/wsman During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/connection/connection.py", line 26, in test run_script = util.powershell(self, script, self.logger, False) File "/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/util/util.py", line 37, in powershell run_script = powershell_session.run_ps(script) File "/usr/local/lib/python3.7/site-packages/winrm/__init__.py", line 50, in run_ps rs = self.run_cmd('powershell -encodedcommand {0}'.format(encoded_ps)) File "/usr/local/lib/python3.7/site-packages/winrm/__init__.py", line 37, in run_cmd shell_id = self.protocol.open_shell() File "/usr/local/lib/python3.7/site-packages/winrm/protocol.py", line 157, in open_shell res = self.send_message(xmltodict.unparse(req)) File "/usr/local/lib/python3.7/site-packages/winrm/protocol.py", line 234, in send_message resp = self.transport.send_message(message) File "/usr/local/lib/python3.7/site-packages/winrm/transport.py", line 272, in send_message response = self._send_message_request(prepared_request, message) File "/usr/local/lib/python3.7/site-packages/winrm/transport.py", line 282, in _send_message_request raise InvalidCredentialsError("the specified credentials were rejected by the server") winrm.exceptions.InvalidCredentialsError: the specified credentials were rejected by the server During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py", line 311, in handle_step output = self.start_step(input_message['body'], 'action', logger, log_stream, is_test, is_debug) File "/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py", line 421, in start_step output = func() File "/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/connection/connection.py", line 30, in test assistance=f"An error occurred while fetching the SCCM site based on site " komand.exceptions.ConnectionTestException: Connection test failed! There is likely an issue with the connection details, or the plugin can not communicate via WinRM on the SCCM host. An error occurred while fetching the SCCM site based on site path: <SITE CODE>, please review the error for additional details: the specified credentials were rejected by the server

I cannot figure out for the life of me why it’s rejecting the credentials. I have used the credentials from another windows client to connect via Powershell and it’s successful.

evan_nichols · November 6, 2020, 1:48pm

Do the results of Test-WSMan -ComputerName $SiteServer -Credential $SvcCreds -UseSSL look fine? Can you post the results of the following commands?

winrm get winrm/config/service/auth
(Get-Item -Path WSMan:\localhost\Service\RootSDDL).Value

It’s possible the WinRM endpoint defaults are wonky.

samuel_hernandez · November 6, 2020, 3:34pm

I ran the following commands from my computer and not the site server. I also had to add the -Authentication Default flag to the command since I was getting an error without. I pasted the entire output below.

PS C:\WINDOWS\system32> Test-WSMan -ComputerName <Site Server Name> -Credential $SvcCreds -UseSSL
    Test-WSMan : The WinRM client could not process the request because credentials were specified along with the 'no authentication' flag.   No user name, password or client certificate should be specified with the 'no 
    authentication' option. 
    At line:1 char:1
    + Test-WSMan -ComputerName <Site Server Name> -Credential $SvcCreds -UseSSL
    + \~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: ( : ) [Test-WSMan], InvalidOperationException
        + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.TestWSManCommand
 

PS C:\WINDOWS\system32> Test-WSMan -ComputerName <Site Server Name> -Credential $SvcCreds -UseSSL -Authentication Default
Test-WSMan : <f:WSManFault xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="12175" Machine="<My Computer> "><f:Message>The server certificate on the destination computer (<IP>:5986) 
has the following errors:     
The SSL certificate contains a common name (CN) that does not match the hostname.     </f:Message></f:WSManFault>
At line:1 char:1
+ Test-WSMan -ComputerName <Site Server Name> -Credential $SvcCreds -UseSSL  ...
+ \~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (<IP>:String) [Test-WSMan], InvalidOperationException
    + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.TestWSManCommand
 

PS C:\WINDOWS\system32> winrm get winrm/config/service/auth
Auth
    Basic = false [Source="GPO"]
    Kerberos = true
    Negotiate = true
    Certificate = false
    CredSSP = false
    CbtHardeningLevel = Relaxed


PS C:\WINDOWS\system32> (Get-Item -Path WSMan:\localhost\Service\RootSDDL).Value
O:NSG:BAD:P(A;;GA;;;BA)(A;;GXGR;;;S-1-5-21-3142625534-566024925-2387882230-77249)(A;;GR;;;IU)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;;;WD) `

evan_nichols · November 6, 2020, 5:14pm

In your second Test-WSMan - are you defining your Site Server by the FQDN? Connections to WinRM over SSL will fail if the target machine doesn’t match the Common Name value in the certificate you’ve bound to the WinRM endpoint.

The SDDL looks fine though - BUILTIN\Administrators has Full Control - I have seen in the past where a GPO has mucked with that.

samuel_hernandez · November 6, 2020, 10:50pm

I was using the FQDN. What I have done is to recreate the self signed certificate using the proper CN name (someone else created it). Now I am getting this error message:

PS C:\WINDOWS\system32> Test-WSMan -ComputerName "<SITE SERVER FQDN>" -Credential $SvcCreds -UseSSL -Authentication Default
Test-WSMan : <f:WSManFault xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="12175" Machine="<MY COMPUTER>"><f:Message>The server certificate on the destination computer (<SITE SERVER FQDN>:5986) has the following errors:    
The SSL certificate is signed by an unknown certificate authority.      </f:Message></f:WSManFault>
At line:1 char:1
+ Test-WSMan -ComputerName "<SITE SERVER FQDN>" -Credential $SvcCred ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (<SITE SERVER FQDN>):String) [Test-WSMan], InvalidOperationException
    + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.TestWSManCommand

evan_nichols · November 7, 2020, 12:40am

Getting closer - if you export the SSL cert that is bound to WinRM on the Site Server, and install it in the Trusted Root Certification Authority store on the machine you are running Test-WSMan from, you shouldn’t have that error anymore. I believe you’d need to follow the instructions found here in order to have the SCCM plugin trust your self-signed cert.

samuel_hernandez · November 9, 2020, 4:18pm

Over the weekend, I exported the cert from the SCCM server and installed it on the Windows client to test WinRM using the Test-WSMan command you suggested and that worked fine. Followed the instructions from R7 to install a private CA. I installed the cert of the orchestration server and restarted the orchestrator, but I get the same exact error.

evan_nichols · November 13, 2020, 11:37pm

Drat. Does the output of Get-WSManInstance -ResourceURI winrm/config/Listener -SelectorSet @{Address='*';Transport='HTTPS'} look fine?

I deploy my HTTPS listeners with this script and it works reliably. You might try to delete/recreate the listener using this.

I was trying to find out whether or not the plugin relied on Kerberos or NTLM, but I can’t seem to find the source code on the git repo. It may be using Kerberos, and if I remember correctly that requires a principal to boost tickets. Just spitballing, but the plugin may have failed to create that object. I think you might want to open a ticket in this case.

samuel_hernandez · January 8, 2021, 9:58pm

Finally, I have some good news. The configuration was right and permissions were correct. It looks like the credentials were not properly synchronizing from the cloud side of InsightVM to the orchestration server so we simply deleted the credentials in the InsightVM console and recreated them. After that, we initiated a test, and voila! Thanks for everyone’s help on this forum.

holly_wilsey · January 11, 2021, 2:52pm

Glad you got it figured out! I’ll go ahead and mark this as the solution for now, in case anyone else runs into something like this. Kudos to you and @evan_nichols for working through it!

srafaliya · March 31, 2022, 6:24pm

R7 - SCCM connection, I am seeing below error in R7 log. Can you guys please help.

Connect: Connecting…
rapid7/Microsoft SCCM:2.0.4. Step name: add_devices_to_collection
Connection test failed!

There is likely an issue with the connection details, or the plugin can not communicate via WinRM on the SCCM host. An error occurred while fetching the SCCM site based on site path: PWC, please review the error for additional details: HTTPSConnectionPool(host=‘x.x.x.’, port=5985): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, ‘[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1056)’)))
Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/connectionpool.py”, line 603, in urlopen
chunked=chunked)
File “/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/connectionpool.py”, line 344, in _make_request
self._validate_conn(conn)
File “/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/connectionpool.py”, line 843, in validate_conn
conn.connect()
File “/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/connection.py”, line 370, in connect
ssl_context=context)
File "/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/util/ssl.py", line 368, in ssl_wrap_socket
return context.wrap_socket(sock)
File “/usr/local/lib/python3.7/ssl.py”, line 412, in wrap_socket
session=session
File “/usr/local/lib/python3.7/ssl.py”, line 853, in _create
self.do_handshake()
File “/usr/local/lib/python3.7/ssl.py”, line 1117, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1056)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/requests-2.22.0-py3.7.egg/requests/adapters.py”, line 449, in send
timeout=timeout
File “/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/connectionpool.py”, line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File “/usr/local/lib/python3.7/site-packages/urllib3-1.25.3-py3.7.egg/urllib3/util/retry.py”, line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=‘x.x.x.’, port=5985): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, ‘[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1056)’)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/connection/connection.py”, line 26, in test
run_script = util.powershell(self, script, self.logger, False)
File “/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/util/util.py”, line 37, in powershell
run_script = powershell_session.run_ps(script)
File “/usr/local/lib/python3.7/site-packages/winrm/init.py”, line 50, in run_ps
rs = self.run_cmd(‘powershell -encodedcommand {0}’.format(encoded_ps))
File “/usr/local/lib/python3.7/site-packages/winrm/init.py”, line 37, in run_cmd
shell_id = self.protocol.open_shell()
File “/usr/local/lib/python3.7/site-packages/winrm/protocol.py”, line 157, in open_shell
res = self.send_message(xmltodict.unparse(req))
File “/usr/local/lib/python3.7/site-packages/winrm/protocol.py”, line 234, in send_message
resp = self.transport.send_message(message)
File “/usr/local/lib/python3.7/site-packages/winrm/transport.py”, line 272, in send_message
response = self._send_message_request(prepared_request, message)
File “/usr/local/lib/python3.7/site-packages/winrm/transport.py”, line 277, in _send_message_request
response = self.session.send(prepared_request, timeout=self.read_timeout_sec)
File “/usr/local/lib/python3.7/site-packages/requests-2.22.0-py3.7.egg/requests/sessions.py”, line 646, in send
r = adapter.send(request, **kwargs)
File “/usr/local/lib/python3.7/site-packages/requests-2.22.0-py3.7.egg/requests/adapters.py”, line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host=‘x.x.x.’, port=5985): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, ‘[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1056)’)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py”, line 311, in handle_step
output = self.start_step(input_message[‘body’], ‘action’, logger, log_stream, is_test, is_debug)
File “/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py”, line 421, in start_step
output = func()
File “/usr/local/lib/python3.7/site-packages/microsoft_sccm_rapid7_plugin-2.0.4-py3.7.egg/komand_microsoft_sccm/connection/connection.py”, line 30, in test
assistance=f"An error occurred while fetching the SCCM site based on site "
komand.exceptions.ConnectionTestException: Connection test failed!

There is likely an issue with the connection details, or the plugin can not communicate via WinRM on the SCCM host. An error occurred while fetching the SCCM site based on site path: PWC, please review the error for additional details: HTTPSConnectionPool(host=‘x.x.x.’, port=5985): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, ‘[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1056)’)))

jfigueroa1 · September 20, 2022, 9:44pm

Hi there,
I experienced an issue where SCCM integration kept failing indicating the site path was incorrect.
To resolve this issue, our service account needed local admin access.
After that, we needed to log into the SCCM console using the service account. Probably needed to generate some cache.
Went back to R7 SCCM Integration, test the integration and it passed.