AlertLogic via Python No Longer Authenticating

InsightConnect Workflows
1

Hi, we’ve had a workflow running for quite some time that queries AlertLogic for any new incidents and runs through some steps to block IPs and create tickets. I noticed when I was checking Connect this morning that we had a high number of failed jobs from that workflow, and that it didn’t appear to be completing any at all when I checked the jobs menu.

The step that is failing is Python, and it’s returning this error:
Exception: Could not run supplied script. Error: HTTPSConnectionPool(host=‘api.alertlogic.net’, port=443): Max retries exceeded with url: /api/incident/v3/incidents?customer_id=all_children&create_date=1565395200…1565913600&fields=acknowledge_status,attackers,class_name,closed_type,devices,victims,threat_rating,summary,incident_id (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe249d40470>: Failed to establish a new connection: [Errno -2] Name or service not known’))

We haven’t made any changes to that workflow that would cause this, so the only thing I can think that’s changed is something about the API on AlertLogic’s end. However I’m unable to find any documentation that indicates the API version has undergone any recent changes. Any thoughts on this?

1 Like
2

@valen_arnette - can you please let us know the plugin and the version that you are using.

1 Like
3

It’s the Python 3 Script plugin, version 2.0.4.

4

Hi @valen_arnette - that type of exception is usually thrown when the request to the URL/service is invalid. My best guess is that something about the AlertLogic API has changed which is now causing this.

Can you share some documentation about the API you’re trying to work with, as well as some information about your Python 3 Script configuration (and possibly the actual Python code you’re using)? Myself or someone else may be able to help you get back up and running!

5

Hey @mike_rinehart - that was definitely my thinking as well. I did a little more digging and did actually find an article that AlertLogic posted about their old incident API being migrated on the 1st of this month. First I’d heard of it, but I guess it means we have no choice but to migrate now.

Looks like this is their new API for querying incidents: API Documentation

And this is the script we’ve been running:

Python Code
def run(params):
import json
import requests
start = params[‘start’]
end = params[‘end’]
l = []
al_api_key = “*”
headers = { ‘Accept’: ‘application/json’ }
url = “https://api.alertlogic.net/api/incident/v3/incidents?customer_id=all_children&create_date=” + str(start) +"…" + str(end) + “&fields=acknowledge_status,attackers,class_name,closed_type,devices,victims,threat_rating,summary,incident_id”
#url = “http://10.0.2.233:3488/alertlogic/api/incident/v3/incidents?customer_id=all_children&create_date=” + str(start) +"…" + str(end) + “&fields=acknowledge_status,attackers,class_name,closed_type,devices,victims,threat_rating,summary,incident_id”
s = requests.get(url,auth=(al_api_key,’’), headers=headers, timeout=(3.05, 120))
s = s.content.decode(‘utf-8’).replace(’\n’,’’)
obj = json.loads(s)
#process json into object
for i in obj:
incident = {}
incident[‘acknowledge_status’]=i.get(“acknowledge_status”,"")
incident[‘attackers’]=i.get(“attackers”,[])
incident[‘class_name’]=i.get(“class_name”,"")
incident[‘closed_type’]=i.get(“closed_type”,0)
incident[‘devices’]=i.get(“devices”,[])
incident[‘victims’]=i.get(“victims”,[])
incident[‘threat_rating’]=i.get(“threat_rating”,"")
incident[‘summary’]=i.get(“summary”,"")
incident[‘incident_id’]=i.get(“incident_id”,-1)
l.append(incident)
#audit attacker/victim - look for “private” attackers
newl = []
for i in l:
for ip in i[‘attackers’]:
ipsplitt = ip.split(".")
ipsplit = []
for temp in ipsplitt:
try:
ipsplit.append(int(temp))
except:
#must be ipv6 - don’t check for private ip addresses
ipsplit = ipsplitt
continue
if(len(ipsplit)==4):
if(ipsplit[0] == 10 or (ipsplit[0] == 192 and ipsplit[1] == 168) or (ipsplit[0] == 172 and (ipsplit[1] in [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]))):
#attacker from private, is victim public?
for ipv in i[‘victims’]:
ipvsplitt = ip.split(".")
ipvsplit = []
for tempv in ipvsplitt:
try:
ipvsplit.append(int(tempv))
except:
#must be ipv6 - don’t check for private ip addresses
continue
if(len(ipvsplit)==4):
if(ipvsplit[0] == 10 or (ipvsplit[0] == 192 and ipvsplit[1] == 168) or (ipvsplit[0] == 172 and (ipvsplit[1] in [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]))):
pass
else:
#attacker and victim reversed
i[‘attackers’].remove(i)
i[‘attackers’].append(ipv)
newl.append(i)
result = {}
result[‘incident_list’]=newl
return result
print(run({“start”:“1565395200”,“end”:“1565913600”}))

I’m no Python guru, so I’d definitely appreciate the help with trying to get this functional again!

6

Hi Valen - I don’t have an AlertLogic test environment to test this against, but looking at their API documentation you provided as well as your example script, I believe this code (below) should work for the API call portion of your code:

headers = {"Accept": "application/json",
           "x-aims-auth-token": "<your API token here>"}

url = "https://api.cloudinsight.alertlogic.com/iris/v3/<your account ID here>/<your incident ID here>"

# query_string is used to limit the returned results to just a subset
query_string = {"return_value": "acknowledge_status,"
                                "attackers,"
                                "class_name,"
                                "closed_type,"
                                "devices,"
                                "victims,"
                                "threat_rating,"
                                "summary,"
                                "incident_id"}

s = requests.get(url,
                 params=query_string,
                 headers=headers,
                 timeout=(3.05, 120))

results = s.json()
example_value_from_the_results = results.get("some_example_value")

I believe you can use that to replace the following lines of your snippet:

al_api_key = “*”
headers = { ‘Accept’: ‘application/json’ }
url = “https://api.alertlogic.net/api/incident/v3/incidents?customer_id=all_children&create_date=” + str(start) +"…" + str(end) + “&fields=acknowledge_status,attackers,class_name,closed_type,devices,victims,threat_rating,summary,incident_id”
#url = “http://10.0.2.233:3488/alertlogic/api/incident/v3/incidents?customer_id=all_children&create_date=” + str(start) +"…" + str(end) + “&fields=acknowledge_status,attackers,class_name,closed_type,devices,victims,threat_rating,summary,incident_id”
s = requests.get(url,auth=(al_api_key,’’), headers=headers, timeout=(3.05, 120))
s = s.content.decode(‘utf-8’).replace(’\n’,’’)
obj = json.loads(s)
#process json into object
for i in obj:
incident = {}
incident[‘acknowledge_status’]=i.get(“acknowledge_status”,"")
incident[‘attackers’]=i.get(“attackers”,[])
incident[‘class_name’]=i.get(“class_name”,"")
incident[‘closed_type’]=i.get(“closed_type”,0)
incident[‘devices’]=i.get(“devices”,[])
incident[‘victims’]=i.get(“victims”,[])
incident[‘threat_rating’]=i.get(“threat_rating”,"")
incident[‘summary’]=i.get(“summary”,"")
incident[‘incident_id’]=i.get(“incident_id”,-1)
l.append(incident)

One thing I did notice, however, is that your code seems to be reliant upon retrieving multiple incidents(?) in a single API call - however, the API call documentation you linked me to seems to only return a single incident now. You may be able to use this API call instead, with a limited time scope, to get multiple incidents.

So, give this a shot and let me know how it goes! Good luck!

7

Hey @mike_rinehart, thanks for that. Does this look logical based on the above? Tried running it a few times and been getting various errors, so it’s pretty likely I botched something somewhere. Thoughts on this?

def run(params):
	import json
	import requests
	start = params['start']
	end = params['end']
	l = []
headers = {"Accept": "application/json",
           "x-aims-auth-token": "<API key>"}

url = "https://api.cloudinsight.alertlogic.com/iris/v3/<Org Code>/incidents_by_time?start=1577836800&end=1577923200"

# query_string is used to limit the returned results to just a subset
query_string = {"return_value": "acknowledge_status,"
                                "attackers,"
                                "class_name,"
                                "closed_type,"
                                "devices,"
                                "victims,"
                                "threat_rating,"
                                "summary,"
                                "incident_id"}

s = requests.get(url,
                 params=query_string,
                 headers=headers,
                 timeout=(3.05, 120))

results = s.json()
example_value_from_the_results = results.get("some_example_value")
newl = []
for i in l:
	for ip in i['attackers']:
		ipsplitt = ip.split(".")
		ipsplit = []
		for temp in ipsplitt:
			try:
				ipsplit.append(int(temp))
			except:
				#must be ipv6 - don't check for private ip addresses
				ipsplit = ipsplitt
				continue
		if(len(ipsplit)==4):
			if(ipsplit[0] == 10 or (ipsplit[0] == 192 and ipsplit[1] == 168) or (ipsplit[0] == 172 and (ipsplit[1] in [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]))):
				#attacker from private, is victim public?
				for ipv in i['victims']:
					ipvsplitt = ip.split(".")
					ipvsplit = []
					for tempv in ipvsplitt:
						try:
							ipvsplit.append(int(tempv))
						except:
							#must be ipv6 - don't check for private ip addresses
							continue
						if(len(ipvsplit)==4):
							if(ipvsplit[0] == 10 or (ipvsplit[0] == 192 and ipvsplit[1] == 168) or (ipvsplit[0] == 172 and (ipvsplit[1] in [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]))):
								pass
							else:
								#attacker and victim reversed
								i['attackers'].remove(i)
								i['attackers'].append(ipv)
	newl.append(i)
result = {}
result['incident_list']=newl
return result



print(run({"start":"1565395200","end":"1565913600"}))
8

Hi Valen - you’re welcome! I’ve made a few adjustments to your code, specifically in the request portion of it. Their API documentation isn’t too specific on what some of the outputs included in the response look like, so unfortunately I can’t help too much past making the request here.

If you can include a sanitized sample of the output then I can help a bit further!

Here is the adjusted code:

def run(params):
    import requests
    start = params['start']
    end = params['end']

    # Setup authentication headers for the request
    headers = {"Accept": "application/json",
               "x-aims-auth-token": "<API key>"}

    # Define the base URL for the API call. Don't include querystring here, since this will be included below
    url = "https://api.cloudinsight.alertlogic.com/iris/v3/<Org Code>/incidents_by_time"

    # query_string is used to limit the returned results to just a subset.
    query_string = {
        "start": 1577836800,  # Replace with an input param if needed
        "end": 1577923200,  # Replace with an input param if needed
        "return_value": "acknowledge_status,"
                        "attackers,"
                        "class_name,"
                        "closed_type,"
                        "devices,"
                        "victims,"
                        "threat_rating,"
                        "summary,"
                        "incident_id"}

    # Send the API request with the URL, authentication headers, query string parameters, and timeout
    s = requests.get(url,
                     params=query_string,
                     headers=headers,
                     timeout=(3.05, 120))

    # Get the entire response, as a dictionary, from the API response
    results = s.json()
    newl = []

    # The API returns a list of incidents, so this code will now loop through those incidents (for result in results)
    for result in results:

        # Valen - here is where you may want to use a tool like PostMan (https://www.postman.com/)
        # to make a sample request to their API and then examine the output. It's not very clear from their
        # documentation on what the response all contains, so looking at real output and then manipulating/extracting
        # the data based off that may work better.
        for ip in result['attackers']:
            ipsplitt = ip.split(".")
            ipsplit = []
            for temp in ipsplitt:
                try:
                    ipsplit.append(int(temp))
                except:
                    # must be ipv6 - don't check for private ip addresses
                    ipsplit = ipsplitt
                    continue
            if (len(ipsplit) == 4):
                if (ipsplit[0] == 10 or (ipsplit[0] == 192 and ipsplit[1] == 168) or (ipsplit[0] == 172 and (
                        ipsplit[1] in [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]))):
                    # attacker from private, is victim public?
                    for ipv in result['victims']:
                        ipvsplitt = ip.split(".")
                        ipvsplit = []
                        for tempv in ipvsplitt:
                            try:
                                ipvsplit.append(int(tempv))
                            except:
                                # must be ipv6 - don't check for private ip addresses
                                continue
                            if (len(ipvsplit) == 4):
                                if (ipvsplit[0] == 10 or (ipvsplit[0] == 192 and ipvsplit[1] == 168) or (
                                        ipvsplit[0] == 172 and (
                                        ipvsplit[1] in [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
                                                        31]))):
                                    pass
                                else:
                                    # attacker and victim reversed
                                    result['attackers'].remove(result)
                                    result['attackers'].append(ipv)
        newl.append(result)
    result = {}
    result['incident_list'] = newl
    return result


print(run({"start": "1565395200", "end": "1565913600"}))
9

@mike_rinehart much appreciated! I tried that adjusted code and this was the log output:

rapid7/Python 3 Script:2.0.4. Step name: run
Input: (below)

{'end': 1646370000, 'start': 1646283600}

Function: (below)

def run(params):
    import requests
    start = params['start']
    end = params['end']

    # Setup authentication headers for the request
    headers = {"Accept": "application/json",
               "x-aims-auth-token": "<API Key>"}

    # Define the base URL for the API call. Don't include querystring here, since this will be included below
    url = "https://api.cloudinsight.alertlogic.com/iris/v3/<Org Code>/incidents_by_time"

    # query_string is used to limit the returned results to just a subset.
    query_string = {
        "start": 1577836800,  # Replace with an input param if needed
        "end": 1577923200,  # Replace with an input param if needed
        "return_value": "acknowledge_status,"
                        "attackers,"
                        "class_name,"
                        "closed_type,"
                        "devices,"
                        "victims,"
                        "threat_rating,"
                        "summary,"
                        "incident_id"}

    # Send the API request with the URL, authentication headers, query string parameters, and timeout
    s = requests.get(url,
                     params=query_string,
                     headers=headers,
                     timeout=(3.05, 120))

    # Get the entire response, as a dictionary, from the API response
    results = s.json()
    newl = []

    # The API returns a list of incidents, so this code will now loop through those incidents (for result in results)
    for result in results:

        # Valen - here is where you may want to use a tool like PostMan (https://www.postman.com/)
        # to make a sample request to their API and then examine the output. It's not very clear from their
        # documentation on what the response all contains, so looking at real output and then manipulating/extracting
        # the data based off that may work better.
        for ip in result['attackers']:
            ipsplitt = ip.split(".")
            ipsplit = []
            for temp in ipsplitt:
                try:
                    ipsplit.append(int(temp))
                except:
                    # must be ipv6 - don't check for private ip addresses
                    ipsplit = ipsplitt
                    continue
            if (len(ipsplit) == 4):
                if (ipsplit[0] == 10 or (ipsplit[0] == 192 and ipsplit[1] == 168) or (ipsplit[0] == 172 and (
                        ipsplit[1] in [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]))):
                    # attacker from private, is victim public?
                    for ipv in result['victims']:
                        ipvsplitt = ip.split(".")
                        ipvsplit = []
                        for tempv in ipvsplitt:
                            try:
                                ipvsplit.append(int(tempv))
                            except:
                                # must be ipv6 - don't check for private ip addresses
                                continue
                            if (len(ipvsplit) == 4):
                                if (ipvsplit[0] == 10 or (ipvsplit[0] == 192 and ipvsplit[1] == 168) or (
                                        ipvsplit[0] == 172 and (
                                        ipvsplit[1] in [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
                                                        31]))):
                                    pass
                                else:
                                    # attacker and victim reversed
                                    result['attackers'].remove(result)
                                    result['attackers'].append(ipv)
        newl.append(result)
    result = {}
    result['incident_list'] = newl
    return result


print(run({"start": "1565395200", "end": "1565913600"}))

Could not run supplied script. Error: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/python_3_script_rapid7_plugin-2.0.4-py3.7.egg/komand_python_3_script/actions/run/action.py", line 20, in run
    exec(func)  # noqa: B102
  File "<string>", line 83, in <module>
  File "<string>", line 34, in run
  File "/usr/local/lib/python3.7/site-packages/requests-2.22.0-py3.7.egg/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py", line 311, in handle_step
    output = self.start_step(input_message['body'], 'action', logger, log_stream, is_test, is_debug)
  File "/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py", line 419, in start_step
    output = func(params)
  File "/usr/local/lib/python3.7/site-packages/python_3_script_rapid7_plugin-2.0.4-py3.7.egg/komand_python_3_script/actions/run/action.py", line 24, in run
    raise Exception("Could not run supplied script. Error: " + str(e))
Exception: Could not run supplied script. Error: Expecting value: line 1 column 1 (char 0)

I was comparing it to some of the old Python outputs and I think it wasn’t seeing the tab usage in certain places in the input when I copied and pasted. I went through and put it all in by hand and got this error back;

Could not run supplied script. Error: 'return' outside function (<string>, line 60)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/python_3_script_rapid7_plugin-2.0.4-py3.7.egg/komand_python_3_script/actions/run/action.py", line 20, in run
    exec(func)  # noqa: B102
  File "<string>", line 60
SyntaxError: 'return' outside function

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py", line 311, in handle_step
    output = self.start_step(input_message['body'], 'action', logger, log_stream, is_test, is_debug)
  File "/usr/local/lib/python3.7/site-packages/komand-1.0.1-py3.7.egg/komand/plugin.py", line 419, in start_step
    output = func(params)
  File "/usr/local/lib/python3.7/site-packages/python_3_script_rapid7_plugin-2.0.4-py3.7.egg/komand_python_3_script/actions/run/action.py", line 24, in run
    raise Exception("Could not run supplied script. Error: " + str(e))
Exception: Could not run supplied script. Error: 'return' outside function (<string>, line 60)
10

@valen_arnette Ah, yes, your Python script code cannot include a print function. Remove that and you should be good to go!

11

@mike_rinehart removed that print statement at the end and looks like I’m still getting hit with that second error about the ‘return’ outside the function. Anything else you think I should add or remove that could fix that?

12

In re-reading your code snippet, I think it is actually the results = s.json() (based on the json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)) causing the issue - it looks like it’s returning a non-JSON response. Have you tried making this request with cURL or Postman to ensure it works fine outside of the Python code?