Python Step - Inputs and TypeError: Object of type set is not JSON serializable

[tward1 (tward)]

Hey there,

The error I am getting is TypeError: Object of type set is not JSON serializable

I have a step where I am passing 2 json arrays into a python step where i need to join the

My input -
{“solutions”:“["Convert Solution Query Results To JSON"].[json]”,“vulnerabilities”:“["Get-Adjusted-Vulns"].[items]”}

My function
def run(params={}):
Vulnerabilities1 = params.get(‘vulnerabilities’)
#solutions1 = params.get(‘solutions’)
json_string_variable = ‘{“name”: “Alice”, “age”: 30, “city”: “New York”}’
return {json_string_variable}

Obviously I am just trying to make sure the inputs and outputs work.

When testing the workflow, the input for the python step is:
{
“function”: “def run(params={}):\n Vulnerabilities1 = params.get(‘vulnerabilities’)\n #solutions1 = params.get(‘solutions’)\n json_string_variable = ‘{"name": "Alice", "age": 30, "city": "New York"}’\n return {json_string_variable}”,
“input”: {
“solutions”: “["Convert Solution Query Results To JSON"].[json]”,
“vulnerabilities”: “["Get-Adjusted-Vulns"].[items]”
},
“timeout”: 30
}

It seems to me if it was the json instead of the strings of the variables i would see JSON which means the value of the variable isn’t being passed in. What am i going wrong?

The error:
Traceback (most recent call last):
File “/workspace/.py”, line 28, in
sys.stdout.write(“” + json.dumps(run({‘solutions’: ‘[“Convert Solution Query Results To JSON”].[json]’, ‘vulnerabilities’: ‘[“Get-Adjusted-Vulns”].[items]’})))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/layers/paketo-buildpacks_cpython/cpython/lib/python3.12/json/init.py”, line 231, in dumps
return _default_encoder.encode(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/layers/paketo-buildpacks_cpython/cpython/lib/python3.12/json/encoder.py”, line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/layers/paketo-buildpacks_cpython/cpython/lib/python3.12/json/encoder.py”, line 258, in iterencode
return _iterencode(o, 0)
^^^^^^^^^^^^^^^^^
File “/layers/paketo-buildpacks_cpython/cpython/lib/python3.12/json/encoder.py”, line 180, in default
raise TypeError(f’Object of type {o.class.name} ’
TypeError: Object of type set is not JSON serializable

[Eric-Wilson]

Couple of things I noticed.

1. If you are feeding arrays into the input you don't want to put quotes around them otherwise the python step will interpret that as a string. Your input should look like this:

{
    "vulnerabilities": {{[$input].[vulnerabilities]}},
    "solutions": {{[$input].[solutions]}}
}

2. Your return needs to be in json format. A simple script that would combine two arrays on a specific key looks like this:

def run(params={}):
    # Get the arrays from input parameters
    vulnerabilities = params.get("vulnerabilities", [])
    solutions = params.get("solutions", [])

    # Convert solutions list into a dictionary for fast lookup by id
    solution_lookup = {item.get("id"): item for item in solutions}

    merged_results = []

    # Merge vulnerabilities with solutions based on the shared "id" field
    for vuln in vulnerabilities:
        vid = vuln.get("id")
        matching_solution = solution_lookup.get(vid, {})

        combined = {
            "id": vid,
            "severity": vuln.get("severity"),
            "description": vuln.get("description"),
            "solution": matching_solution.get("solution"),
            "eta_days": matching_solution.get("eta_days")
        }

        merged_results.append(combined)

    return {
        "merged": merged_results
    }

This is based off my two sample arrays below:

{
  "vulnerabilities": [
    {
      "id": "VULN-001",
      "severity": "high",
      "description": "Remote code execution vulnerability in service XYZ."
    },
    {
      "id": "VULN-002",
      "severity": "medium",
      "description": "Sensitive information disclosure due to misconfiguration."
    },
    {
      "id": "VULN-003",
      "severity": "low",
      "description": "Deprecated protocol support detected."
    }
  ],
  "solutions": [
    {
      "id": "VULN-001",
      "solution": "Apply patch KB12345 and restart the service.",
      "eta_days": 3
    },
    {
      "id": "VULN-002",
      "solution": "Update configuration to disable public access.",
      "eta_days": 1
    },
    {
      "id": "VULN-003",
      "solution": "Upgrade to version 2.5 or later.",
      "eta_days": 7
    }
  ]
}

Combined arrays from python output:

{
  "$success": true,
  "merged": [
    {
      "description": "Remote code execution vulnerability in service XYZ.",
      "eta_days": 3,
      "id": "VULN-001",
      "severity": "high",
      "solution": "Apply patch KB12345 and restart the service."
    },
    {
      "description": "Sensitive information disclosure due to misconfiguration.",
      "eta_days": 1,
      "id": "VULN-002",
      "severity": "medium",
      "solution": "Update configuration to disable public access."
    },
    {
      "description": "Deprecated protocol support detected.",
      "eta_days": 7,
      "id": "VULN-003",
      "severity": "low",
      "solution": "Upgrade to version 2.5 or later."
    }
  ]
}

[Eric-Wilson]

Additionally after working through your python questions it dawned on me that you could replicate the input from the python step into the jq step. I've got it set up like below.
Input:

{
  "vulnerabilities": {{[$input].[vulnerabilities]}},
  "solutions": {{[$input].[solutions]}}
}

filter:

{
  merged:
    (
      . as $root
      | [
          $root.vulnerabilities[]
          as $v
          | $v + (
              ($root.solutions[] | select(.id == $v.id)) // {}
            )
        ]
    )
}

Result:

[tward1 (tward)]

good morning, totally appreciate the jq example as well and will probably go that route but i want to get a python step working as an example since i need to use it it for the jira api steps.

For the input I had to quote the variable/step name, the final looking like

{
"vulnerabilities": {{["Get-Adjusted-Vulns"].[items]}},
"solutions": {{["Convert-Solution-Query-Results-To-JSON"].[json]}}
}

That is passing in the json. figuring out where and when to quote seems to be the trick. Correct me if i am wrong, rule of thumb is the step name gets quotes, the output variable name does not.

The python step is failing for a type error again but i think that is because, as you pointed out i don't have a json output var set. Two questions

1. in the step log i see a lot of ^

Is this a just a product of the parser? is there something i should/could do to minimize?

2. I have the output of the python step as
   
   

I didn't add(or at least remember adding) Result 1 and 2, but i did add Myjsonoutput. Should i remove result 1 and and two and just leave the json output?

[Eric-Wilson]

1. The step names are always quoted. If you're using the variable picker by clicking the plus sign in the bottom right of the input box it should format the variable correctly for you without you having to type anything else.
2. I'm not sure what the carrots are about. Could just be code logs from a number of modules that are connected to that plugin.
3. Result1 and 2 can be removed or renamed, but they are there by default.

[tward1 (tward)]

i want to make sure i understand. Are you saying i can use jq instead of the python step? I didn't think you could have multiple inputs(except for python steps) or does have a single map referring to 2 variables count as a single input? if so, that opens up a lot of steps.

[Eric-Wilson]

Yes, you can use jq instead of python. The JQ step can have as many variables in the input as needed. I forgot that you just have to build out the structure of the object correctly like you would in the python step with some static text. From my screenshot you can see I've built an object statically and put two different variables in place where the arrays are supposed to go.

[tward1 (tward)]

gotcha, going to work on this, stay tuned, thank you!