Elasticsearch plugin - Query DSL not functioning as expected when referencing objects


I’ve recently started using InsightConnect and ran into a problem with the Elasticsearch plugin - The “Query” field seems to have an odd interaction when referencing object fields, e.g. pasting below query and hitting save:
{“query”:{“match_phrase”:{“related.ip”: “”}}}
and opening the edit screen again will show us that the query’s been changed to this:
The query it creates is incorrect and produces errors in both Elastic and the InsightConnect’s Elastic plugin as well.

Getting rid of dots fixes the issue. Obviously it’s not a fix as we can no longer reference objects cleanly. Is there a solution to this? Am I doing something wrong?


1 Like

Give me a minute, I’m going to try to reproduce this.


I sort of got around this issue by modifying the PATCH request and correcting it to what it should be but then I noticed another issue which seems to be related to the plugin running on the orchestrator itself.

I am getting below error message:

Failed validating ‘type’ in schema[‘properties’][‘hits’][‘properties’][‘total’]:
{‘description’: ‘Number of Total Hits’,
‘order’: 1,
‘title’: ‘Total’,
‘type’: ‘integer’}

On instance[‘hits’][‘total’]:
{‘relation’: ‘eq’, ‘value’: 1011}

As you can see, the schema expects the “total” field to be an integer but instead it is an object with nested values. This seems to be the case for all of the calls I make to Elastic (e.g. via cURL).

Schema for the plugin was last updated in November 2019:


1 Like

Yup, that’s a bug on our end. I’ll put a ticket in and we’ll get on a fix.

Thanks for reporting this and sorry for the inconvenience. We’ll get a fix out as soon as we can.

So this part where the paste was acting weird. That line has “smart quotes” in it.
vs "

That bombs out most string parsers (including JSON parsing). It’s a weird copy-paste bug in OSX we run into every now and again. There are instances where OSX just decides to convert quotes on the 2nd sunday of a month with a blue moon.

I’m not sure what we can do to improve the user experience around that, unfortunately.

Hi Joey,

I am not using Mac. The whole string was also pasted into notepad prior to me copying and pasting it into the JSON Query field so I am not sure if that’s correct?

The very same query also works flawlessly in Kibana. Maybe it’s the Rapid7 forum that changed the quotation marks? Anyways, I have just went into the InsightConnect instance and typed in the whole string myself and I’m getting the same behaviour.

Could you test it with normal quotes instead of the smart quotes and see if you can reproduce it again?


1 Like

Ok, got it. I’m putting in a ticket for this.

I think you’re right about the “.” being the problem. My suspicion is when we do the handlebar parsing it’s trying to replace based on the dot. Not sure yet though.

Thanks for sticking with me to get that bug right. We’ll get it fixed.


That’s much appreciated, thank you.

Is it possible you could also look into the other thing I mentioned? It looks like the schema.py file contains fields that’ve been incorrectly addressed so it throws an error when it tries to interpret an object field called “total” as integer.



The “total” field is mentioned in the article below. Could it be that the schema.py was never updated to work with Elastic 7.X?