Log searching across multiple, disparate log sets using field qualifiers

InsightIDR Log Search
1

Hi,

I’m wanting to find a way that I can search multiple log sources, and extract logs by user “where” queries, which are able to search across the different log sources, by “qualifying” the fields. The problem I have at the moment, is that if I use where clauses to find logs from one log source, then I cannot find logs in another, as the fields do not necessarily exist, if that makes sense. This is due to multiple products (antivirus), which report using different field names. i.e., one logs using the field “action”, and the other as “utmaction”.

Looking at a Rapid7 built-in tile for a dashboard, it used the format "where (action and “action”=“result”) ", which I was hoping would be able to qualify the field only if it existed, however this doesn’t work, so I’m not sure how or why this would be used in a query.

Hopefully that makes sense, any tips or tricks would be appreciated!

2

hey Nick, you would have to string together the query using an OR condition, such as where(action=result OR event=xyz)

This would return logs where action is present and equals result as well as logs where event as present and equal to xyz. action doesn’t need to be present in logs where event is present and vice versa.

David

1 Like
3

Thanks David, managed to get it working off the back of your advise, using some heavily nested OR condition groups inside of brackets. Doesn’t look pretty, but does the job!

Out of interest, I had another couple of questions. Is there any method of selecting a log source in it’s entirety? i.e. One of the log sets (antivirus) would always generate logs which should be included, rather than trying to match fields in the logs.

Another question, is there an equivalent of match a log based on a field existing i.e. !=null?

Thanks,
Nick

4

The only way to match a log in its entirety would be to include a key that always appears in the log itself, that and selecting the log for search of course. Heres the schema for Virus Alert Keys to Use in Your Queries

As for your second question, yes the easiest way would be to use

where(key=/.*/)

Which is to say the key should be present and equal to anything
David