Recently myself and my team have been testing the new functionality of custom detection rules.
After some testing and migrating some rules we had in the “Basic Detection rule” format, we have to say that this is a top tier functionality and we love it.
But I have a question. In the FAQ it says that you can only create 50 custom detection rule. Why is that? I mean, could this number change in the future?
If you have not tried the new custom detection rule, I encourage you to try it!
Glad you like the feature, and thank you for the feedback!
The reason for the current 50 limit is primarily because it’s a new feature and we are being cautious that customers may write rules that are overly noisy and swamp either themselves or their detection engine. We are constantly adding new features and safeguards to assist in testing, validating and preventing these potential problems, and have already increased the limit from an initial 20 to 50. I think the next jump will be to 200, but we are intending to increase this further (and can increase upon request individually in the meantime)
Will the Basic Detection Rules be deprecated one day? Iam using those right now to notify myself (via E-Mail) about specific log appearances without triggering an investigation because I just want to be informed when something happens, e.g. when someone logs into a switch or our backup solution. I don’t need an investigation for that, but I want to be informed when it happenes and the Basic Detection Rules cover this case perfectly.
However, It seems like I can’t do that with the new Detection Rules, is that right? The rule action implies what will happen when a detection triggers, right? So I can create an investigation and thus get an E-Mail, like for all investigations, but I can’t only get an E-Mail. Will this feature be adopted to “just notify via E-Mail”?
A rule action for “Tracks Notable Events” & “Notify via E-Mail” would be perfect for my usecase.
Hi Robert!
Do not worry, Basic Detection Rules won’t be deprecated until we have exactly the feature parity that you require (and a few other BDR specific capabilities - most relevant to this thread would be removal of limitations on total number of active rules).
Right now, if you have an InsightConnect licence, you can achieve the same outcomes - but we are aware that the additional built in notification options with BDR are superior to the out of the box options with CDR and this is an area we are tackling as part of a broader automation upgrade across InsightIDR as a whole.
What are some good custom detection rule recommendations? I know this will depend on the environment and based on what is considered an anomaly on a given server but wondering if you have any recommendations or is there any source on the web that might be helpful?
My only issue with the basic detection rule, is the output I get. It basically contains the whole log, is there a way to custom the notification email to only contain certain variables? Our previous option, has this, so the notification only produced the information we wanted from the log. As its stated earlier, this notification itself is too noisy, would be nice to only produce, the information I want. Perhaps using set variables?
Its currently not possible to have the output only include the specific key/values you are interested in. This would be considered a Request for Enhancement
I am trying to create custom detection rule from raw logs but even my data is in Key value pair, I am getting Unrecognized keys error in LEQL rule logic section.
Hello hello! A couple of things here, firstly, as the LEQL is valid you should still be able to save your rule as written. The message isn’t actually an error - as the rule is based on the ‘raw’ logs we are currently not able to provide schema validation, so we’re just letting you know that we can’t confirm it exists. However, if there are known examples of this happening previously, you can validate the query with the ‘Evaluate in log search’ option.
Secondly and just fyi, there are a few improvements we are working on in this exact area to help explain some of the frequently asked questions we are beginning to see when creating Custom Rules on raw and unparsed logs (along with some longer term ones to provide raw log schema validation) so please let us know if you have any more questions for us!
