Has anyone come across a good way to do idle log source alerting when the SIEM is ingesting a large number of sources? The issue we are facing is that we either have to have an alert per log source (500 custom alerts) or an alert per group of log sources (which leads to difficulty determine which actually failed.) The investigations generated are ambiguous and do not seem to include detail as to the specific source which went idle.
You should be able to set up a single inactivity custom alert that monitors all logs. It will trigger when any one of those logs become inactive in line with the threshold set on the alert.
For example, I have an inactivity alert for two logs A and B. The rule is configured to trigger when a log stops getting data for ten minutes.
A continues to send data while B stops sending data for 10 minutes, you will get a notification stating log B is inactive, see the result of my test below to a slack webhook:
The alert triggered at 13:49 UTC+1, just over ten minutes after the log (name: ub2logs2) was last active (13:38:31 UTC+1).
Now, you it doesn’t make sense to create a single inactivity alert for all logs because you may have varying thresholds (some logs are quieter than others and it can be normal to go for an hour or more without activity).
My advice here is to break it out into multiple inactivity alerts, one of which can have a tighter threshold for more critical log sources, while others are more relaxed.
Hope this helps!