I am wondering if anyone has attempted to get SAP audit logs into InsightIDR. The SAP audit logs (.AUD) are a strange beast. No new-lines or carriage returns, 200 character entries, some sort of UTF16 format. I am sure there is a way to bring them into NXLOG, break them down into individual lines and then parse out some fields based on fixed width or similar. If anyone has attempted this before or something close (single line log files) some input would be appreciated before I start this from scratch.
Thanks for reaching out.
Unfortunately SAP is one of those Event Sources that do not come often enough for us, so, nothing official on our roadmap at this point.
As far as trying alternative methods, you say they are single line logs with ~200 characters? and per your comment I would say that those segments are consecutive or is there any sort of limiter between them?
I would recommend 2 things, the first would be to dump those logs into a txt file in a collector (I’d say linux) and then create an event source to tail a file.
The second involves a couple more things, cause I’d suggest a bash script perhaps to transform the log file from option 1 into something readable fo IDR.
Thanks Felipe. As you inferred they are all consecutive with no delimiter. There is a fairly simple regex statement that does apply to the beginning of all entries though.
I have started working with NXLog to see if it can always grab the last 200 chars on an edit however the log files can get pretty long (still one single line) so performance might get questionable.
If that doesn’t work I’ll look at scripting something as you mentioned. Ideally I get these log entries in real time but pulling the log file at the end of the day and running it through a script to rip it into individual lines and potentially grab some fields out of it would be ok as well.
We have successfully started bringing in SAP audit logs into InsightIDR with the assistance of NXLog. Basically there is a python script that is monitoring the last .AUD file and pulling in 200 char chunks when the file changes. This is then parsed into the field lengths (all fields have a common length), assigned a field name and then written to a file. Some of the data in the AUD files does not make much sense as I think SAP uses some dictionary lookups etc to expand the info but we are getting valuable info. We get users, systems, tcodes and the “message” which once I work with the SAP team we’ll know what we want to alert on. Nxlog handles the rotation of the logs as well. I do not think it would be appropriate of me to give away the work NXLog did for this but in the end it is very simple. Roughly 50 lines each for the python script and NXLog config. The enterprise license of NXLog and a couple hours consulting are less than $700 USD so well worth the investment for us. Let me know if you have any questions.
As we paid NXLog to develop the log parser I do not think it appropriate to hand out the script code. As I mentioned it was a pretty simple and inexpensive engagement with NXLog so I would start there. They also mentioned NXLog had been asked numerous times for this and were looking at developing a dedicated solution.
