Alienvault USM – Flat File Log Capture – Part 1

We’ve been working with and on Alienvault since the beginning of 2016 and a lot has changed since then. When we started out with Alienvault, they were a small-ish company still, with big ambitions, working with a very technical group out of Cork, Ireland. We had direct access to their technical team (I think even to one engineer) and the amount of knowledge we got from those early days are pretty much invaluable to where we are right now. Of course, Alienvault has changed a lot since then, and now being part of AT&T – for the record, we believe they have the right roadmap to go into cloud with their USM Anywhere concept, and their product right now is much more robust and enterprise ready. They are on the right trajectory.

However, back in the days, for Alienvault USM Appliance (not Anywhere), which is their Appliance offering, we could literally ‘jailbreak’ the system and go into the underlying OS and do stuff to Alienvault that we can’t do anymore in Anywhere. Some of the changes we made were to increase optimisation, put in our own scripts to clean up the system, troubleshoot the system and of course, create plugins for custom applications. We would write custom plugins in 1 – 2 days for multiple applications because of deadlines, I remember and had to do so much in so little time – but we did it anyway. We had to write a plugin for one of the oldest mainframes for a financial institution that was so difficult to interpret, we had to dig up old manuals to sort out the entries for log and events. It was like we were interpreting Egyptian hieroglyphs. But that’s what it took – 2 days, I think because of compliance requirements and customer breathing down our neck to get it done.

Writing plugins was the easier part of the battle – in some old machines or legacy applications, getting the logs was the problem. If Alienvault doesn’t get the logs, it can’t do anything with it. One solution was to leverage on the HIDs (Host IDS), or OSSEC as it was known, to grab log files from systems. It wasn’t so elegant, and we still had to end up writing plugins for it to normalise, but it resolve the issue where application was not able to forward logs to the SIEM, or not able to write the logs to the Windows Event service, or any other way to get logs out to a syslogger. So the solution here is for the application to just write the logs to a file, and Alienvault go ahead and grab this and interpret it. It may not be real time, but it works.

There’s a good write up over in Alienvault at
https://www.alienvault.com/documentation/usm-appliance/ids-configuration/process-reading-log-file-with-hids-agent-windows.htm. So a lot of it is just a repeat and probably an exposition on why we are doing certain things in a certain way.

So the first thing to do here is to ensure that you are able to install HIDs on the server. HIDs will be the key to get this file out to Alienvault. Technically, you could actually use NXLog as well but let’s explore that another time.

Once HIDs is installed, get into the ossec configuration file to define the <localfile> location. Now assuming that you have configured your application to write to a flat file called database.log.txt.

Go ahead and restart OSSEC. That’s pretty much what you need to do to start off so it’s pretty simple.

The rest of it is all done on Alienvault.

To summarise the steps:

Enable “logall” on your USM Appliance. You want to dump whatever you are getting in that flat file database log to a log inside your Alienvault so you can start doing stuff to it. In this case, in your AV User Interface:

Environment > Detection > HIDS > Config > Configuration.

Add <logall>yes</logall> to the <global> section of the file .

You are dumping these logs into /var/ossec/logs/archives/archives.log.

Restart the HIDs service through UI.

You should be able to see new logs coming into archives.log. Just do a tail -f on it, edit the log file (database.log.txt) in your remote system (just write something on it) and see if it appears in your archives.log. Once you see it, you are almost done. Very simple.

So for now, you have customised logs coming into your Alienvault. The next thing to do is to interpret these logs and make sure events are able to be derived from these logs to something that is useful to you!

Drop us an email at alienvault@pkfmalaysia.com for more information on Alienvault or any technical queries you have, and we will attend to it.

Leave a Reply

Your email address will not be published. Required fields are marked *