System monitoring web app?

[ZiaTioN]

Hello all, long time no talk..

Anyway down to business. I am interested in finding a web based application that records system statistics and generates (static or dynamic) html pages. I have checked sourceforge and have seen many that come close to what I am looking for but none that really incapsulate all.

I want an app that will monitor all input and output traffic via all ethernet interfaces. I would also like to be able to track dns queries (not a must but would be nice) and other stats like cpu and memory of course. I see void has a decent one here on his site but this does not do dns or show real time graphs of net in/out traffic.

Basically I want something I can refer to as a first line of defense (or intelligence gathering) of any possible network anomoly (ie. dos, ddos, excessive cpu or memory usage, etc). Does anyone know of a decent application that can do all this? I am looking for one in either perl or php but any language will do I guess.

Something like mrtg is what I am looking for but for a server and not a router.

[Void Main]

Cacti will do most everything you want to do. I do have it running on my site but only have a minimal amount of information in it. At work we do track DNS queries. We graph all the different query types and errors and it is a GREAT indicator of a worm for instance. We also have a plugin installed (which is about it become a standard part of Cacti) where you can configure alerting on anomolies. For instance you can take a bandwidth graph and tell it if it goes over a certain percentage higher or lower than it was a week prior then send an alert. You can graph *anything* in Cacti as you can create your own custom scripts to gather the data for the cacti graphs. There is nothing else close to being as capable as Cacti for what it does.

Now, that is just one tool we use though. We also have developed a lot of our own custom security applications built around "snort", "base", "cacti", "snortcon" , "flow-tools", etc. Some of those apps don't look anything like their original selves in our setup. We have several snort sensors at various points in our network to gather data and dump into a database. We also have developed correlation apps to correlate our snort, netflow, firewall logs and application data.

Of course for just server alerting you might want to look at something like Nagios. I used to use to be a big fan of Big Brother but ditched it when it went commercial.

[ZiaTioN]

Awesome, I had looked into Nagios a little and thought this was the closest to what I was looking for; however, cacti sounds like what I will try first. I also looked into developing my own, but did not want to re-invent the wheel if it had already been invented.

As for the custom apps you guys have created, how do you go about obtaining the information from other apps such as snort and the others you mentioned?

[Void Main]

We dump everything into a database (MySQL). Snort has a mysql logging option that must be used if you want to be able to view the alerts with "base". We use syslog-ng and some custom programming to dump our pix firewall logs to a data base in real time. I use flow-tools to dump netflow data to a database. Basically we dump everything we can into tables and then writing correlation apps is fairly simple. For instance, we have a page where you can plug in an IP address (src, dst or both) a port (src, dst or both), date/time ranges etc and then check off which logs you want to search and it will pull up all instances of that IP and where it has been seen, what it's been doing (snort data and ISA log data) and when. So basically it's just querying tables at this point. Regarding the snort table specifically the acid alert cache is the easiest and most interesting place to grab the snort data from.

The opening page on our custom security console shows a high level overview. It has a large Cacti graph from each of our OC3 lines, it shows a large combined snort graph of all of our sensors showing tcp/udp/icmp/portscan alterts over time. Then we have smaller graphs of each sensor individually under those. It also shows real time snort overview statistics like largest number of alerts per IP address and per alert, last 30 minute attack detail, etc, etc. Addresses and alerts are also links that can be clicked on to get more detail in apps like base, ISA log app, correlation app, etc.

We also have something we wrote we call the "snort console" that slices and dices snort alerts a lot better than "base" does it for a much better high level look at the snort alerts. It also has drill down capabilities and base is still linked to for viewing alert details. That's a very rough overview of about half of the system.