[jamesRaybould]

Having a quick poke around and I notice that they are storing all exceptions in an SQL database. I've been looking at storing all the errors we get in our various applications in a central repository and was wondering what the general consensus was?

Currently I'm going a centralised logstash server and using a logstash shipper on each of my servers to push the exceptions, from a standard logfile to it. I was toying with the idea of pushing all my errors at source to an SQL database but figured if I was having database problems I'd be missing all the exceptions that I could be using to trigger the alerts that I'm having database problems!

[SEJeff]

Seriously take a look at Sentry[1]. It supports just about every major language out there, is open source and used on some very large web properties (disqus, which powers comments for cnn.com amongst others), and is just generally awesome software. If you don't want to set it up yourself, use their hosted version[2].

If you are a .net / C# guy check out their csharp raven client[3]. Raven is the client to sentry which automatically sends all exceptions.

[1] https://github.com/getsentry/sentry

[2] https://getsentry.com/welcome

[3] https://github.com/getsentry/raven-csharp

[rhizome]

There is no general consensus, but there's no rule that say you can only have one central watchdog. Collect errors one place, watch over you db somewhere else, watch over your watchdogs somewhere else still. This is basic sysadmin CYA.

[Nick-Craver]

We indeed use SQL for errors (though Exceptional has a couple of stores...and you can add a new one). The reason a SQL outage isn't an issue is a) we have other alarms for that, and b) Exceptional will fall back to an in-memory exception queue and flush to the DB when it's available again.

In the event of a store loss (file share, SQL server...whatever your store is) then it queues exceptions in memory with rollups to reduce memory usage, and will flush to the store when it's available again. It does a connectivity test every 2 seconds in the event of failure.

Exceptional is open source and is the basis for what's used in Opserver...the UI is even very, very similar it's just that Opserver has more features for a multi-application view. You can see the source here: https://github.com/NickCraver/StackExchange.Exceptional

[sk5t]

We built our monitoring package at Zetetic to be extremely flexible about what events go where, with multiple levels of non-blocking filtering and routing, so that everything remotely interesting could go to SQL (with or without a durable local queue in front of it), and just very critical stuff could also go to a local embedded database / lpr / MQ / ZeroMQ publisher, etc. My preference is for decoupled ZMQ subscribers to handle anything very gnarly.

Internal errors in the monitoring software itself first surface in NLog, which could--but probably oughtn't--be configured to feed even more errors into the monitoring system; obviously this could create a terrible feedback loop if left unchecked.

[js2]

Take a look at http://squash.io before you build your own.