Re: Conserver logfiles

On Thu, Nov 27, 2003 at 07:07:33PM -0800, Bryan Stansell wrote:
> yep, all versions just grow.  i usually set things up so that files are
> moved to a subdirectory then conserver is sent a SIGUSR2 (if 7.2.5 or
> above, otherwise a SIGHUP).  if anyone has a good methodology for
> trimming/rolling/size-restricting logfiles automatically, i'd love to

This is the way log rolling should happen within a server.  A couple
of log rolling scripts in the contrib/ directory should be used to
take care of the rest :)

> so, those are the kinds of things i worry about.  when it's controlled
> externally, folks can schedule things appropriately and just opening a
> bunch of files is (usually) very quick.

A program with similar restrictions as conserver, syslog-ng, tries to
address the needs of some folks who would like to not roll files.
Syslog-ng can use template filenames, one option in the template is
portions of the date and time. 

Thus a LogfileTemplate=/var/log/conserv/$YEAR/$MONTH/$DAY/$HOST is
possible.  It doesn't eliminate partions of rolling - removing old
data and compressing - but it does keep files smaller automatically.

To handle some blocking issues syslog-ng uses buffers and threads with
blocking I/O or asyncronous I/O.  Thus I/O calls that take time can be
used *and* you don't loose any data.  Yes, buffers can overflow, but
then you are accepting data faster than you can write to disk so you
would have lost data anyway, its just a matter of which portion of the
data.

http://www.balabit.hu/

Samba has a log filesize limit you can set, if the log file exceeds
this size (perhaps just on start or when SIGHUP is received) the log
file is moved to <logfilename>.old and a new logfile is created.  The
logic is fuzzy, so the limit isn't a hard size, its more of a
suggestion of a limit that gets rolled when it makes sense to do so
(so not half way through a log message).

Apache gives you the ability to log to named pipes allowing for a
dedicated logging program to go nuts with compressing on the fly and
rolling at size, or line, or time limits.
-----------------------------------------------------------------------
   __o		Bradley Arlt			Security Team Lead
 _ \<_		arlt@cpsc.ucalgary.ca		University Of Calgary
(_)/(_) 	Joyously Canadian	 	Computer Science