Log4Net Hack: Customizing Log File Paths After Configuration

Image may be NSFW.
Clik here to view.I get a lot of log4net questions through my blog because of the tutorials I've written up.  One item that comes up frequently is how to configure a FileAppender to be able to write a single log file from multiple instances of the same application.  The truth is, you can't do it.  Or more precisely, there is no way to do it and expect not to lose log events and have your application performance suffer greatly.

First let me show you how to allow multiple applications access to a single log file.  It's actually quite easy:

 <?xml version="1.0" encoding="utf-8" ?> <configuration>  <configSections>    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>    </configSections>  <log4net>    <appender name="FileAppender" type="log4net.Appender.FileAppender">      <file value="log-file.txt" />      <appendToFile value="true" />      <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />      <layout type="log4net.Layout.SimpleLayout" />             </appender>    <root>      <level value="DEBUG" />      <appender-ref ref="FileAppender" />          </root>      </log4net>  </configuration> 

Since I discuss the parameters and pitfalls of the FileAppender elsewhere, I will leave it to you to read up on them more if you want to.  The locking mode being used here is causing log4net to acquire the file handle before each log event, then close the handle after the log event is written.  Doing this allows other applications to get write access to the file when no one else is currently logging, but the technique has a few serious flaws that should prevent you from using it:

1. All of that file opening and closing seriously hampers performance;
2. The log file will be shared, but access conflicts will still occur between applications attempting to log events at the same time, resulting in more performance degredation and "dropped" log events.

You may be able to address some of the performance issues using a BufferingForwardingAppender that sends large chunks of events to the minimally-locking FileAppender; however this will not resolve the contention over the single log file that is at the root of the issue.

The simplest solution to this problem is to use a different type of appender that is designed for concurrent use.  For instance, the EventLogAppender or AdoNetAppender use technologies that will manage concurrency issues for you.  If you're dead set on using the filesystem, the next simplest solution is to have each application log to a unique file, thus removing any log file contention at runtime.  The separate log files can be collated once the run is over using a tool like LogParser.  The drawback to this approach is that you have to hack it in: there is no direct way to modify the filename in the FileAppender based on the runtime environment.

That said, it's not hard.  Check out this simple console application:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using log4net;
using log4net.Appender;
namespace log4netPostConfig
{    class Program    {               static void Main( string[] args )        {            log4net.Config.XmlConfigurator.Configure();            var filePostfix = "_" + Guid.NewGuid().ToString( "N" );                        var fileAppenders = from appender in log4net.LogManager.GetRepository().GetAppenders()                                where appender is FileAppender                                select appender;            fileAppenders.Cast<FileAppender>()                .ToList()                .ForEach(                    fa =>                    {                        fa.File += filePostfix;                        fa.ActivateOptions();                    }                );            ILog Log = log4net.LogManager.GetLogger( System.Reflection.MethodBase.GetCurrentMethod().DeclaringType );            Log.InfoFormat( "this process is using log file postfix [{0}]", filePostfix );        }    }
} 

This example loads the logging configuration from the app.config (line 14).  The log4net configuration is searched for instances of FileAppenders (line 16), which have their filename parameters handrolled with some process-specific information (line 25) - a GUID in this case, the current process identifier may be another good choice.  Calling the ActivateOptions on each modified appender is vital (line 26), as it recreates each file handle using the new filename configuration parameter set in the code.

The app.config for this example is just a plain vanilla logging configuration:

<?xml version="1.0" encoding="utf-8" ?> <configuration>  <configSections>    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>    </configSections>  <log4net>    <appender name="FileAppender" type="log4net.Appender.FileAppender">      <file value="log-file.txt" />      <appendToFile value="true" />      <encoding value="utf-8" />      <layout type="log4net.Layout.SimpleLayout" />             </appender>    <root>      <level value="DEBUG" />      <appender-ref ref="FileAppender" />          </root>      </log4net>  </configuration>

Note that the log-file.txt specified in the app.config will be created when the XML logging configuration is loaded (line 13 in my code example above), but it will never be written to.

Edit Notes

I just noticed after publishing this that a very similar example was written almost 2 months ago by Wil Peck.