powershell multithreading log4net intercalated log issue

Question

I have a PowerShell script that uses log4Net for the management of logs. The logs are written in log files and in MS SQL database. The script is using multi-threading with run spaces.

The issue is multiple threads are managing several objects and logging lots of data on different objects at the same time. And I need to regroup logs by object. An example will help me to explain myself better! ^^

log file line 1 OBJECT 1.ACTION 1
log file line 2 OBJECT 1.ACTION 2
log file line 3 OBJECT 1.ACTION 3
log file line 4 object2.action1
log file line 5 object3.action1
log file line 6 OBJECT 1.ACTION 4
log file line 7 object2.action2
log file line 8 OBJECT 1.ACTION 5
…

To manage this intercalated logging issue, I planned to log in memory, for example in a table, and at the end of the treatment of the objects; block other threads using mutex and write all logs with a foreach loop.

Main {
    Treat object {
    Action1 -> Logs +=log1
    Action2 -> Logs +=log2
    …
    }

    System.Threading.Mutex WaitOne()
    For each ($log in $Logs) {
        Write in log file
        Write in SQL DB
    }
    System.Threading.Mutex ReleaseMutex()
}

I would like to know if there is any better solution to manage intercalated logging issue with multiple run spaces please.

Log4Net can perhaps natively manage this; stocks all logs in memory and “commits” the writes only when I type a command? Or some other solutions without using Mutex?

score 0 · Answer 1 · edited May 23 '17 at 11:51

One way would be to use one logger per group, and if you don't know the number of groups in advance you could just create the loggers dynamically.

If you prefer to work with one logger, the best practice usually is to log things as they happen and do the grouping afterwards, for example when the logs are displayed. It's trivial to do in SQL by adding a column for the grouping criteria, or for a text file you could use the unix sort command.

powershell multithreading log4net intercalated log issue

1 Answers1