You need to call ActivateOptions on the layout as well as on the appender:
var layout = new log4net.Layout.PatternLayout()
{
ConversionPattern =
"%date [%thread] %-5level %logger %ndc - %message%newline"
};
layout.ActivateOptions();
appender.Layout = layout;
Sample output:
2014-03-26 20:29:49,816 [1] DEBUG test logger (null) - log test