I finally managed to get it working by following this guide: Here
Basically the only difference was in the OnStart method of my web role and rather than the code above which is found in almost all of the examples online, I had to use the following, perhaps because I am using multiple instances?
public override bool OnStart()
{
string wadConnectionString = "Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue(wadConnectionString));
RoleInstanceDiagnosticManager roleInstanceDiagnosticManager = storageAccount.CreateRoleInstanceDiagnosticManager(RoleEnvironment.DeploymentId, RoleEnvironment.CurrentRoleInstance.Role.Name, RoleEnvironment.CurrentRoleInstance.Id);
DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
config.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1D);
config.Logs.ScheduledTransferLogLevelFilter = LogLevel.Undefined;
roleInstanceDiagnosticManager.SetCurrentConfiguration(config);
return base.OnStart();
}
Thank you elastacloud!