Detect when ETW drops events

Question

How can I determine if an ETW session is dropping events?

If it is dropping events, how can I configure the tracing session so that events are not dropped?

I've written a custom ETW provider to help with some debugging efforts. I'm currently capturing the trace data using logman.exe.

In viewing the results, it appears that some of the events are being dropped. Basically I'm seeking something like:

Event A Event C

where their should be an intervening Event B, but one does not appear in the trace file. It should be impossible for that to happen, which leads me to believe that ETW is dropping events.

Of course, I'd like to verify that the problem I'm seeing is due to dropped events, and not caused by a bug in my code. I've tried Google, but wasn't able to come up with anything. Does any one know how I can check to see if events are being dropped?

Answer 1

I've been using logman to capture the results. It looks like tracelog will give me info about lost events, and I can tweek its buffer parameters to reduce the event loss.

Answer 2

It doesn't answer the question directly (how to detect drops), but it might explain drops:

EVENT_TRACE_NO_PER_PROCESSOR_BUFFERING

Writes events that were logged on different processors to a common buffer. Using this mode can eliminate the issue of events appearing out of order when events are being published on different processors using system time. This mode can also eliminate the issue with circular logs appearing to drop events on multiple processor computers.

If you do not use this mode and you use system time, the events may appear out of order on multiple processor computers. This is because ETW buffers are associated with a processor instead of a thread. As a result, if a thread is switched from one CPU to another, the buffer associated with the latter CPU can be flushed to disk before the one associated with the former CPU.

If you expect a high volume of events (for example, more than 1,000 events per second), you should not use this mode.

Note that the processor number is not included with the event. Not available prior to Windows 7 and Windows Server 2008 R2.

Answer 3

If you use xperf to collect the logs, it generates a warning when events are lost. With xperf you can also play with the buffer size and can divide the logging to several loggers.