Does perf lock profile user space mutexes?

Question

Summary: Does perf lock profile pthread_mutex?

Details:

The tool perf has an option perf lock. The man page says:

You can analyze various lock behaviours and statistics with this perf lock command.
   'perf lock record <command>' records lock events
    between start and end <command>. And this command
    produces the file "perf.data" which contains tracing
    results of lock events.

    'perf lock trace' shows raw lock events.

    'perf lock report' reports statistical data.

But when I tried running perf lock record I got an error saying: invalid or unsupported event: 'lock:lock_acquire'. I looked and it seems that error is probably because my kernel is not compiled with CONFIG_LOCKDEP or CONFIG_LOCK_STAT.

My question is: does perf lock report events related to user-space locks (like pthread_mutex) or only kernel locks? I'm more interested in profiling application that mostly run in user-space. I thought this option in perf looked interesting, but since I can't run it without compiling (or getting) a new kernel I'm interested in getting a better idea of what it does before I try.

Answer 1

Summary: Does perf lock profile pthread_mutex?

Summary: no, because there are no any tracepoint defined in user-space pthread_mutex.

According to source file tools/perf/builtin-lock.c (http://lxr.free-electrons.com/source/tools/perf/builtin-lock.c#L939) cmd_lock calls __cmd_record, which defines several tracepoints for perf record (via -e TRACEPOINT_NAME) and also pass options -R -m 1024 -c 1 to perf report. List of tracepoints defined: lock_tracepoints:

842 static const struct perf_evsel_str_handler lock_tracepoints[] = {
843         { "lock:lock_acquire",   perf_evsel__process_lock_acquire,   }, /* CONFIG_LOCKDEP */
844         { "lock:lock_acquired",  perf_evsel__process_lock_acquired,  }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
845         { "lock:lock_contended", perf_evsel__process_lock_contended, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
846         { "lock:lock_release",   perf_evsel__process_lock_release,   }, /* CONFIG_LOCKDEP */
847 };

TRACE_EVENT(lock_acquire,.. is defined in trace/events/lock.h. And trace_lock_acquire is defined only in kernel/locking/lockdep.c (recheck in debian codebase: http://codesearch.debian.net/search?q=trace_lock_acquire). Only CONFIG_LOCKDEP is missing from your kernel according to kernel/locking/Makefile: obj-$(CONFIG_LOCKDEP) += lockdep.o (tracepoints are defined unconditionally in the lockdep.c.

According to https://www.kernel.org/doc/Documentation/trace/tracepoints.txt all tracepoints are kernel-only, so perf lock will not profile user-space locks.

You can try tracepoints from LTTng, the project which declares user-space tracepoints (http://lttng.org/ust). But there will be no ready lock statistics, only raw data on tracepoints. Also you should define tracepoints with tracef() macro (recompile pthreads/glibc, or try to create your own wrapper around pthread).

Answer 2

No. But maybe what you need can be done by recording sched_stat_sleep and sched_switch events:

$ perf record -e sched:sched_stat_sleep,sched:sched_switch -g -o perf.data.raw yourprog
$ perf inject -v -s -i perf.data.raw -o perf.data
$ perf report --stdio --no-children

Note: make sure your kernel is compiled with CONFIG_SCHEDSTATS=y and it is enabled at /proc/sys/kernel/sched_schedstats

See more at https://perf.wiki.kernel.org/index.php/Tutorial#Profiling_sleep_times.