Systemtap for java on ubuntu

Question

I'd like, not only to trace the java process, but use the new support for openjdk tracing in systemtap, both the hotspot tracing and the method tracing.

Accordingly i installed the ddebs.ubuntu.com repository to install the kernel debuging symbols - i can then call a stap script that uses kernel tapsets but not yet java ones. I did notice a package called openjdk-7-jdk-dbgsym and tried to install it to see if this had the systemtap tapsets for the openjdk, but this conflicts with the openjdk-7-dbg package (which then ubuntu doesn't let me bug report since the openjdk-7-jdk-dbgsym package is not from the 'official' servers. And if i uninstall that one and install the other it doesn't help anyway.

Has anyone successfully did this on ubuntu?

edit: in order to build systemtap from source successfully on ubuntu with java byteman support you have to pass the

--with-java=/usr/lib/jvm/default-java

(or your strange jvm location)

Otherwise building will not do the jars and so needed. Then you have to do make install follow the steps in the source dir java/README file (and don't forget to modify the path).

There is also another --with-dyninst option which i haven't tried but might 'fix' it for the other invocation modes

edit2: well, it compiles and even runs, but it never outputs nothing even on the examples given and with BYTEMAN_HOME set...

Answer 1

There are several different tactics to trace openjdk from systemtap.

The first relies on sys/sdt.h dtrace-style markers compiled into the JVM, but not dbgsym data:

% stap -L 'process("/usr/lib/jvm/java*/jre/lib/*/server/libjvm.so").mark("*")'

If this shows an empty result, (and if I have the ubuntu libjvm.so path right), then this suggests your openjdk was compiled without the sys/sdt.h stuff, so this option is closed to you. If it shows a lovely list, you can use those .mark probes directly, or snarf/adapt a copy of the hotspot*.stp tapset sources from anywhere and transcribe it into your .stp file, or let stap find it via

% stap -I PATH ...

The second method relies on dwarf debuginfo compiled into the JVM, for which the dbgsym* stuff should be handy. If that is installed properly,

% stap -L 'process("/usr/lib/jvm/java*/jre/lib/*/server/libjvm.so").function("*")'

should show a gajillion functions. You may not have the benefit of tapsets, but with clever choice of functions, you should be able to get some decent tracing going.

The third method relies on byteman to do intra-JVM self-instrumentation under systemtap control. This requires no java dbgsym, but does require byteman and a bunch of auxiliary stuff. If this is available and compiled into the ubuntu systemtap, then something like:

% stap -e 'probe java("org.my.MyApp").class("^java.lang.Object").method("foo(int)")
           { println($$parms) }'

may work for you.

Answer 2

I ended up using byteman directly for java and systemtap for the kernel. Unwieldy but it works.