I have a bit of experience in this area, having done some work on debugging dynamically compiled groovy scripts though the JDT. I never got this to work perfectly, and I think it is mostly a limitation of the JDT, which was never designed to handle dynamically compiled code.
TLDR: My guess is that your dynamically compiled scripts have an incorrect source file attribute in the byte code. This attribute is set in the class file by the compiler. See https://en.wikipedia.org/wiki/Java_class_file
Your confusion, I think, is that the debugger correctly stops at breakpoints you set in the scripts, but the IDE cannot load the source. This is confusing of course, but there is a good explanation for this.
Breakpoints are actually handled by the VM and the VM keeps track of them through a fully qualified name and a line number. This allows breakpoints to be hit regardless of which classloader loads the class file, but it can lead to some confusion if multiple class files are loaded through different classloaders with the same qualified name, but different source code. This algorithm for determining when to stop the VM has nothing to do with actually looking for source code when the VM stops.
Looking for source code is handled by the IDE. Since even in the statically compiled world, the source file name may not match with the class name (inner classes, anonymous classes, etc). The class name cannot be used to look up the source file.
Here is a simplification of what the IDE does when it stops at a breakpoint:
- Find the class file at that breakpoint
- Get the source attribute
- Find a source file in the source lookup path that matches the name of the source attribute
- Use some heuristics if multiple source files of same name are found (I think this would be the ranking in the source lookup tab)
- Return the most appropriate source file.
(Caveat, I think that the source attribute is only the simple name of the source file (ie- no directory), so I think the IDE converts the package name to a directory structure as part of the lookup, but I might be wrong about that).
So, the lookup will fail if your dynamically compiled script does not have a proper source attribute. You can check this theory by looking at the byte code. You will have to somehow compile a script and save the bits to disk. Then you can run javap -v myScript
on it. I would bet that this is the problem. I have seen this happen before in other dynamically compiled languages.