NoClassDefFoundError when dynamically adding jar files at runtime

Question

UPDATE: I think the issue may be caused by the fact that the TestFramework.jar depends on JUnit and somehow is not finding the junit jar when it (TestFramework) loads. Still not sure on a solution, if I'm right I need a way to specify the order in which jars are loaded.

First here is some background on what I'm doing: I'm trying to create a tool that will allow users to specify a java source file which is an extension of a JUnit TestCase and contains test data in a method named getTestData. This tool will then compile the java file, place the resulting class file in the "classes" directory, load the class and access the getTestData method, produce XML files representing the test data.

Obviously by allowing the user to specify a source file I have to also make sure that all dependencies for that file are included on the classpath when I compile. So for now I have hardcoded the dependencies for one specific file I'm trying to use for testing purposes. The compilation of the user's file is currently working, but loading the resulting class is causing some issues. Again it's obvious that to load the class I need to make sure that all files it depends on are on the classpath, which I have. Somehow during the loading of the class it still can't find required classes and gives me a NoClassDefFoundError.

The following is a method I'm using to create a URLClassLoader containing the URL for each of the jar files in the "lib" directory. Note that the loader is a class variable so it is available to all of my methods but is initialized in this one only.

private void loadLibJars()
{
    try {
        File jarDir = new File("lib");
        ArrayList<URL> jarFiles = new ArrayList<URL>();

        for(File file: jarDir.listFiles())
        {
            if(file.isFile() && file.getName().endsWith(".jar"))
            {
                jarFiles.add(file.toURI().toURL());
            }
        }

        for(URL url:jarFiles)
            System.out.println(url);

        URL[] urlArray = new URL[jarFiles.size()];
        for(int i=0; i<jarFiles.size(); i++)
            urlArray[i] = jarFiles.get(i);

        loader = new URLClassLoader(urlArray);
    } 
    catch (MalformedURLException ex) {
        Logger.getLogger(XmlDataGenerator.class.getName()).log(Level.SEVERE, null, ex);
    }

}

Here is the code that is actually locating the compiled class file and loading it. The first parameter is the directory to search inside of, the second parameter is a list of all the compiled class files that the user wants to generate the XML files for.

private void findAndLoadClasses(File classesDir, ArrayList<String>classFileNames)
{
    for(File classFile: classesDir.listFiles())
    {
        if(classFile.isDirectory())
        {
            System.out.println(classFile+" is a directory, searching inside of it now");
            findAndLoadClasses(classFile,classFileNames);
        }
        else if(classFile.isFile() && classFileNames.contains(classFile.getName()))
        {
            try
            {
                String fullClassName = classFile.getPath();
                fullClassName = fullClassName.substring(fullClassName.indexOf("\\")+1,fullClassName.lastIndexOf("."));
                fullClassName = fullClassName.replaceAll("\\\\", ".");

                System.out.println("Full class name: "+fullClassName);

                IvrTest testCase =  (IvrTest) Class.forName(fullClassName,true,loader).newInstance();

                System.out.println("Test data from "+classFile.getName()+": "+testCase.getTestData(...));
            }
            catch(Exception ex)
            {
                Logger.getLogger(XmlDataGenerator.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
    }
}

Something else to note that could be important is that not all the dependencies for the user-selected source file are jars, some are other source files that are also compiled and placed in the "classes" directory. The "classes" directory is included in the runtime classpath via project settings (I'm using Netbeans to simplify the GUI creation). Since I want the user to be able to dynamically add jars to the "lib" directory I do not specify the jars in the project settings.

Regarding the output and the problem I'm having here is what I see in my console when I run the code:

List of class files to search for: [MyTest.class]
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/client.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/delegate.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/model.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/common.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/framework.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/TestFramework.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/junit.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/log4j-1.2.15.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/org.springframework.beans-3.0.1.RELEASE-A.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/org.springframework.context-3.0.1.RELEASE-A.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/org.springframework.core-3.0.1.RELEASE-A.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/org.springframework.web-3.0.1.RELEASE-A.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/org.springframework.web.servlet-3.0.1.RELEASE-A.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/reports.jar
file:/C:/Documents%20and%20Settings/username/My%20Documents/NetBeansProjects/XmlDataGenerator/lib/servlet.jar
classes\my is a directory, searching inside of it now
classes\my\package is a directory, searching inside of it now
classes\my\package\name is a directory, searching inside of it now
Full class name: my.package.name.MyTest
Exception in thread "AWT-EventQueue-0" java.lang.NoClassDefFoundError: junit/framework/TestCase
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:296)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at xmldatagenerator.main.XmlDataGenerator.findAndLoadClasses(XmlDataGenerator.java:299)
        at xmldatagenerator.main.XmlDataGenerator.findAndLoadClasses(XmlDataGenerator.java:285)
        at xmldatagenerator.main.XmlDataGenerator.findAndLoadClasses(XmlDataGenerator.java:285)
        at xmldatagenerator.main.XmlDataGenerator.findAndLoadClasses(XmlDataGenerator.java:285)
        at xmldatagenerator.main.XmlDataGenerator.findAndLoadClasses(XmlDataGenerator.java:285)
        at xmldatagenerator.main.XmlDataGenerator.findAndLoadClasses(XmlDataGenerator.java:285)
        at xmldatagenerator.main.XmlDataGenerator.generateXmlBtnActionPerformed(XmlDataGenerator.java:242)
        at xmldatagenerator.main.XmlDataGenerator.access$300(XmlDataGenerator.java:33)
        at xmldatagenerator.main.XmlDataGenerator$4.actionPerformed(XmlDataGenerator.java:104)
        at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1995)
        at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2318)
        at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
        at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
        at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
        at java.awt.Component.processMouseEvent(Component.java:6267)
        at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
        at java.awt.Component.processEvent(Component.java:6032)
        at java.awt.Container.processEvent(Container.java:2041)
        at java.awt.Component.dispatchEventImpl(Component.java:4630)
        at java.awt.Container.dispatchEventImpl(Container.java:2099)
        at java.awt.Component.dispatchEvent(Component.java:4460)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4577)
        at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4238)
        at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4168)
        at java.awt.Container.dispatchEventImpl(Container.java:2085)
        at java.awt.Window.dispatchEventImpl(Window.java:2478)
        at java.awt.Component.dispatchEvent(Component.java:4460)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Caused by: java.lang.ClassNotFoundException: junit.framework.TestCase
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        ... 73 more

XmlDataGenerator.java line 299 is the line that reads IvrTest testCase = (IvrTest) Class.forName(fullClassName,true,loader).newInstance();

At this point I'm at a loss because I have tried everything I can think of such as:

1. Adding the junit.jar to the runtime classpath (in project settings) to make sure this error goes away, which it does. This doesn't help fix the problem though since I want to load jars dynamically.
2. Specifying only the junit.jar instead of trying to load all jars in the directory. This still results in a failure same as above.
3. Removing the 'classes' directory from the project config and using the same code to bring in those classes. This works to bring in the compiled sources but does not fix the jar issue.
4. Using relative paths in the URLs instead of absolute. This does not help.
5. Adding extra System.out before and after attempting to load the jars - the results are the same as when I print the list of URLs during the creation of the URLClassLoader nothing has changed between when I first created it and after the load is attempted.

The only thing I can come up with is that somehow the URLClassLoader I'm trying to use is not being used but I have no idea why that would be happening. Any help would be greatly appreciated. Thanks for your time.

Answer 1

I found the problem - since the TestFramework jar was included in the project classpath it would be loaded using whatever default loader is used. Then when I add more jars later the loader that handled the TestFramework jar does not see them so it thinks the dependencies are missing. To fix it I created two separate jars, one with interface classes only and the other with all classes so that I could load it together with its dependencies.

Answer 2

Getting it to always use your classloader can be tricky. I remember running into this issue and ended up having to create a classloader that is then used to start up the entire app.

It seemed that a lot of classes wanted to use the standard class loader instead of the one that you loaded them with.

You can also try setting the classloader that you want Java to use using Thread.currentThread().setContextClassLoader()

One rule about classloaders I remember is that your classloader should always try its parent classloader first before trying to load the class itself. I see URLClassLoader lets you pass a parent classloader in when constructing. Try passing in the current classloader.