How should I load Jars dynamically at runtime?

Question

Why is it so hard to do this in Java? If you want to have any kind of module system you need to be able to load jars dynamically. I'm told there's a way of doing it by writing your own ClassLoader, but that's a lot of work for something that should (in my mind at least) be as easy as calling a method with a jar file as its argument.

Any suggestions for simple code that does this?

Answer 1

The reason it's hard is security. Classloaders are meant to be immutable; you shouldn't be able to willy-nilly add classes to it at runtime. I'm actually very surprised that works with the system classloader. Here's how you do it making your own child classloader:

URLClassLoader child = new URLClassLoader(
        new URL[] {myJar.toURI().toURL()},
        this.getClass().getClassLoader()
);
Class classToLoad = Class.forName("com.MyClass", true, child);
Method method = classToLoad.getDeclaredMethod("myMethod");
Object instance = classToLoad.newInstance();
Object result = method.invoke(instance);

Painful, but there it is.

Answer 2

The following solution is hackish, as it uses reflection to bypass encapsulation, but it works flawlessly:

File file = ...
URL url = file.toURI().toURL();

URLClassLoader classLoader = (URLClassLoader)ClassLoader.getSystemClassLoader();
Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
method.invoke(classLoader, url);

Answer 3

You should take a look at OSGi, e.g. implemented in the Eclipse Platform. It does exactly that. You can install, uninstall, start and stop so called bundles, which are effectively JAR files. But it does a little more, as it offers e.g. services that can be dynamically discovered in JAR files at runtime.

Or see the specification for the Java Module System.

Answer 4

How about the JCL class loader framework? I have to admit, I haven't used it, but it looks promising.

Usage example:

JarClassLoader jcl = new JarClassLoader();
jcl.add("myjar.jar"); // Load jar file  
jcl.add(new URL("http://myserver.com/myjar.jar")); // Load jar from a URL
jcl.add(new FileInputStream("myotherjar.jar")); // Load jar file from stream
jcl.add("myclassfolder/"); // Load class folder  
jcl.add("myjarlib/"); // Recursively load all jar files in the folder/sub-folder(s)

JclObjectFactory factory = JclObjectFactory.getInstance();
// Create object of loaded class  
Object obj = factory.create(jcl, "mypackage.MyClass");

Answer 5

Here is a version that is not deprecated. I modified the original to remove the deprecated functionality.

/**************************************************************************************************
 * Copyright (c) 2004, Federal University of So Carlos                                           *
 *                                                                                                *
 * All rights reserved.                                                                           *
 *                                                                                                *
 * Redistribution and use in source and binary forms, with or without modification, are permitted *
 * provided that the following conditions are met:                                                *
 *                                                                                                *
 *     * Redistributions of source code must retain the above copyright notice, this list of      *
 *       conditions and the following disclaimer.                                                 *
 *     * Redistributions in binary form must reproduce the above copyright notice, this list of   *
 *     * conditions and the following disclaimer in the documentation and/or other materials      *
 *     * provided with the distribution.                                                          *
 *     * Neither the name of the Federal University of So Carlos nor the names of its            *
 *     * contributors may be used to endorse or promote products derived from this software       *
 *     * without specific prior written permission.                                               *
 *                                                                                                *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS                            *
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT                              *
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR                          *
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR                  *
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,                          *
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,                            *
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR                             *
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF                         *
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING                           *
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS                             *
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.                                   *
 **************************************************************************************************/
/*
 * Created on Oct 6, 2004
 */
package tools;

import java.io.File;
import java.io.IOException;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;

/**
 * Useful class for dynamically changing the classpath, adding classes during runtime. 
 */
public class ClasspathHacker {
    /**
     * Parameters of the method to add an URL to the System classes. 
     */
    private static final Class<?>[] parameters = new Class[]{URL.class};

    /**
     * Adds a file to the classpath.
     * @param s a String pointing to the file
     * @throws IOException
     */
    public static void addFile(String s) throws IOException {
        File f = new File(s);
        addFile(f);
    }

    /**
     * Adds a file to the classpath
     * @param f the file to be added
     * @throws IOException
     */
    public static void addFile(File f) throws IOException {
        addURL(f.toURI().toURL());
    }

    /**
     * Adds the content pointed by the URL to the classpath.
     * @param u the URL pointing to the content to be added
     * @throws IOException
     */
    public static void addURL(URL u) throws IOException {
        URLClassLoader sysloader = (URLClassLoader)ClassLoader.getSystemClassLoader();
        Class<?> sysclass = URLClassLoader.class;
        try {
            Method method = sysclass.getDeclaredMethod("addURL",parameters);
            method.setAccessible(true);
            method.invoke(sysloader,new Object[]{ u }); 
        } catch (Throwable t) {
            t.printStackTrace();
            throw new IOException("Error, could not add URL to system classloader");
        }        
    }

    public static void main(String args[]) throws IOException, SecurityException, ClassNotFoundException, IllegalArgumentException, InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException{
        addFile("C:\\dynamicloading.jar");
        Constructor<?> cs = ClassLoader.getSystemClassLoader().loadClass("test.DymamicLoadingTest").getConstructor(String.class);
        DymamicLoadingTest instance = (DymamicLoadingTest)cs.newInstance();
        instance.test();
    }
}

Answer 6

With Java 9, the answers with URLClassLoader now give an error like:

java.lang.ClassCastException: java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to java.base/java.net.URLClassLoader

This is because the class loaders used have changed. Instead, to add to the system class loader, you can use the Instrumentation API through an agent.

Create an agent class:

package ClassPathAgent;

import java.io.IOException;
import java.lang.instrument.Instrumentation;
import java.util.jar.JarFile;

public class ClassPathAgent {
    public static void agentmain(String args, Instrumentation instrumentation) throws IOException {
        instrumentation.appendToSystemClassLoaderSearch(new JarFile(args));
    }
}

Add META-INF/MANIFEST.MF and put it in a JAR file with the agent class:

Manifest-Version: 1.0
Agent-Class: ClassPathAgent.ClassPathAgent

Run the agent:

This uses the byte-buddy-agent library to add the agent to the running JVM:

import java.io.File;

import net.bytebuddy.agent.ByteBuddyAgent;

public class ClassPathUtil {
    private static File AGENT_JAR = new File("/path/to/agent.jar");

    public static void addJarToClassPath(File jarFile) {
        ByteBuddyAgent.attach(AGENT_JAR, String.valueOf(ProcessHandle.current().pid()), jarFile.getPath());
    }
}

Answer 7

The best I've found is org.apache.xbean.classloader.JarFileClassLoader which is part of the XBean project.

Here's a short method I've used in the past, to create a class loader from all the lib files in a specific directory

public void initialize(String libDir) throws Exception {
    File dependencyDirectory = new File(libDir);
    File[] files = dependencyDirectory.listFiles();
    ArrayList<URL> urls = new ArrayList<URL>();
    for (int i = 0; i < files.length; i++) {
        if (files[i].getName().endsWith(".jar")) {
        urls.add(files[i].toURL());
        //urls.add(files[i].toURI().toURL());
        }
    }
    classLoader = new JarFileClassLoader("Scheduler CL" + System.currentTimeMillis(), 
        urls.toArray(new URL[urls.size()]), 
        GFClassLoader.class.getClassLoader());
}

Then to use the classloader, just do:

classLoader.loadClass(name);

Answer 8

If you are working on Android, the following code works:

String jarFile = "path/to/jarfile.jar";
DexClassLoader classLoader = new DexClassLoader(jarFile, "/data/data/" + context.getPackageName() + "/", null, getClass().getClassLoader());
Class<?> myClass = classLoader.loadClass("MyClass");

Answer 9

The solution proposed by jodonnell is good but should be a little bit enhanced. I used this post to develop my application with success.

Assign the current thread

Firstly we have to add

Thread.currentThread().setContextClassLoader(classLoader);

or you will not able to load resource (such as spring/context.xml) stored into the jar.

Do not include

your jars into the parent class loader or you will not able to understand who is loading what.

see also Problem reloading a jar using URLClassLoader

However, OSGi framework remain the best way.

Answer 10

Another version of the hackish solution from Allain, that also works on JDK 11:

File file = ...
URL url = file.toURI().toURL();
URLClassLoader sysLoader = new URLClassLoader(new URL[0]);

Method sysMethod = URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{URL.class});
sysMethod.setAccessible(true);
sysMethod.invoke(sysLoader, new Object[]{url});

On JDK 11 it gives some deprecation warnings but serves as a temporary solution those who use Allain solution on JDK 11.

Answer 11

Another working solution using Instrumentation that works for me. It has the advantage of modifying the class loader search, avoiding problems on class visibility for dependent classes:

Create an Agent Class

For this example, it has to be on the same jar invoked by the command line:

package agent;

import java.io.IOException;
import java.lang.instrument.Instrumentation;
import java.util.jar.JarFile;

public class Agent {
   public static Instrumentation instrumentation;

   public static void premain(String args, Instrumentation instrumentation) {
      Agent.instrumentation = instrumentation;
   }

   public static void agentmain(String args, Instrumentation instrumentation) {
      Agent.instrumentation = instrumentation;
   }

   public static void appendJarFile(JarFile file) throws IOException {
      if (instrumentation != null) {
         instrumentation.appendToSystemClassLoaderSearch(file);
      }
   }
}

Modify the MANIFEST.MF

Adding the reference to the agent:

Launcher-Agent-Class: agent.Agent
Agent-Class: agent.Agent
Premain-Class: agent.Agent

I actually use Netbeans, so this post helps on how to change the manifest.mf

Running

The Launcher-Agent-Class is only supported on JDK 9+ and is responsible for loading the agent without explicitly defining it on the command line:

 java -jar <your jar>

The way that works on JDK 6+ is defining the -javaagent argument:

java -javaagent:<your jar> -jar <your jar>

Adding new Jar at Runtime

You can then add jar as necessary using the following command:

Agent.appendJarFile(new JarFile(<your file>));

I did not find any problems using this on documentation.

Answer 12

Here is a quick workaround for Allain's method to make it compatible with newer versions of Java:

ClassLoader classLoader = ClassLoader.getSystemClassLoader();
try {
    Method method = classLoader.getClass().getDeclaredMethod("addURL", URL.class);
    method.setAccessible(true);
    method.invoke(classLoader, new File(jarPath).toURI().toURL());
} catch (NoSuchMethodException e) {
    Method method = classLoader.getClass()
            .getDeclaredMethod("appendToClassPathForInstrumentation", String.class);
    method.setAccessible(true);
    method.invoke(classLoader, jarPath);
}

Note that it relies on knowledge of internal implementation of specific JVM, so it's not ideal and it's not a universal solution. But it's a quick and easy workaround if you know that you are going to use standard OpenJDK or Oracle JVM. It might also break at some point in future when new JVM version is released, so you need to keep that in mind.

Answer 13

This can be a late response, I can do it as this (a simple example for fastutil-8.2.2.jar) using jhplot.Web class from DataMelt (http://jwork.org/dmelt)

import jhplot.Web;
Web.load("http://central.maven.org/maven2/it/unimi/dsi/fastutil/8.2.2/fastutil-8.2.2.jar"); // now you can start using this library

According to the documentation, this file will be download inside "lib/user" and then dynamically loaded, so you can start immediately using classes from this jar file in the same program.

Answer 14

please take a look at this project that i started: proxy-object lib

This lib will load jar from file system or any other location. It will dedicate a class loader for the jar to make sure there are no library conflicts. Users will be able to create any object from the loaded jar and call any method on it. This lib was designed to load jars compiled in Java 8 from the code base that supports Java 7.

To create an object:

    File libDir = new File("path/to/jar");

    ProxyCallerInterface caller = ObjectBuilder.builder()
            .setClassName("net.proxy.lib.test.LibClass")
            .setArtifact(DirArtifact.builder()
                    .withClazz(ObjectBuilderTest.class)
                    .withVersionInfo(newVersionInfo(libDir))
                    .build())
            .build();
    String version = caller.call("getLibVersion").asString();

ObjectBuilder supports factory methods, calling static functions, and call back interface implementations. i will be posting more examples on the readme page.

Answer 15

I personally find that java.util.ServiceLoader does the job pretty well. You can get an example here.