Why does my custom SecurityManager cause exceptions the 16th time I create an object with Constructor.newInstance?

Question

I am currently working on developing a small Java application in which trusted code must be run alongside untrusted code. To accomplish this, I have installed a custom SecurityManager that throws SecurityExceptions any time a permission is checked.

As a bridge between the trusted and untrusted code, I have a thread that uses Constructor.newInstance() to instantiate an object of an untrusted type. At the time that it makes this call, the security manager is configured to block everything. Interestingly, the first 15 times that I try to create objects using Constructor.newInstance(), everything works fine, but the 16th time I get a SecurityException.

I've managed to get this down to a simple test program:

import java.lang.reflect.*;
import java.security.*;

public class Main {
    /* Track how many instances have been created so that we can see when the exception
     * is thrown.
     */
    private static int numInstances = 0;
    public Main() {
        System.out.println("Number created: " + ++numInstances);
    }

    public static void main(String[] args) {
        /* Get the constructor for Main so that we can instantiate everything
         * later on.
         */
        Constructor<Main> ctor;
        try {
            ctor = Main.class.getConstructor();
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
            return;
        }

        /* Install a super prohibitive security manager that disallows all operations. */
        System.setSecurityManager(new SecurityManager() {
            @Override
            public void checkPermission(Permission p) {
                /* Nothing is allowed - any permission check causes a     security
                 * exception.
                 */
                throw new SecurityException("Not permitted: " + p);
            }
        });

        /* Continuously create new Main objects. */
        try {
            while (true) {
                ctor.newInstance();
            }
        } catch (Exception e) {
            e.printStackTrace();
            return;
        }
    }
}

This program installs a SecurityManager whose checkPermission always throws an exception regardless of what permission is requested. It then sits in a loop and uses ctor.newInstance() to instantiate a harmless Main object that prints out the number of instances generated so far. The output of this program, on my system, is as follows:

Number created: 1
Number created: 2
Number created: 3
Number created: 4
Number created: 5
Number created: 6
Number created: 7
Number created: 8
Number created: 9
Number created: 10
Number created: 11
Number created: 12
Number created: 13
Number created: 14
Number created: 15
java.lang.SecurityException: Not permitted: ("java.lang.RuntimePermission" "createClassLoader")
    at Main$1.checkPermission(Main.java:32)
    at java.lang.SecurityManager.checkCreateClassLoader(SecurityManager.java:611)
    at java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:274)
    at java.lang.ClassLoader.<init>(ClassLoader.java:316)
    at sun.reflect.DelegatingClassLoader.<init>(ClassDefiner.java:72)
    at sun.reflect.ClassDefiner$1.run(ClassDefiner.java:60)
    at sun.reflect.ClassDefiner$1.run(ClassDefiner.java:58)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:57)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:396)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:395)
    at sun.reflect.MethodAccessorGenerator.generateConstructor(MethodAccessorGenerator.java:94)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:48)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at Main.main(Main.java:39)

According to the Javadoc for RuntimePermission, the createClassLoader permission is a risky one to grant:

This is an extremely dangerous permission to grant. Malicious applications that can instantiate their own class loaders could then load their own rogue classes into the system. These newly loaded classes could be placed into any protection domain by the class loader, thereby automatically granting the classes the permissions for that domain.

I have two questions:

1. What specifically is causing this error? Why is it that on the 16th time around, I get a request for a classloader? I suspect this has to do with Java trying to optimize the reflection by generating bytecode to directly instantiate the object, but I'm not sure.

2. Without whitelisting the createClassLoader privilege, which is dangerous, is there a way to instantiate the untrusted objects from trusted code?

3. Am I fundamentally approaching this the wrong way?

Thanks!

Answer 1

Check out this at GrepCode:

72  private static int     inflationThreshold = 15;

15 is the default value for the inflation threshold, the count of reflective calls before a more aggressive optimization is introduced in NativeConstructorAccessorImpl:

47  if (++numInvocations > ReflectionFactory.inflationThreshold()) {
48      ConstructorAccessorImpl acc = (ConstructorAccessorImpl)
49          new MethodAccessorGenerator().
50              generateConstructor(c.getDeclaringClass(),
51                                  c.getParameterTypes(),
52                                  c.getExceptionTypes(),
53                                  c.getModifiers());
54      parent.setDelegate(acc);

And that particular code causes a new class loader to be instantiated, resulting in your exception at the 16th iteration. Bytecode generation happens in the MethodAccessorGenerator class, and this is the most interesting bit:

387  // Load class
388  vec.trim();
389  final byte[] bytes = vec.getData();
390  // Note: the class loader is the only thing that really matters
391  // here -- it's important to get the generated code into the
392  // same namespace as the target class. Since the generated code
393  // is privileged anyway, the protection domain probably doesn't
394  // matter.
395  return AccessController.doPrivileged(
396      new PrivilegedAction<MagicAccessorImpl>() {
397          public MagicAccessorImpl run() {
398                  try {
399                  return (MagicAccessorImpl)
400                  ClassDefiner.defineClass
401                          (generatedName,
402                           bytes,
403                           0,
404                           bytes.length,
405                           declaringClass.getClassLoader()).newInstance();
406                  } catch (InstantiationException e) {
407                      throw (InternalError)
408                          new InternalError().initCause(e);
409                  } catch (IllegalAccessException e) {
410                      throw (InternalError)
411                          new InternalError().initCause(e);
412                  }
413              }
414          });

As for granting that permission, you do still have the choice of carefully forming a protection domain for your code, to which you grant the permission, without granting it to foreign code.