How to detect whether String.substring copies the character data

Question

I know that for Oracle Java 1.7 update 6 and newer, when using String.substring, the internal character array of the String is copied, and for older versions, it is shared. But I found no offical API that would tell me the current behavior.

Use Case

My use case is: In a parser, I like to detect whether String.substring copies or shares the underlying character array. The problem is, if the character array is shared, then my parser needs to explicitly "un-share" using new String(s) to avoid memory problems. However, if String.substring anyway copies the data, then this is not necessary, and explicitly copying the data in the parser could be avoided. Use case:

// possibly the query is very very large
String query = "select * from test ...";
// the identifier is used outside of the parser
String identifier = query.substring(14, 18);

// avoid if possible for speed,
// but needed if identifier internally 
// references the large query char array
identifier = new String(identifier);

What I Need

Basically, I would like to have a static method boolean isSubstringCopyingForSure() that would detect if new String(..) is not needed. I'm OK if detection doesn't work if there is a SecurityManager. Basically, the detection should be conservative (to avoid memory problems, I'd rather use new String(..) even if not necessary).

Options

I have a few options, but I'm not sure if they are reliable, specially for non-Oracle JVMs:

Checking for the String.offset field

/**
 * @return true if substring is copying, false if not or if it is not clear
 */
static boolean isSubstringCopyingForSure() {
    if (System.getSecurityManager() != null) {
        // we can not reliably check it
        return false;
    }
    try {
        for (Field f : String.class.getDeclaredFields()) {
            if ("offset".equals(f.getName())) {
                return false;
            }
        }
        return true;
    } catch (Exception e) {
        // weird, we do have a security manager?
    }
    return false;
}

Checking the JVM version

static boolean isSubstringCopyingForSure() {
    // but what about non-Oracle JREs?
    return System.getProperty("java.vendor").startsWith("Oracle") &&
           System.getProperty("java.version").compareTo("1.7.0_45") >= 0;
}

Checking the behavior There are two options, both are rather complicated. One is create a string using custom charset, then create a new string b using substring, then modify the original string and check whether b is also changed. The second options is create huge string, then a few substrings, and check the memory usage.

Answer 1

Right, indeed this change was made in 7u6. There is no API change for this, as this change is strictly an implementation change, not an API change, nor is there an API to detect which behavior the running JDK has. However, it is certainly possible for applications to notice a difference in performance or memory utilization because of the change. In fact, it's not difficult to write a program that works in 7u4 but fails in 7u6 and vice-versa. We expect that the tradeoff is favorable for the majority of applications, but undoubtedly there are applications that will suffer from this change.

It's interesting that you're concerned about the case where string values are shared (prior to 7u6). Most people I've heard from have the opposite concern, where they like the sharing and the 7u6 change to unshared values is causing them problems (or, they're afraid it will cause problems).

In any case the thing to do is measure, not guess!

First, compare the performance of your application between similar JDKs with and without the change, e.g. 7u4 and 7u6. Probably you should be looking at GC logs or other memory monitoring tools. If the difference is acceptable, you're done!

Assuming that the shared string values prior to 7u6 cause a problem, the next step is to try the simple workaround of new String(s.substring(...)) to force the string value to be unshared. Then measure that. Again, if the performance is acceptable on both JDKs, you're done!

If it turns out that in the unshared case, the extra call to new String() is unacceptable, then probably the best way to detect this case and make the "unsharing" call conditional is to reflect on a String's value field, which is a char[], and get its length:

int getValueLength(String s) throws Exception {
    Field field = String.class.getDeclaredField("value");
    field.setAccessible(true);
    return ((char[])field.get(s)).length;
}

Consider a string resulting from a call to substring() that returns a string shorter than the original. In the shared case, the substring's length() will differ from the length of the value array retrieved as shown above. In the unshared case, they'll be the same. For example:

String s = "abcdefghij".substring(2, 5);
int logicalLength = s.length();
int valueLength = getValueLength(s);

System.out.printf("%d %d ", logicalLength, valueLength);
if (logicalLength != valueLength) {
    System.out.println("shared");
else
    System.out.println("unshared");

On JDKs older than 7u6, the value's length will be 10, whereas on 7u6 or later, the value's length will be 3. In both cases, of course, the logical length will be 3.

Answer 2

This is not a detail you need to care about. No really! Just call identifier = new String(identifier) in both cases (JDK6 and JDK7). Under JDK6 it will create a copy (as desired). Under JDK7, because the substring is already a unique string the constructor is essentially a no-op (no copy is performed -- read the code). Sure there is a slight overhead of object creation, but because of object reuse in the Younger generation, I challenge you to qualify a performance difference.

Answer 3

In older Java versions, String.substring(..) will use the same char array as the original, with a different offset and count.

In the latest Java versions (according to the comment by Thomas Mueller: since 1.7 Update 6), this has changed, and substrings are now be created with a new char array.

If you parse lots of sources, the best way to deal with it is to avoid checking the Strings' internals, but anticipate this effect and always create new Strings where you need them (as in the first code block in your question).

String identifier = query.substring(14, 18);
// older Java versions: backed by same char array, different offset and count
// newer Java versions: copy of the desired run of the original char array

identifier = new String(identifier);
// older Java versions: when the backed char array is larger than count, a copy of the desired run will be made
// newer Java versions: trivial operation, create a new String instance which is backed by the same char array, no copy needed.

That way, you end up with the same result with both variants, without having to distinguish them and without unnecessary array copy overhead.

Answer 4

Are you sure, that making string copy is really expensive? I belive that JVM optimizer has intrinsics about strings and avoids unnecessary copies. Also large texts are parsed with one-pass algorithms such as LALR automata, generated by compiler compilers. So, the parser input usually be an java.io.Reader or another streaming interface, not a solid String. Parsing is usially expensive itself, still not as expensive as type checking. I don't think that copying strings is a real bottleneck. You better experience with profiler and with microbenchmarks before your assumptions.