More on String and the StringBuilder Performance

In February, I demonstrated the most efficient way I could think of for conditionally building a string. This compared the use of StringBuilder and the ‘+=’ operator whilst checking for null values. The results showed that StringBuilder.append(..) was much faster than using ‘+=’.

Today’s blog covers the much simpler scenario: trying to decide which is the quickest way of of creating a simple string. In order to test this out, I modified the code I’d used last time adding a couple of new methods:

public class StringPerformance2 {

  public static void main(String[] args) {

    long start = System.nanoTime();
    plusMethod("Test Message", "Watery Lane");
    long duration = System.nanoTime() - start;
    System.out.println("Using '+' took " + duration + "nS");

    start = System.nanoTime();
    goodMethodNoNullChecks("Test Message", "Watery Lane");
    duration = System.nanoTime() - start;
    System.out.println("Using StringBuilder without null checks took " + duration + "nS");
  }

  /*
   * Use this method - it's optimised
   */
  private static String plusMethod(String message, String location) {

    return "Exception closing down " + message + " at " + location
        + " Press any key to exit";
  }

  private static String goodMethodNoNullChecks(String message, String location) {

    // Create a message the proper way..
    StringBuilder str = new StringBuilder("Exception closing down ");
    str.append(message);
    str.append(" at ");
    str.append(location);
    str.append(" Press any key to exit");
    return str.toString();
  }
}

Running this code gave methe following results:

Using '+' took 9034nS
Using StringBuilder without null checks took 9855nS

As you can see, using the ‘+’ operator in this case is 821nS quicker, but does this mean anything, especially when trying to measure such small intervals? My guess is that it doesn’t although you can answer that the JVM does some special jiggery-pokery in converting the String ‘+’ operator into a StringBuilder.

I'd also guess that if you want a better comparison then you’d need to do a large number of iterations of the test and then take an average... which is exactly what I did.

public class StringPerformance3 {

  public static void main(String[] args) {

    // Create Objects first
    UsePlusEqualsConcatenationWithNullCheck test1 = new UsePlusEqualsConcatenationWithNullCheck();
    UseStringBuilderWithNullCheck test2 = new UseStringBuilderWithNullCheck();
    UseStringBuilderWithOutNullCheck test3 = new UseStringBuilderWithOutNullCheck();
    UsePlusOperator test4 = new UsePlusOperator();

    final int iterations = 100;

    // This is an extra test - it seems to set up the JVM
    // before doing the real tests - probably allocating memory etc.
    // Uncommenting this line changes the results...
    // test1.timePerformance(iterations * 20);

    long duration1 = test1.timePerformance(iterations) / iterations;
    long duration2 = test2.timePerformance(iterations) / iterations;
    long duration3 = test3.timePerformance(iterations) / iterations;
    long duration4 = test4.timePerformance(iterations) / iterations;

    System.out.println("Using String += took " + duration1 + "nS");
    System.out.println("Using StringBuilder took " + duration2 + "nS");
    System.out.println("Using StringBuilder without null checks took " + duration3 + "nS");
    System.out.println("Using '+' took " + duration4 + "nS");
  }

  static abstract class TestMethod {

    long timePerformance(int iterations) {

      long duration = 0;
      long startTime = System.nanoTime();
      for (int i = 0; i < iterations; i++) {
        testMethod("Test Message", "Watery Lane");
        duration += (System.nanoTime() - startTime);
      }
      return duration;
    }

    public abstract String testMethod(String message1, String message2);
  }

  private static class UsePlusEqualsConcatenationWithNullCheck extends TestMethod {

    /**
     * @see misc.StringPerformance3.TestMethod#testMethod(java.lang.String,
     *      java.lang.String)
     */
    @Override
    public String testMethod(String message, String location) {

      String str = "Exception closing down ";

      if (message != null)
        str += message;

      if (location != null) {
        str += " at ";
        str += location;

        str += " Press any key to exit";
      }

      return str;
    }
  }

  private static class UseStringBuilderWithNullCheck extends TestMethod {

    @Override
    public String testMethod(String message, String location) {

      // Create a message the proper way..
      StringBuilder str = new StringBuilder("Exception closing down ");

      if (message != null)
        str.append(message);

      if (location != null) {
        str.append(" at ");
        str.append(location);
      }

      str.append(" Press any key to exit");

      return str.toString();
    }
  }

  private static class UseStringBuilderWithOutNullCheck extends TestMethod {

    @Override
    public String testMethod(String message, String location) {

      StringBuilder str = new StringBuilder("Exception closing down ");
      str.append(message);
      str.append(" at ");
      str.append(location);
      str.append(" Press any key to exit");
      return str.toString();
    }
  }

  private static class UsePlusOperator extends TestMethod {

    @Override
    public String testMethod(String message, String location) {

      return "Exception closing down " + message + " at " + location
          + " Press any key to exit";
    }
  }

}

...and this still proves very little. The only point of interest to note is the line adding an extra test that’s commented out. It turns out that adding this extra test drastically changes the results of the whole test. Compare the two sets of results below:

The first set runs the test with the extra test commented out, so that the first test is the '+=' operator test:

Using String += took 607876nS
Using StringBuilder took 290837nS
Using StringBuilder without null checks took 255126nS
Using '+' took 183696nS

Notice that the first value ‘Using String +=’ is twice the second and about three times the third. Compare that with the next set of results. In this case the extra test line was uncommented the test ran:

Using String += took 149886nS
Using StringBuilder took 136833nS
Using StringBuilder without null checks took 138454nS
Using '+' took 119271nS

Notice that the results are much similar with ‘String +=’ giving a comparable value to the other three. From this it seems that any performance test code should really be taken with a pinch of salt as the results will depend upon the state of the JVM before the code runs, the state of the machine that’s running the test together with the construction of the code. This leads me to think that knowing that concatenating strings with a ‘+’ operator rather than a StringBuilder.append() call gives you slightly better performance, but real tuning and performance enhancements should only be done on production code as the shape of the code and condition of the JVM and physical machine are important, which seems to lead me back to Jackson's Rules for code optimisation.