Code Performance

In this article, I will talk about the conventional (traditional) way of benchmarking the performance of a portion of java code. Next, I will emphasize the inconvenient of this method. And in the last paragraph, I will show you how to build up a consistent way of benchmarking your java code (Thanks to JMH Benchmark library).

The Timer of The JVM: 

Often, we are confronted to measure the execution speed of a portion of code but this task is very complicated, when it comes to use traditional tools like JVM timer. Take, for example, the mathematical function "logarithm": measuring the time of calculation of the logarithm of a java float does not seem very complicated; a simple "timing" using the system date in a main ()

/**
 * PackageName PACKAGE_NAME
 * Created by mhi on 24/09/2017.
 */
public class MyBenchmarkClass
{
  public static void main(String[] args)
  {
    // start stopwatch
    long startTime = System.nanoTime();
    // Here is the code to measure
    double log42 = Math.log(42);
    // stop stopwatch
    long endTime = System.nanoTime();
    System.out.println("log(42) is computed in : " + (endTime - startTime) + " ns");
  }
}

Unfortunately, things are not that simple. Indeed, the recorded durations are quite variable and, on the other hand, they aren’t really representative; There are several reasons for this:

 

- The JVM does not execute code as it is written: the JIT in runtime can optimize the java code, reorder instructions, or even delete unnecessary instructions (this is typically what happens here: log42 is unused!)

 

- Moreover, it does not execute the same code deterministically in every execution: the JIT can decide to compile on-the-fly the bytecode in native code, instead of interpreting it (by default, in the 10000th call)

 

- Also, the physical load of the machine (CPU, memory, other processes ...) at the runtime can vary according to its overall use, and slow down the program. Basing the study on a single measure and hoping for a representative result is therefore illusory ...

 

Finally, How can we measure the performance of a given code?

 

Let's Benchmark our code Using JMH:

First steps to follow to set up the JMH library for use:

- Download JMH benchmark project generator (the Maven archetype jmh-java-benchmark-archetype)

- Keep an eye on the associated java-doc

- It is recommended to use JMH with Maven, to create a benchmark project. 

 

Creation of the projet JMH

To generate the benchmark project, use the archetype jmh-java-benchmark-archetype:

$ mvn archetype:generate \
-DinteractiveMode=false \
-DarchetypeGroupId=org.openjdk.jmh \
-DarchetypeArtifactId=jmh-java-benchmark-archetype \
-DarchetypeVersion=1.5.2 \
-DgroupId=fr.mhi \
-DartifactId=jmh-sample-benchmak \
-Dversion=1.0-SNAPSHOT

The archetype generates a JAR project called jmh-example-benchmark, containing a pom.xml, declaring dependencies to the JMH JARs and plugins required for the build.

Now, here is the way we write method to benchmark with JMH: "The archetype has also generated a benchmark class skeleton, called MyBenchmark, in which there is a testMethod () method, annotated by a @Benchmark indicating to JMH where the code to benchmark is"

/**
 * PackageName PACKAGE_NAME
 * Created by mhafidi on 24/09/2017.
 */
public class MyBenchmarkClass
{

  @Benchmark
  public void testMethod()
  {
    // This is a demo/sample template for building your JMH benchmarks. Edit as needed.
    // Put your benchmark code here.</h4>
  }

}

One or more annotated methods will be written in the same way as a JUnit test class; each will be benchmarked by JMH. It will be possible to compare different codes. I will take as an example the calculation of the logarithm, using different libraries:

- java.lang.Math.log () of the JDK8
- org.apache.commons.math3.util.FastMath.log () from Apache commons Maths
- odk.lang.FastMath.log () from javafama
- odk.lang.FastMath.logQuick () from javafama also

/**
 * PackageName PACKAGE_NAME
 * Created by mhafidi on 24/09/2017.
 */
public class MyBenchmarkClass
{

  @Benchmark
  public double benchmark_logarithm_jdk()
  {
    return java.lang.Math.log(42);
  }

  @Benchmark
  public double benchmark_logarithm_apache_common()
  {
    return org.apache.commons.math3.util.FastMath.log(42);
  }

  @Benchmark
  public double benchmark_logarithm_jafama()
  {
    return odk.lang.FastMath.log(42);
  }

  @Benchmark
  public double benchmark_logarithm_jafama_logQuick()
  {
    return odk.lang.FastMath.logQuick(42);
  }

}

Before launching the benchmark, of course, you have to build a Maven build of the project, generate technical code, assemble it with the JMH Runner, and package it all in an executable "uber" JAR benchmark.jar:

jmh-sample-benchmark$ mvn clean package

Now we will run the jar to start the benchmark:

jmh-sample-benchmark$$ java -jar target/benchmark.jar

Here we go! The benchmark runs. The execution logs are displayed on the standard output and gives the final results:

Benchmark                                                            Mode    Cnt         Score       Error    Units

MyBenchmarkClass.benchmark_logarithm_apache_common                   thrpt   200         40,112      ± 0,113  ops/us
MyBenchmarkClass.benchmark_logarithm_jafama                          thrpt   200         95,502      ± 0,255  ops/us
MyBenchmarkClass.benchmark_logarithm_jafama_logQuick                 thrpt   200         142,486     ± 0,604  ops/us
MyBenchmarkClass.benchmark_logarithm_jdk                             thrpt   200         341,494     ± 3,196  ops/us

The result of each method tested is obtained by line. The contents of the columns give us:

the mode of benchmark, means the type of measurements performed: here thrpt (for Troughput), that is to say an average flow of operations (operations performed per unit of time)
Cnt (for count) gives us the number of measurements taken to calculate our score: here 200 measurements
Score, means the calculated average throughput value
Error, represents the margin of error of this score
Units, is the unit of the score: here the score means the operations per microsecond
In our example the benchmark result gives that: java.lang.Math.log () of the JDK8 gets the best result, with an average of 341,494 operations per second!