AdoptOpenJDK HotSpot vs OpenJ9

Eclipse OpenJ9 version 0.17.0 was released 2019-10-18. It is available with AdoptOpenJDK that comes in 2 versions – HotSpot and OpenJ9. Here we benchmark those 2 versions doing some linear algebra. They are compared using the latest version of OpenJDK (v13) but also include an alternative with HotSpot on OpenJDK 8, that corresponds to a (still) very commonly used alternative.

The JDK:s used are:

• JDK 13.0.1, Eclipse OpenJ9 VM, openj9-0.17.0
• JDK 13.0.1, OpenJDK 64-Bit Server VM, 13.0.1+9
• JDK 1.8.0_232, OpenJDK 64-Bit Server VM, 25.232-b09

…all downloaded from https://adoptopenjdk.net on 2019-11-03.

The JDK:s are evaluated by executing 3 different linear algebra operations implemented in 3 different code libraries. This is done for a number of different matrix sizes ranging from 100×100 to 1000×1000.

Results

The charts show the number of operations per minute on the y-axis and matrix size (square) on the x-axis.

Matrix Multiplication

Solve Equation System

Singular Value Decomposition

Interpretation

First thing to realise is that any clearly visible line separation in the charts represents a significant speed difference, and there is (up to) a x20 speed difference between the slowest and fastest library-JDK combinations.

Funny thing is that the JDK:s “win” one benchmark each (which JDK was used in the fastest library-JDK combination for each of the operations).

The results are somewhat contradictive. OpenJ9, for instance, is both the slowest and fastest alternative when doing matrix multiplication, depending on which library is used.

With HotSpot, OpenJDK 13 is mostly a slight improvement over OpenJDK 8, but sometimes the differences are huge – and v13 in not always faster than v8.

Benchmark Details

The tests were executed on a Google Cloud Platform Compute Engine: n1-highmem-4 (4 vCPUs Intel Skylake Xeon, 26 GB memory) running CentOS.

The benchmark code is here: https://github.com/optimatika/ojAlgo-linear-algebra-benchmark

The raw results (*.log and *.csv files) can be found here: https://github.com/optimatika/ojAlgo-linear-algebra-benchmark/tree/develop/results/2019/11/adopt-v13