Skip to main content
Last updated March 20, 2013 10:16, by neomatrix369
Feedicon  


Command-line arguments for build performance optimisation

Note: please refrain from using old variables or a mix of old and new ones, unless necessary, when in doubt please refer to the links provided in the References section (see bottom of the page).

The build times and outcomes of using the below flags in different combinations and, on various platforms are not known or documented, so please continue to experiment with these flags and post your responses on the Adopt OpenJDK Google Group. Responses will be used to fine-tune the below list and sample results.

The configure script is situated in the $SOURCE_CODE/jdk8_tl/common/makefiles/ folder and can be run from most places in the jdk8_tl sub-folder structure provided the relative reference is made to this location, for e.g.

 
 bash ../autoconf/configure            - if you are in the $SOURCE_CODE/jdk8_tl/common/makefiles/ folder
 
Alternatively you can run bash configure and make from the root (i.e. $SOURCE_CODE/jdk8_tl/) without having to specify any relative or absolute folder location, for e.g.
 
 bash configure                        - if you are in the $SOURCE_CODE/jdk8_tl folder
 
If it is not valid to run the configure command from within a folder, a verbose error message is shown stating that.

You run configure for the first time or any time when you wish to bring about a change in the configuration settings to your build environment. Here after the configuration settings are stored for further use. Every make command will make use of the stored configuration from the previously run configure command execution. See below in clean-up before... to know how to clear the configuration settings generated by the configure command.

configure command arguments

A number of command arguments are available with the new build system (Build-infra) that can help improve build performance and in other cases also build under critical system configuration. Some of these can be enabled when running the configure command, i.e.

--with-num-cores=n - where n > 1 (by default the the make process will detect and use of the number of available cores), creates configuration to use the specified number of cores on the CPU of your build system,

  for e.g. 
   bash [relative or absolute folder location]/configure  --with-num-cores=8


--with-memory-size - creates configuration (by default the the make process will detect and use of the maximum amount of memory) to use the specified memory (in MB) as available on the build system,

  for e.g. 
   bash [relative or absolute folder location]/configure --with-memory-size=1024


--enable-sjavac - use sjavac (Smart Javac wrapper) to do fast incremental compiles (still considered experimental), this requires a clean build the first time it's used. After the first build, incremental rebuilds will be very fast since only packages that have been changed, and their dependants will be recompiled. Please try out this argument as an experiment, it is known to be erratic at times and may not result in a successful build.

  for e.g. 
   bash [relative or absolute folder location]/configure --enable-sjavac


Other configure flag(s) - not related to optimisation
--enable-debug - will build a fastdebug version of the OpenJDK system, by creating a new directory, e.g. build/linux-x64-normal-server-fastdebug, with the new configuration in it (set the debug level to fastdebug (shorthand for --with-debug-level=fastdebug).

  for e.g. 
   bash [relative or absolute folder location]/configure --enable-debug
   or 
   bash [relative or absolute folder location]/configure --with-debug-level=fastdebug


[relative or absolute folder location]/ does not apply when positioned in the root folder i.e. the $SOURCE_CODE/jdk8_tl/.

make command arguments

A number of command arguments are available with the new build system (Build-infra) that can help improve build performance and in other cases also build under critical system configuration. Some of these can be enabled when running the make command, i.e.

JOBS=[n] - Run [n] parallel make jobs (by default the the make process will detect run as many jobs in paralell as it can, also note that MAKEOPTS="-jN" does not work as expected!),

 for e.g.
  make images JOBS=2

will run two jobs in parallel.

CFLAGS=-"[XXXX]" - where [XXX] can be CPU related flags depending on the platform on which you are running the make command,

 for e.g. 
  make clean images CFLAGS="-march=athlon-xp -msse3 -O2 -pipe -fomit-frame-pointer"

on a system running an Athlon-XP CPU the above CFLAGS parameter with the make command can be used. Similarly flags for system running an Intel CPU can be used. In correct parameters passed via the CFLAGS flag may result in an unsuccessful build.

Note: before using this flag please find out the exact CPU model/platform information using one of the two links depending on the CPU make:

 Flags for AMD: http://en.gentoo-wiki.com/wiki/Safe_Cflags/AMD
 Flags for Intel: http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel

Note: a successful build should then be validated using known methods, see validating builds.

Alternate make command arguments instead of using the --enable-sjavac argument with the configure command are:

[project-name]-only

 for e.g.
  make jdk-only


And JDK_FILTER="[classes]" - where [classes] are names of Java classes to include with the build, it limits what Java classes to recompile in the jdk project. This is done by setting the variable JDK_FILTER on the command line to,

 for e.g.
  make jdk-only JDK_FILTER="java/lang,java/io"


For Linux distro Gentoo:
CCACHE_SIZE="nG" - by default (n=2, i.e. "2G"), specifies the size of the cache used by ccache (use ccache -s to see the cache size and the currently configured limits, in addition to other various statistics like location of the cache files), use this option when performing a clean build the first time to create a cache of compiled objects to facilitate future incremental builds (theoretically the bigger the ccache size the more compiled objects it can store, and lesser cache misses it will experience, but in reality the results may differ from system-to-system),

 for e.g.
  make clean images CCACHE_SIZE="3G"


For other OSes:
The same idea as above applies except it should be implemented as below:
ccache -M nG - where n > 1 and <= maximum physical memory available on your system, for e.g. if your system has 3GB or more of memory the below command should be used:

 ccache -M 3G

followed by

  make clean images


See validating builds after you have successfully built an OpenJDK image or the entire project, to test and verify your builds.


clean-up before or after...

Information on how to clean-up your environment before or after running configure or make is provided in this section.

 make clean — remove all files generated by make, but not those generated by configure.
 make dist-clean — remove all files generated by both make and configure (forces re-run of the configure command) - this command is recommended rather than zapping the build/ folder.
 ccache -c — Clean up the cache by removing old cached files until the specified file number and cache size limits are not exceeded. This also recalculates the cache file count and size totals. Normally, it’s not needed to initiate cleanup manually as ccache keeps the cache below the specified limits at runtime and keeps statistics up to date on each compilation. Forcing a cleanup is mostly useful if you manually modify the cache contents or believe that the cache size statistics may be inaccurate.
 ccache -C — Clear the entire cache, removing all cached files.


handy Linux/Unix commands

$ cat /proc/cpuinfo - displays information about the CPU, such as its vendor (and CPU family, model and model names which should allow users to identify the CPU) and its speed (CPU clockspeed), cache size, number of siblings, cores, and CPU flags.

$ cat /proc/meminfo - displays a summary of how the kernel is managing its memory.displays both high-level and low-level statistics on memory usage.

$ dmesg - stands for "display message" or "driver message", is a command on most Linux and Unix based operating systems that prints the message buffer of the kernel.

$ lspc - is a command on Unix-like operating systems that prints detailed information about all PCI buses and devices in the system.

$ htop - an interactive process viewer for Linux (http://linux.die.net/man/1/htop). Install using command sudo apt-get install htop

sample build times

a) Below are build times as a result of using the initial configure and make commands:

 bash ../autoconf/configure
 ccache -M 3G
 make clean images 

(and ensuring ccache was installed and in use):

 ----- Build times -------
 Start 2013-02-21 21:06:57
 End   2013-02-21 21:41:06
 00:00:22 corba
 00:00:43 demos
 00:25:32 hotspot
 00:00:45 images
 00:00:20 jaxp
 00:00:24 jaxws
 00:05:37 jdk
 00:00:25 langtools
 00:34:09 TOTAL
 -------------------------

Followed by re-running the below make command:

 make clean images

resulting in the following

 ----- Build times -------
 Start 2013-02-22 00:36:46
 End   2013-02-22 00:43:42
 00:00:24 corba
 00:00:33 demos
 00:01:12 hotspot
 00:00:43 images
 00:00:21 jaxp
 00:00:25 jaxws
 00:02:53 jdk
 00:00:25 langtools
 00:06:56 TOTAL
 -------------------------


b) Below build times was a result of using the initial configure and make commands:

 bash ../autoconf/configure --with-num-cores=2
 ccache -M 3G
 make clean images 

resulting in the following

 ----- Build times -------
 Start 2013-02-21 21:48:15
 End   2013-02-21 22:25:07
 00:00:22 corba
 00:00:44 demos
 00:28:18 hotspot
 00:00:41 images
 00:00:20 jaxp
 00:00:25 jaxws
 00:05:36 jdk
 00:00:25 langtools
 00:36:52 TOTAL
 -------------------------

Followed by re-running the below make command:

 make clean images

resulting in the following

  ----- Build times -------
 Start 2013-02-21 22:26:13
 End   2013-02-21 22:32:48
 00:00:23 corba
 00:00:30 demos
 00:01:09 hotspot
 00:00:40 images
 00:00:20 jaxp
 00:00:27 jaxws
 00:02:40 jdk
 00:00:26 langtools
 00:06:35 TOTAL
 -------------------------


validating builds

Once binaries are created using the above switches, it is necessary validate them using the known JTReg test suite. See Install jtreg - on how to install and run JTReg tests.


references

Next Step

 
 
Close
loading
Please Confirm
Close