japex
  1. japex
  2. JAPEX-24

Support for JVM forking to run benchmarks that use native code

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: current
    • Fix Version/s: milestone 1
    • Component/s: engine
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

    • Issuezilla Id:
      24

      Description

      Due to the way each Japex driver is contained in it's own classloader it is not
      always possible to create benchmarks that rely on native code.

      The main problem is that the running of different drivers can result in multiple
      calls to System.loadLibrary().

      Japex should be able to support the following benchmark scenarios that involve
      native code:

      • Benchmarks that compare Java implementations that share the same native code.
      • Benchmarks that compare Java implementations that contain different native
        code.

      When benchmarking drivers that share the same native code you have to ensure
      that System.loadLibrary() is not called multiple times in the same JVM instance.
      Similarly for running tests that use different versions of a library you will
      need to use a different JVM if the shared library has the same name.

      The former case could be solved by having a hook in the Japex runtime that
      allows you to customize the JapexClassLoader to ensure that the native library
      is only loaded once but this assumes that you have control over the native
      library which is not always the case: for example your benchmark could rely on a
      3rd party Java library that internally calls System.loadLibrary(). Having such a
      hook would not also work for the 2nd case where you want to test different
      version of the same native library together.

      One mechanism that could help here is if Japex supported the following:

      • A japex.libraryPath property that can be set per driver instance.
      • An attribute on the driver called 'fork' that would fork a new JVM.

      Even if the native library could be shared across drivers, you probably still
      want complete isolation to ensure micro-benchmarks are fair, so forking should
      probably the recommended solution for any kind of tests that involve a native
      dependency.

        Activity

        jeffrey_sinclair created issue -
        kenaiadmin made changes -
        Field Original Value New Value
        issue.field.bugzillaimportkey 24 19482
        Hide
        bimargulies added a comment -

        One approach is to wait for the maven surefire refactoring to produce easily stealable forking code. Another is to borrow from ant.

        Show
        bimargulies added a comment - One approach is to wait for the maven surefire refactoring to produce easily stealable forking code. Another is to borrow from ant.

          People

          • Assignee:
            Santiago Pericas-Geertsen
            Reporter:
            jeffrey_sinclair
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated: