[GLASSFISH-19893] performance hotspot in org.glassfish.jersey.server.ApplicationHandler.initialize Created: 15/Mar/13  Updated: 20/Dec/16  Resolved: 29/Apr/13

Status: Resolved
Project: glassfish
Component/s: jax-rs
Affects Version/s: 4.0_dev
Fix Version/s: 4.0_dev

Type: Bug Priority: Critical
Reporter: Tom Mueller Assignee: martin.mares
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Tags: 4_0-approved, devx_web


During a profiling run, the Jersey service startup in GF takes 8763 ms, and 6165 ms of that is in one method:

18.1% - 6,165 ms - 1 inv. org.glassfish.jersey.server.ApplicationHandler.initialize

Is it expected that this method would take quite a bit a time?

None of the methods that are called by that method take nearly that much time:

  • validate takes 1491ms
  • 37 invocations of org.glassfish.jersey.server.model.Resource.from take 1302 ms

Nothing else takes more than 1000 ms.

Note: the times are distorted because of the profiling so just compare the relative times. The 18% is the total CPU time of the process.

Comment by Tom Mueller [ 15/Mar/13 ]

This issue has been created as JERSEY-1795.

This issue will be kept open until there is a resolution in GlassFish for JERSEY-1795.

Comment by Marek Potociar [ 26/Mar/13 ]

Targeting for a fix after 4.0 release.

Comment by Tom Mueller [ 01/Apr/13 ]

Marking the fix for 4.0 until we determine that a fix in this area is not necessary to meeting the developer scenario benchmark requirement for 4.0.

Comment by Hong Zhang [ 03/Apr/13 ]

(Update the issue with the information from email exchanges on loading weld-integration module)

This initialization has triggered the loading of the weld-integration jar (stack trace below). We need to look into whether we can avoid loading CDI related things in the initialization.

Module [44] started org.glassfish.main.web.weld-integration

Inhabitants / stack combination

Complete thread stack Trace

Comment by Tom Mueller [ 09/Apr/13 ]

Reassigning to Martin since he is looking into this.

Comment by martin.mares [ 11/Apr/13 ]


I made several different measurements:
A) visualvm CPU profiling: Resource.from average CPU duration is 1.5 ms and 18 resp. 37 iterations
B) Overal Jersey initialisation on my environment with lazy loading (loads for first command call) takes 850 - 950 ms => 30ms is under standard deviation and is hardly seen able.
C) I add logging into Resource.from method of Jersey. In most cases it takes 0-2ms. Only one exception is main resource it takes 55 ms on my environment and 77 ms on my Oracle virtual development environment.
D) I start official performance test on GF with reduced JAX-RS resources (from 37 to 18) and without. There is no difference.

Result. I can not reproduce issue with Resource.from() and I will stop
work on this point.

Comment by martin.mares [ 11/Apr/13 ]

How much did the validation time go down when the number of resources was reduced?

Validation, more exactly: ApplicationHandler.validate() method duration is aprox. 125ms for original resource set and 105ms for reduced resource set. While whole init time for Jersey oscillates between 900-960ms on my environment.

Comment by martin.mares [ 11/Apr/13 ]

What is the rough break down of the 900 ms the way you see it?

Comment by Tom Mueller [ 11/Apr/13 ]

Martin, please continue to work on evaluating what takes more than 900 ms to initialize. This is huge relative to the rest of the server initialization time. I also suspect that this initialization time is effecting the overall devx_web test run.

Here's a simple test that would seem to demonstrate this. I ran two versions of a script. The first is like the devx_web test:

asadmin start-domain
time asadmin deploy helloworld.war
asadmin undeploy helloworld
asadmin stop-domain

The second is similar, except it adds a sleep 3 after the start-domain:

asadmin start-domain
sleep 3
time asadmin deploy helloworld.war
asadmin undeploy helloworld
asadmin stop-domain

In the second test, the server is given time to finish the Jersey/command framework initialization before the deploy starts. If this initialization is not effecting the benchmark time, then this test should show deploy times that are the same in both versions. Here are the results:

no sleep: 3.34 sec (average of 10 runs)
with sleep: 2.77 sec (average of 10 runs)

This shows that the initial deployment of an application could be over a half a second faster if the server was really ready to process a command after the domain is started.

Comment by martin.mares [ 11/Apr/13 ]

Today I was back in this task. Trying determine what we can do and what initialization is consist of. I found following possibilities - topics for future investigation (overal init is 900ms):

  1. install of base set of Jersey Binders (about 20 binders) takes 240ms. - TODO: Investigate these binders
  2. Validation: 120ms - TODO: Ask Jersey to be possible switch it off. As I wrote before reduction of resources does not help much
  3. Resources introspection: 100ms - TODO: Try to use programmatic API. But I am expecting that it helps just tens of milis.
  4. Stage building 110ms - TODO: Just try to understand this process. But I don't thing that it will be place to reduce init time
  5. Initialisation of component providers 90ms - TODO: Check if we need it and what is it.
Comment by martin.mares [ 12/Apr/13 ]

Currently I have two major ideas:
a) Reflect older idea from Marek and ask Jersey to make validation optional based some configuration variable. Validation takes 120ms on my environment.
b) Currently working on Jersey's plugin to ServiceFinder implementation. It is already plug-able but I want to ask for revision or maybe small API improvement from Jersey team. I don't want to destroy something. If my measurements are OK my plugin can improve performance for aprox 200ms.
Overal initialization takes 900ms these two improvements can change it to 600ms. It looks good - so, cross fingers.

Comment by martin.mares [ 25/Apr/13 ]
  • What is the impact on the customer of the bug?

Solution improves performance of Jersey initialization in metter of time and footprint. Jersey initialization is part of start sequence but running in parallel with other staff.

  • What is the cost/risk of fixing the bug?

This fix has 3 parts:

  1. switch of deployed Jersey application model validation - reduce init duration by 120 ms on my laptop - low risk
  2. reduce number of deployed Jersey providers - reduce init duration by 20 ms and reduce footprint - middle risk
  3. remove non necessary resource auto discovery in Jersey initialization by providing own custom provider - reduce init duration about 100ms - middle risk
  • Is there an impact on documentation or message strings?


  • Which tests should QA (re)run to verify the fix did not destabilize GlassFish?

CLI tests

  • Which is the targeted build of 4.0 for this fix?


  • If this an integration of a new version of a component from another project,
    what are the changes that are being brought in? This might be list of
    Jira issues from that project or a list of revision messages.


Comment by martin.mares [ 29/Apr/13 ]

svn 61705

Generated at Mon Apr 24 15:05:28 UTC 2017 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.