Thanks for replying; more goodies inline.
On 11/28/2012 07:29 PM, michael keith wrote:
I appreciate your comments, David.
On 28/11/2012 4:04 PM, David M. Lloyd wrote:
Comments inline. Also please note that any time I use the word
"module" I am referring to modules as defined in the Java EE
specification, not any proposed SE modularity solution, unless I
On 11/28/2012 01:56 PM, michael keith wrote:
As you know, modularity was recently deferred from SE 8 (and
transitively from EE 8), but since there will now be a longer delay
before core modularity can be leveraged, a few people were wondering if
the spec could provide some specific solutions to some of the more
dominant problems that developers are facing now. The most common one
that modularity would have solved is the "conflicting library problem".
This is when there exists a version of a library that is bundled with
the application but conflicts with a different version of the library
used by the server and that is exposed on the server classpath.
One way to solve this would be to provide a way for an application to
indicate that a bundled library is to be placed on the classpath
*logically ahead* of all server libraries. Most app servers offer some
proprietary solution to this problem, but there is currently no portable
way of doing it. If folks are agreeable then we would like to
standardize a solution in EE 7. We would first like to get your input as
to whether it is worth doing this, though.
Speaking from experience I think it's very safe to say that ordering
is not a complete solution by any means, especially because in a
simple class path system, classes can "leak through" very easily, and
it only solves the most superficial part of the problem ("I can't see
the classes I need") in the most superficial way possible ("just layer
this one on top of that one and hope for the best"). Any solution
that can regularly result in substantially different behavior for
applications based upon import order of dependencies is suspect, and
is certain to cause the user a great deal of trouble.
Yes, those issues can arise. To reiterate, we are not trying to come up
with a complete solution, just asking whether people thought it was
worth coming up with a specific solution to a fairly specific problem.
Is it your position, then, that a feature of this nature would not be
worth offering, or that the value it offered would be offset by the
problems that it would introduce?
I think that the value is not worth the problems. Especially when vendors (such as, well, us) have other solutions to the same problem already. And because I basically agree with Markus Eisele in this point: in general, applications *shouldn't* see third-party libraries in the first place unless they specifically want to, in which case it's not a problem anyway.
In conjunction with the above question there is an additional question
relating to the limitation that we would need to impose in this initial
solution. Given that a) the majority of problem occurrences are due to
3rd party libraries being exposed by the server, and b) overriding a
Java EE API library is going to introduce a larger problem space than
what we would likely be able to solve in the time remaining in Java EE,
do you think it is worth providing a soution in EE 7 if that solution
was limited to 3rd party libraries and did not include support for
overriding Java EE APIs? Or, would you prefer to leave it until the next
release and include support for both, or even wait until EE 9 when
modularity will allow a more encompassing solution?
I think that under no circumstances should the packages in the Java SE
or EE APIs be overridable by user applications or modules.
One can already override certain packages in Java SE in an SE
environment using the "endorsed" mechanism. There are some similar use
cases for upgrading certain Java EE APIs in a server as well, but given
your answer it sounds like you would not have a problem if the ability
to override these was not included immediately (or ever, from the sound
Yeah the problem is that unfortunately not all SE or EE APIs are purely implementation-free, so simply upgrading them is often not meaningful (though superficially attractive in some cases).
If enough people would like to solve the limited scope probem in EE 7
then we will follow up with a proposal and some additional questions.
I think that it would not be hard to offer a more complete solution
without introducing or conflicting with a JDK modularity concept.
To me the range of the problem is as follows:
1) Applications, in some containers, may be able to "see" libraries
they don't want to "see" for various reasons (perhaps it provides a
SPI implementation that is undesirable, or it conflicts with a library
in the application itself)
2) Application and module authors often desire the ability to link to
other top-level applications or modules via Class-Path or similar
3) Application authors often require the ability to establish
non-transitive dependencies between applications, modules or JARs therein
4) The Java EE specification does not clearly or satisfactorily
specify the semantics of Class-Path (it could imply that classes are
copied between class loaders, or it could imply that class loaders
link to one another), though it does clearly state that in most (maybe
all) cases, Class-Path always imports the transitive closure of the
Class-Paths of the referenced URI
I think that the first steps to cleaning up this mess are as follows:
1) Update the Java EE platform specification to clearly define
Class-Path as establishing a link between existing class loaders
(possibly allowing for vendor-specific extensions which might perform
old-style "class copying" for special cases)
2) Add a provision to the specification to allow for a non-transitive
variation of Class-Path
I know where you are coming from, and I agree it would be nice and clean
if that were the case. However, the spec has purposely defined class
loading in terms of what should be available where, rather than being
prescriptive about class loaders, which are really an implementation
artifact. The spec does not even prescribe which class loaders actually
exist (apart from the context class loader), mostly to allow vendors the
flexibility to be able to implement it the way they so choose. I can't
see this ever changing (for a bunch of reasons, not the least of which
is that it would likely break backwards compatibility for at least some
vendor products), and most certainly not in this release.
I understand the desire to keep the spec flexible. But I think in this case, more flexibility for us vendors results in less flexibility for the end user. I don't really *want* flexibility, really; I want my specifications to be specific. :)
But, that conceptual stuff aside, I see the compatibility argument and I acknowledge its validity, though I think that the same problems will still exist when modularity rolls around. If we can find a way to provide more clear and strict definitions for the way linkage works in terms of class loaders now, with some provision for vendor-specific behavior, then it will only make modularity simpler when the time comes.
3) Add a provision to the specification to allow the application or
module to choose what non-mandatory APIs and server libraries are
requested, providing for standard (Java SE and EE) and vendor-specific
library namespaces (a Java SE modularity solution *could* be used to
effect this, but does not *have* to be)
Yeah, this is right down modularity lane. The intent was not to try and
solve all of the problems that modularity is going to solve, just get
people past one or two of the pain points until modularity is available.
I agree solidly with what you you are describing, I just don't think we
are in a position to try to tackle it until we have a proper module
system to work with.
I don't know; we managed to do it pretty easily and fairly completely (there are perhaps one or two lessons to be carried forward from that experience), and I don't think the solution we have chosen (which is more or less what I described above) is going to be substantially incompatible with SE modularity (even if it ends up being strongly Jigsaw-flavored). But even if only #1/#2 are accomplished then I think that will represent a significant improvement in the current situation - though at that point we have drifted pretty far away from the original problem statement I think!
4) Add a provision to the specification to allow for a vendor-specific
mechanism by which the user may add additional libraries to this set
5) Ensure that the specification requires that any imports of such
libraries are generally non-transitive with respect to their
dependencies (unless specific transitive dependencies are required;
The modularity guy in you is really shining through, here :-)
I know the EE specification has, until now, been pretty careful about
avoiding specifying the behavior of class loaders. But, if this
problem is to be truly solved (either in conjunction with SE
modularity or simply in a way that would allow for future SE
modularity), this problem has to be addressed head-on, and I think
that these simple steps would make for a much more solid starting point.
At least the last three of these were being planned as part of an EE
modularity strategy... until modularity was dropped from SE 8 and it
became clear that we weren't going to be able to add modularity into EE
until EE 9 (at which point it became somewhat less urgent).
|David M. Lloyd||11/28/2012|
|David M. Lloyd||11/29/2012|
[javaee-spec users] Re: bundle overriding