I have encountered a problem when sending compressed JMS ObjectMessages from a client to Glassfish. In the GLassfish log file we see a "java.io.StreamCorruptedException: invalid stream header: 789C5BF3" exception.
I did not have this problem when changing the MQ broker to LOCAL/REMOTE in Glassfish.
Further analysis shows that the client and the LOCAL/REMOTE broker sends there jms messages as com.sun.messaging.jmq.jmsclient.ObjectMessageImpl. The embedded MQ broker uses com.sun.messaging.jms.ra.DirectObjectPacket. The message classes are the same for compressed and uncompressed. However when using compressed messages the embedded broker could not deserialize it.
We also tested this with Glassfish v3.1.2. When we use a TextMessage instead of ObjectMessage, the message can be processed but the Text is corrupted.
The problem can be reproduced with the attached maven project. You need to create a JMS queue with name "TestQueue" and deploy the ejb jar file. The UnitTest can be used to send a compressed and uncompressed message to Glassfish. In the server.log you will get the StreamCorruptedException for the compressed message.