EJBContext#isCallerInRole(String) is documented to be allowed to throw an IllegalStateException when the caller is not in a state to be allowed to call the method (i.e. if the method is called outside of a security context).
Glassfish's implementation uses this vague wording to mean that isCallerInRole() should also throw an IllegalStateException when it is handed a role name that Glassfish doesn't know anything about.
I've filed this as an enhancement since I don't really know whether strictly speaking this is a specification violation or not. But to my eyes if I call someEjbContext.isCallerInRole("fred") I should get true or false, without the danger of an IllegalStateException (provided of course I'm actually calling this method from within an EJB business method).