When establishing an XmlOutput writer via the createWriter method and also when creating a CharacterEscapeHandler via the createEscapeHandler method, MarshallerImpl checks the desired encoding against the values "UTF-8" and "UTF" (respectively) in a case sensitive manner (equals instead of equalsIgnoreCase and startsWith instead of encoding.toLowerCase().startsWith("utf") or something similar). This causes the marshaller to behave differently when the JAXB_ENCODING is set to UTF-8 or utf-8. Specifically, when utf-8 encoding is specified on the marshaller a NioEscapeHandler is used which unnecessarily escapes certain UTF-8 characters. The W3C Recommendations for XML 1.0 mention that encoding names should be matched in a case insensitive manner (see http://www.w3.org/TR/2006/REC-xml-20060816/#charencoding).
I've attached a small junit test which exercises the issue.