Specification Lead: Ronald Toegl, IAIK, Graz University of Technology
Copyright © 2011 IAIK, Graz University of Technology.
This TEXT document is citing material from the JSR 299 TCK documentation and therefore licensed under the the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. The author would like to thank the authors of the JSR299 TCK for their excellent documentation and the release under a liberal license.
This guide describes how to download, install, configure, and run the Technology Compatibility Kit (TCK) used to verify the compatibility of an implementation of the JSR-321: Trusted Computing API for Java specification.
The JSR321 TCK is built atop the JUnit Test framework to define test cases and JT Harness as GUI test management tool. It is targeted for Java SE.
The JSR321 TCK is provide under the GNU Public License v2 with Classpath exception.
This guide is for implementors of JSR321 technology to assist in running the test suite that verifies the compatibility of their implementation.
Before reading this guide, you should familiarize yourself with the Trusted Computing Group's (TCG) specifications regarding the Trusted Platform Module (TPM), version 1.2 and the TCG Software Stack, version 1.2.
The JSR321 TCK must be used to ensure that your implementation conforms to the JSR321 specification. This part introduces the TCK, gives some background about its purpose, states the requirements for passing the TCK and outlines the appeals process.
In this part you will learn where to obtain the JSR321 TCK and supporting software. You are then presented with recommendations of how to organize and configure the software so that you are ready to execute the TCK.
Finally, it discusses the reporting provided by the TCK.
A TCK, or Technology Compatibility Kit, is one of the three required pieces for any JSR (the other two being the specification document and the reference implementation). The TCK is a set of tools and tests to verify that an implementation of the technology conforms to the specification. The tests are the primary component, but the tools serve an equally critical role of providing a framework and/or set of SPIs for executing the tests.
A TCK is entirely implementation agnostic. Ideally, it should validate assertions by consulting the specficiation's public API.
The goal of any specification is to eliminate portability problems so long as the program which uses the implementation also conforms to the rules laid out in the specification.
Executing the TCK is a form of compatibility testing. It's important to understand that compatibility testing is distinctly different from product testing. The TCK is not concerned with robustness, performance or ease of use, and therefore cannot vouch for how well an implementation meets these criteria. What a TCK can do is to ensure the exactness of an implementation as it relates to the specification.
Compatibility testing of any feature relies on both a complete specification and a complete reference implementation. The reference implementation demonstrates how each test can be passed and provides additional context to the implementor during development for the corresponding assertion.
Java platform compatibility is important to different groups involved with Java technologies for different reasons:
The JSR321 specification goes to great lengths to ensure that programs written for Java SE are compatible and that the TCK is rigorous about enforcing the rules the specification lays down.
While the JSR321 TCK is rigorous about enforcing an implementation's conformance to the JSR321 specification, it's reasonable to assume that an implementor may discover new and/or better ways to validate the assertions. This chapter covers the appeals process, defined by the Specification Lead, IAIK. Graz University of Technology, which allows implementors of the JSR-321 specification to challenge one or more tests defined by the JSR321 TCK.
The appeals process identifies who can make challenges to the TCK, what challenges to the TCK may be submitted, how these challenges are submitted, how and by whom challenges are addressed and how accepted challenges to the TCK are managed.
Following the increasing transparency in the JCP, implementors are encouraged to make their appeals public, which this process facilitates. The JCP community should recognize that issue reports are a central aspect of any good software and it's only natural to point out shortcomings and strive to make improvements. Despite this good faith, not all implementors will be comfortable with a public appeals process. Instructions about how to make a private appeal are therefore provided.
Any implementor may submit an appeal to challenge one or more tests in the TCK.
Any test case, test case configuration, and other resources may be challenged by an appeal.
What is generally not challengable are the requirements made by the specification. The specification document is controlled by a separate process and challenges to it should be handled through the JSR-321 EG by contacting the specification lead.
To submit a challenge, a mail should be sent to the tck-mailing-list AT jsr321.java.net. The appellant should provide a title, summary, list of suggested changes including Java code or patch-file and a successful test run protocol of the changed TCK which details the hardware and software environment used. Any communication regarding the issue should be pursed on this mailing list, as the archive is publicly readable for accurate record.
Alternatively, an appeal may be sent in private to the spec lead, but should clearly indicate that it is not intended for public discussion. Accepted challenges will be posted to the mailing list in an anonymity-preserving way.
The challenges will be addressed in a timely fashion by the specification lead, or his/her designate. The appellant can also monitor the process by following the mailing list.
Accepted challenges will be acknowledged via the mailing list. If the Spec Lead and appellant are unable to agree on the issue resolution, it will be referred to the JSR-321 expert group for a majority vote.
Periodically, an updated TCK will be released containing tests altered due to challenges. No new tests will be added. Implementations are required to pass the updated TCK.
The JSR321 TCK is designed as a portable, configurable and automated test suite for verifying the compatibility of an implementation of the JSR321 specification. The test suite is provided as suite of JUnit test cases and helper functionalities that allows integration in the JT Harness test framework.
The TCK provides a suite of tests that cover the JSR321 API specification. It consists of the following folders and files:
../jsr321-tck/ | ---> classes/ | | | ---> javax.trustedcomputing.tpm/** | | | ---> (various test cases) ---> java/ | | | ---> javax.trustedcomputing.tpm/** | | | ---> (the java files for the test cases) ---> lib/ | |---> (dependencies of the test tools and their licences) | | ---> jsr321-tck-utils.jar (helper classes for the test suite) testsuite.jtt (TestSuite configuration for JT Harness) example.AbstractTestCase.properties (an example configuration file) example.run.sh (a template start script for Linux) example.run.cmd (a template start script for Windows) iaik_run.sh (a pre-configured start script for IAIK's RI) (not normative) iaik_run.cmd (a pre-configured start script for IAIK's RI) (not normative) README.txt (this file)
The JSR321 TCK has been tested on following platforms:
Before you can test it you may need to prepare and configure the JSR321 implementation under test and your platform's TPM.
Specifically, you must
You also need to collect information on the classpath dependencies of the implementation under test.
Get the TCK archive and extract it to a working folder.
Complete your system settings in the AbstractTestCase.properties file. A template can be found in example.AbstractTestCase.properties. Be sure to set the correct implementation class for TPMContext, the AIK UUID created in the previous steps and same correct owner passphrase you used when taking ownership of the TPM. The test suite expects the JVM property jsr321.tck.abstracttestcaseconfig to point to the completed configuration file.
Configure the paths in the example.run.sh to fit your needs and rename it to run.sh.
You can either start the test suite in text mode with
As an alternative you can also directly access the JUnit testcases in you favorite IDE, such as Eclipse. The process is straightforward (mimic the run script in your project settings) and out of scope of this document.
Informative Comment: We also provide an informative description of how to use the TCK for IAIK's implementation of JSR321 . It is not a normative part of the TCK.
To access the TCK in GUI mode using JTHarness run the start script with
JTHarness is a powerful, but not trival tool, see this article for an overview of JTHarness.
JTHarness will welcome you with a Quick Start Dialog where you should indicate to Start a new test run. Then specify the JSR321 TCK root folder as test suite and Create a new configuration in a work directory of arbitrary name, such as workdir. Continue with the configuration editor (also available from the Configure/Edit Configuration menu).
Here, the the JTHarness requires some configuration via a configuration interview. The most important steps are step one and two. The first step lets you define the AbstractTestCase.properties file hat should be used for the test.. The second step is responsible for adding the required libraries. You definitely have to add the JSR321 API. the implementation under test and all of its dependencies. I.e. here you need to provide your full classpath again. The remainder of the editor allows you to select a set of tests to run and other test parameters such as concurrency. In the end, save your configuration file. You can now execute the tests with the "play" button. When finished set the view filter to "Status of Last Test Run" to see the results. After saving your configuration the tests are ready to run!
For illustration we provide a screenshot of an only partially successful run:
For more information please refer to the JT Harness Project and its Documentation,
Note: Using JTHarness might require the unlimted strengh cryptography policies in your JRE as the more complex class-loading mechanisms seem to confuse some JCE providers.
A challenge when writing software tests is to assess the quality of a test suite. A practical way to answer that question is to analyze what code is executed during a test run, i.e., to measure the code coverage of the test suite. We include the coverage analysis tool EMMA to transparently measure the test line, branch and method coverage while executing the test suite.
API Signature Coverage statement: The JSR321 TCK covers 100% of all API public methods using EMMA Coverage analysis tool. Here, we provide the Full Coverage Report.
In the past, several TPM implementations have been shown to contain implementation bugs or overly creative interpretations of the specifications. This means that a software implementation of JSR321 might be 100% specification compliant but might at the same time fail a TCK test run on such a buggy hardware TPM. Note that due to the cryptographic nature of TPM I/O even minor derivations will lead to non-software-recoverable errors.
We therefore maintain a list of JSR321 TCK test run protocols where vendor, version and firmware revision are collected for reference. It is acceptable to only validate a JSR321 implementations with compatible TPMs.