Jakarta Batch TCK Reference Guide

The Jakarta Batch TCK is provided under the Eclipse Foundation Technology Compatibility Kit License - v 1.0 [https://www.eclipse.org/legal/tck.php].

This guide will assist in running the test suite verifies implementation compatibility for BOTH:

Building on the previous point, it is convenient to use, as shorthand, the term "EE mode" when talking about the TCK constructs and requirements specifically for users running the TCK to certify against the entire EE platform. It is a convenient shorthand term too, then, to use the term "SE mode" for users that are only trying to certify against the Jakarta Batch specification, though this term in some ways might be misleading. Since the tests and the execution requirements are different for each set of users, it is helpful to use this shorthand, and we do at times in this document.

Before reading this guide, you should familiarize yourself with the Jakarta Batch Version 2.1 specification, which can be found at https://jakarta.ee/specifications/batch/2.1/.

Other useful information and links can be found on the eclipse.org project home page for the Jakarta Batch project [https://projects.eclipse.org/projects/ee4j.batch] and also at the GitHub repository home for the specification project [https://github.com/eclipse-ee4j/batch-api].

We sometimes refer to this Batch TCK as the "standalone" TCK. This usage comes from the fact that Jakarta Batch is part of the Jakarta EE Platform, which has a platform-level TCK, which we’re distinguishing this "standalone" TCK from.

Whether you are using this guide and the TCK to certify compatibility of a batch implementation against the Jakarta Batch specification alone, or certifying compatibility with the entire Jakarta EE Platform, you will in both cases need to run the TCK against your implementation and pass 100% of both the:

The two types of tests are not encapsulated in a single execution or configuration; typically they must be executed via multiple executions, as explained in detail later in the guide.

By "runtime" tests we simply mean tests simulating Jakarta Batch applications running against the batch implementation attempting to certify compatibility. These tests verify that the batch applications behave according to the details defined in the specification, as validated by the TCK test logic.

The set of runtime tests required to use the TCK to certify the Batch portion of Jakarta EE Platform compatibility ("EE mode") is a superset of the set of runtime tests required to satisfy compatibility only with the Jakarta Batch specification ("SE mode").

The JDK used during test execution must be noted and listed as an important component of the certification request. In particular, the Java SE version is important to note, and this version must be used consistently throughout both the JUnit 5 runtime and Signature tests for a given certification request.

For the current TCK version, this can be done with either Java SE Version 11 or Version 17.

The Jakarta Batch TCK is distributed as a zip file, which contains the TCK artifacts (the test suite binary and source, porting package SPI binary and source, the test suite XML definitions, and signature files) in /artifacts, the documentation in /doc, and some example Maven modules showing how to run the TCK in /runners. You can access the current source code from the Git repository: https://github.com/eclipse-ee4j/batch-tck.

The software can simply be extracted from the ZIP file. Once the TCK is extracted, you’ll see the following structure:

In more detail:

artifacts contains all the test artifacts pertaining to the TCK: The TCK test classes and source, the TCK SPI classes and source, the TestNG suite.xml file and the signature test files.

doc contains the documentation for the TCK: this reference guide, plus a script that helps provide an example of how to run the TCK against the 'jbatch' implementation.

runners contains three Maven modules that provide samples for executing different portions of the TCK:

For the runtime test (JUnit) component of the TCK:

All signature tests must be passed. The sigtest tool must be pointed to the TCK-provided signature file with an appropriate configuration and get a clean result without error or failures.

One unusual aspect of a POM like this is that, one the one hand, it has a non-SNAPSHOT version and is released to Maven Central. On the other hand, it is delivered in the TCK ZIP in a way that the user may simply choose to edit it and run the TCK against their implementation, without changing the Maven GAV coordinates to something they own. We don’t consider this a problem but beware, e.g. if you do mvn install you’ll be overwriting this locally. (This shouldn’t affect the TCK execution since nothing else uses any runner as a dependency or parent).

Note: the TCK runners mentioned here are configured to run "against" artifacts referenced as Maven dependencies, and so obtained from a Maven repository (e.g Maven Central). They are configured this way even though as explained in the guide to the TCK ZIP contents, the artifacts are all present in the TCK zip.

This is a significant difference with the previous Jakarta Batch Ant-based TCK.

Starting with the Jakarta Batch 2.1 TCK release, the runner configurations are NOT set up referencing relative paths within the TCK zip. The artifacts packaged in the ZIP (detailed below) only help create a self-contained TCK package, providing an easy view of what artifacts are relevant to the TCK in a single place without requiring the user to follow Maven dependency references.

While it would be possible to construct a valid TCK execution configuration running against the artifacts in the TCK ZIP, an example is not provided.

The runners each inherit from a module within the Batch TCK, in order to leave the minimum amount of configuration needed to be performed by the implementer. Use standard Maven techniques, e.g. mvn help:effective-pom to see the configuration of the module being executed merged with its parent. NOTE: Inheriting from these parents is optional, strictly speaking, but it is a good way to ensure a valid configuration for certifying against the TCK is being used.

First, we look at the 'core' execution, through which the large majority of the runtime tests are executed. It is defined like this in the runner POM:

and which includes four aspects, which we use to illustrate required vs. optional aspects of TCK execution.

Next we look at the 'appbean' execution, through which only a small number of tests are executed. This configuration differs from that of the 'core' execution in that an application JobOperator bean producer is added to the test classpath. It is defined like this in the runner POM:

and we step through the same four details:

Note there is no requirement to configure these two executions from a single POM, as the runner does. E.g. two separate POMs could be used. Hopefully the details above provide clear enough guidance for what is and is not required for each execution. As mentioned above it is not required that the user’s "runner" module inherit from the parent like this example one does.

(Here we abstract out the exact numbers to avoid forgetting to update this count and causing ambiguity with the required test count detailed elsewhere).

The 'core' execution:

The 'appbean' execution:

Note: there are many forced failure scenarios tested by the TCK, so typically the log will show a lot of exception stack traces during a normal, successful execution.

If you experienced a failure, it is possible that you experienced a timing issue. The TCK has several built-in properties allowing for tuning of execution to deal with these, and instructions elsewhere in this guide for doing so.

Without additional configuration the runner POM is not actually configured to run any tests, because the specific runtime with its specific Arquillian adapter is not configured by default.

Instead the runtime-specific pieces are configured within a set of optional profiles, e.g. -Pglassfish-remote or -Pliberty-managed, to run against GlassFish or Open Liberty, respectively.

To run against a specific runtime, then, you would need to combine something like this runtime-specific profile with the common configuration defined in the runner module (independent of any profile).

The runner provides the common setup and configuration needed for any runtime-specific execution via a parent module that can referenced via:

While it is not required, strictly speaking, to inherit from this parent, it is a convenient way to add all the required elements of the TCK execution while still providing a space for customization to adapt to your specific runtime.

NOTE: Because the parent module is referenced by other modules we don’t include the parent module in the TCK distribution, to avoid confusion. An easy, standard Maven technique for merging a given POM with its parent (and ancestors, more generally) is to run the goal: mvn help:effective-pom from the current module directory at the command line.

A requirement for certifying in EE mode is to execute each test in each of a servlet ('web') and a Jakarta Enterprise Bean context. So to run from a "Jakarta Enterprise Bean context" is to have the test perform the JobOperator API calls used in each TCK test method from within a Jakarta Enterprise Bean (provided by the TCK). This mechanism is established through the Arquillian adapter plus our Arquillian extension module, configured with the help of the parent POM.

We also need to run a set of tests (the 'appbean' tests) with a different application classpath than the 'core' majority of the tests.

Therefore we set up our parent POM to establish four executions of the 'failsafe' plugin: for each of the two classpath configurations ('core' and 'appbean') we have one execution for each of the Jakarta Enterprise Beans and servlet contexts.

First, we look at the 'core' executions, through which the large majority of the runtime tests are executed. If we look specifically at the 'core-ejb' execution, is defined like this in the parent POM:

and it includes five aspects, which we use to illustrate required vs. optional aspects of TCK execution.

Next we look at the 'appbean' execution, through which only a small number of tests are executed. This configuration differs from that of the 'core-ejb' execution in that an application JobOperator bean producer is added to the test classpath. For the EE/platform suite, this is accomplished via a custom ShrinkWrap routine defined in our Arquillian extension module.

The requirements for each of the 'core-web', 'appbean-web' executions are identical to those of the 'core-ejb', 'appbean-ejb' executions, respectively, with a single change:

So each of these two "web" executions must be run, and all tests contained must pass.

Note there is no requirement to configure these four executions from a single POM, as the runner does. E.g. one conceivable approach would be to use four separate POMs even, if that were preferred for some reason. Hopefully the details above provide clear enough guidance for what is and is not required for each execution, and if not an issue should be raised with the Jakarta Batch TCK project. As mentioned above it is not required that the user’s "runner" module inherit from the parent like this example one does.

Here is example output when we, as in the sample runner, configure the 'xml' and 'echo' Maven plugins to collect and display the test results.

(Here we abstract out the exact numbers to avoid forgetting to update this count and causing ambiguity with the required test count detailed elsewhere).

Note: there are many forced failure scenarios tested by the TCK, so typically the log will show a lot of exception stack traces during a normal, successful execution.

If you experienced a failure, it is possible that you experienced a timing issue. The TCK has several built-in properties allowing for tuning of execution to deal with these, and instructions elsewhere in this guide for doing so.

The 'sigtest' runner shows an execution of the signature tests against the 'jbatch' implementation, while pulling in its dependencies:

The TCK provides a distinct signature file depending on whether a Java 11 or a Java 17 JDK is used to certify.

The runner provides a profile-based POM config which is automatically activated based on the Java SE level of the JDK used to run Maven.

In calling the setup "OPTIONAL" we are referring to the fact that an implementer does not need to use any of these exact goals or configuration, though it is probably necessary that some analogous setup work will need to be done before executing the signature tests.

In our example runner, the 'maven-dependency-plugin' is used in the 'pre-integration-test' phase to setup the later sigtest execution.

The first execution unpacks (copies) the signature files themselves into place in the target/sigtest-copy location:

The second execution unpacks the jbatch impl, including its API dependencies (it uses the default output directory of target/dependency):

The 'sigtest-maven-plugin' can also be configured to pick up dependencies as Maven dependencies, building the "test classpath" via Maven. However, we chose to show a more explicit approach, "flattening" the dependency tree by copying everything into a single directory we will execute the tests against

NOTE: Also a '0' exit status should be returned from the mvn verify command line (shown by echo $?).

NOTE: Included in the output above is an expected warning about the "incorrect classpath parameter" target/classes. This occurs because our execution project only tests content pulled in as dependencies. It is unfortunate the plugin is this noisy here, and this can be ignored.

NOTE: The Base version: in the above output comes from the signature file under test, so should appear exactly as shown. The Test version:, on the other hand, comes from the version of the module executing the tests, and so a user not using the runner module could see their own value here.

Though it is not required it can be useful to validate the setup by forcing a failure in the signature test.

E.g. one approach would be to to swap the profile activation definition in the runner POM so that the the 'jdk11' profile gets activated in the presence of Java 17 (instead of Java 11), and vice versa, so that when running with each of Java 11 or 17 the opposite, incorrect signature file will be used.

E.g.:

This should produce output like:

and a non-zero exit status should be returned by mvn verify (shown by echo $?).

Though it is not required for running the TCK, it could perhaps be useful debugging to know how to generate the signature files.

Of course, this is also necessary for producing the TCK in the first place, and updating it after making new API changes or supporting new Java versions.

Simply run:

Since the main plugin configuration as well as the profile-based configuration (activated by current JDK level) is added at the plugin-level (rather than the execution-level), no additional configuration is required. The plugin will generate to the same output locations used during the check goal, e.g. target/sigtest-copy/sigtest/sigtest-1.6-batch.standalone.tck.sig-2.1-se11-OpenJDK-J9 (possibly overwriting whatever was there).

The basic test flow simply involves a JUnit test method using the JobOperator API to start (and possibly restart) one or more job instances of jobs defined via one of the test JSLs, making use of some number of com.ibm.jbatch.tck.artifacts Java artifacts. The JobOperator is wrapped by a thin layer which blocks waiting for the job to finish executing (more on this in the discussion of the porting package SPI later in the document).

Several tests intentionally produce failures to test relevant portions of the specification, so a normal execution may cause a number of stack traces, error messages, etc. to stdout.

The Jakarta Batch TCK relies on an implementation of a "porting package" SPI to function, in order to verify test execution results. The reason is that the Jakarta Batch specification API alone does not provide a convenient-enough mechanism to check results.

A default, "polling" implementation of this SPI is shipped within the TCK itself. The expectation is that the typical Jakarta Batch implementation will be content to use the TCK-provided, default implementation of the porting package SPI.

Further detail on the porting package is provided later in this document, in case you wish to provide your own, different implementation.

The JobOperatorBridge is a utility/helper class in the Jakarta Batch TCK which makes use of the following system property:

using a default value of 900000 (900 seconds).

This prevents tests from "hanging" indefinitely if something catastrophic occurs causing the job to never complete (or if the porting package SPI "waiter" is never notified for some reason).

Note that some of the tests (e.g. the chunk tests involving time-based checkpointing) will take at least 15-25 seconds to run on any hardware, so any default value less than that applied to all tests would cause failures simply due to timing (and not because of any failure in the underlying Jakarta Batch implementation).

The value of 900 seconds was chosen, then, to avoid falsely reporting an error because of timing out too soon, allowing plenty of time for a test to finish executing, even on slower hardware, and leaves some time to attach a debugger.

It does not, however, provide "fast failure" in case of a hang or runaway thread.

In any case, this timeout value can be customized (say, to increase when debugging or decrease to force a faster failure in some cases).

Some of the TCK tests sleep for a short period of time to allow an operation to complete or to force a timeout.

These wait times are defaulted via properties that are also specified in the TCK source repo at path: com.ibm.jbatch.tck/src/main/resources/tck.default.sleep.time.properties. (Note this exact file may not be used in the sample 'runner' modules provided with the TCK).

As with many typical decisions regarding timeout values, we attempt to strike a good balance between failing quickly when appropriate but allowing legitimate work to complete.

These values can be adjusted if timing issues are seen in the implementation being tested. Refer to the comments in the test source for a specific test to better understand how the time value is used for that test.

For most development/debug use cases it is recommended to refer to the source in the Jakarta Batch TCK] GitHub repository [https://github.com/eclipse-ee2j/batch-tck], using this documentation, and GitHub tags/releases, etc. to match the official level tested in the TCK distribution.

Note too that for an implementation to pass the TCK, it must run against the shipped TCK test suite binary as-is (and not against a modified TCK).

The EE Platform version of the TCK uses Arquillian to run the JUnit 5 tests. It works by:

The Batch TCK Arquillian extension is implemented in its own module within the Batch TCK. It contains :

One specification detail in particular significantly complicates the TCK. To validate the rules described in Section 10.4 JobOperator and subsections, the TCK needs to test both the cases in which the application does and does not provide a JobOperator CDI Bean. This requires multiple "classpath" variations (with the TCK itself playing the role of "application" here). Because of this we require multiple test executions to be configured in Maven, and the full JUnit portion of the TCK will consist of both of these executions, both of which must be executed successfully in order to pass the TCK and claim compliance.

The challenge process is defined within the Challenges section within the Jakarta EE TCK Process 1.1.

Challenges will be tracked via the issues of the Jakarta Batch Specification repository.

As a shortcut through the challenge process mentioned in the Jakarta EE TCK Process 1.1 you can click here, though it is recommended that you read through the challenge process to understand it in detail.

The certification of compatibility process is defined within the Certification of Compatibility section within the Jakarta EE TCK Process 1.1.

Certifications will be tracked via the issues of the Jakarta Batch Specification repository.

As a shortcut through the certification of compatibility process mentioned in the Jakarta EE TCK Process 1.1 you can click here, though it is recommended that you read through the certification process to understand it in detail.