Bug 4827

Summary: SPEC: Misspoke on collector role on exit status
Product: jbatch Reporter: ScottKurz
Component: SPECAssignee: cvignola
Status: RESOLVED FIXED    
Severity: minor CC: issues, mminella
Priority: P5    
Version: 1   
Target Milestone: ---   
Hardware: PC   
OS: Windows   
Whiteboard:

Description ScottKurz 2013-03-22 14:59:12 UTC
In Sec. 8.7.2 Exit Status for Partitioned Steps
we have:

For a partitioned batchlet, each thread processing a partition may return a separate exit status. These exit status values are ignored unless a partition collector is used to coalesce these separate exit status values into a final exit status value for the step.

---

The collector doesn't play a role here..it's the analyzer.

Should be:
These exit status values are ignored unless a partition analyzer...
Comment 1 ScottKurz 2014-01-07 18:46:28 UTC
Small rewrite of entire paragraph to incorporate this point as well as bug 5533 (in which the effects of the transition element on step exit status were misstated).

Change from:

<BEFORE>

8.7.2 Exit Status for Partitioned Steps

The exit status for a partitioned step follows the same rules as for a regular step except for exit status set by batch artifacts processing individual partitions. This means the step's exit status is set by  
  1. transition elements stop, end, fail
  2. StepContext
the same as for a non-partitioned step. If the exit status is not set, it defaults to batch status, the same as for a non-partitioned step.

For a partitioned batchlet, each thread processing a partition may return a separate exit status. These exit status values are ignored unless a partition collector is used to coalesce these separate exit status values into a final exit status value for the step.

The batch runtime maintains a StepContext clone per partition. For a partitioned batchlet or chunk, any batch artifact running on any of the threads processing a partition set a separate exit status through the StepContext clone. These exit status values are ignored unless a partition collector is used to coalesce these separate exit status values into a final exit status value for the step.

</BEFORE>

to:

<AFTER>
8.7.2	Exit Status for Partitioned Steps

The exit status for a partitioned step follows the same rules as for a regular step except for an exit status set by batch artifacts processing individual partitions. This means any batch artifact running on the main thread of the partitioned step can set the step’s exit status via the exit status setter method on the StepContext object , the same as for a non-partitioned step.   E.g. a step’s partition analyzer, partition reducer, or step listener could each potentially  set the step’s exit status in this simple manner (since each of these artifacts run on the initial thread, not the threads processing an individual partition).  If the exit status is not set, it defaults to batch status, the same as for a non-partitioned step.

For a partitioned batchlet, each thread processing a partition may return a separate exit status.  However, these exit status values are ignored unless a partition analyzer is used to coalesce these separate exit status values into a final exit status value for the step.

The batch runtime maintains a StepContext clone per partition.  For a partitioned batchlet or chunk, any batch artifact running on any of the threads processing a partition would merely set a separate exit status through the StepContext clone.  These exit status values are ignored unless a partition analyzer is used to coalesce these separate exit status values into a final exit status value for the step.

</AFTER>
Comment 2 mminella 2014-01-07 19:04:20 UTC
Scott,  

I may be wrong here but I don't think the batchlet paragraph is correct in the new version.  It's my understanding that the ExitStatus from a partitioned Batchlet is ignored.  Period.  

The PartitionAnalyzer can coalesce any output by the PartitionCollector, but that is a separate artifact that doesn't have any direct interaction with the Batchlet by default.

Am I missing something here?
Comment 3 ScottKurz 2014-01-07 19:50:53 UTC
(In reply to mminella from comment #2)
> Scott,  
> 
> I may be wrong here but I don't think the batchlet paragraph is correct in
> the new version.  It's my understanding that the ExitStatus from a
> partitioned Batchlet is ignored.  Period.  
> 
> ...

Michael,

In mentioning the potential role of the analyzer, we're recognizing that the PartitionAnalyzer's method:

  public void analyzeStatus(BatchStatus batchStatus, String exitStatus) 

could be used to "coalesce" per-partition-only exit statuses into a step-level, overall, exit status.  

That's the case for both partitioned batchlet and chunk steps.  The fact that the batchlet return value can be used to pass a partition-level exit status via analyzeStatus simply extends the full set of exit status options available to batchlets to partitioned batchlets.

We do exercise this requirement in the TCK, in test:
  ParallelExecutionTests#testInvokeJobWithOnePartitionedStepExitStatus
Comment 4 mminella 2014-01-07 20:02:40 UTC
Sorry.  I stand corrected.