org.eclipse.smila.jobmanager.persistence.zk

Class RunStorageZk

java.lang.Object

org.eclipse.smila.jobmanager.persistence.zk.RunStorageZk

All Implemented Interfaces:

RunStorage

public class RunStorageZk
extends java.lang.Object
implements RunStorage

Component for handling jobmanager run data by using zookeeper. Zookeeper structure:

 /smila/jobmanager/jobs/<job-name>/workflow-runs/<workflow-run-id>/data/tasks/<task-id>
 ............................................................................/transient-bulks/<bulkstore+objectId>
 ......................................................................./finishing
 
 ................................./data/<data-node>              // job counters etc.
 ................................./worker-data/<workername>      // worker specific counters etc.
 ................................./jobrun-definitions/              // definitions used in a job run
 ..................................................../jobdef    
 ................................................... /wfdef     
 ..................................................../bucketdef/<bucket-name>    
 ....................../buckets/<bucket-id>/<job-name>        // jobs triggered by bucket
 

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	JOBMANAGER_PREFIX prefix for jobmanager data.
static java.lang.String	NODE_BUCKETS data of a job run.
static java.lang.String	NODE_DATA global data of a job run.
static java.lang.String	NODE_JOBS data of a job run.
static java.lang.String	NODE_RUN_DEFINITIONS root node for stored definitions that are used by a job run.
static java.lang.String	NODE_RUN_DEFINITIONS_BUCKET stores bucket definitions used by a job run.
static java.lang.String	NODE_RUN_DEFINITIONS_JOB stores job definition used by a job run.
static java.lang.String	NODE_RUN_DEFINITIONS_WORKFLOW stores workflow definition used by a job run.
static java.lang.String	NODE_TASKS tasks of a workflow step.
static java.lang.String	NODE_TRANSIENT_BULKS transient bulks of a workflow run.
static java.lang.String	NODE_WORKERDATA worker specific data of a job run.
static java.lang.String	NODE_WORKFLOW_RUN_FINISHING indicates whether a workflow run is currently finished.
static java.lang.String	NODE_WORKFLOW_RUNS workflow runs of a job run.

Constructor Summary

Constructors

Constructor and Description
RunStorageZk()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	activate(ComponentContext context) OSGi Declarative Services service activation method.
void	addJobTrigger(java.lang.String bucketId, java.lang.String jobName)
void	addTransientBulk(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String bulk) Stores a transient bulk for the given workflow run.
java.util.List<java.lang.String>	cancelJobRun(java.lang.String jobName, java.lang.String jobRunId) Cancels the given job run data -> active workflow runs and current tasks will be canceled.
boolean	checkAndCleanupActiveWorkflowRuns(java.lang.String jobName, java.lang.String jobRunId) Checks if there are active workflow runs.
void	clear() Remove all jobmanager run data.
protected void	deactivate(ComponentContext context) OSGi Declarative Services service deactivation method.
void	deleteJobRun(java.lang.String jobName, java.lang.String jobRunId) Removes the job run data of the given job run.
void	deleteWorkflowRun(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId) Deletes the given workflow run.
void	deleteWorkflowRunData(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId) Deletes the data of the given workflow run.
void	failedTask(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId, boolean failedAfterRetry, java.util.Map<java.lang.String,java.lang.String> properties) Called when a task is failed.
void	failedWorkflowRun(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId) Called when a workflow run failed.
boolean	finishJobRun(java.lang.String jobName, java.lang.String jobRunId) Called when a job run is finished.
void	finishTask(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId, java.util.Map<java.lang.String,java.lang.Number> workerCounter, java.util.Map<java.lang.String,java.lang.String> properties) Called when a task is successfully finished.
java.util.Collection<java.lang.String>	getCurrentJobs()
java.lang.String	getCurrentTimestamp()
AnyMap	getJobRunData(java.lang.String jobName, boolean returnDetails)
java.lang.String	getJobRunId(java.lang.String jobName)
JobRunInfo	getJobRunInfo(java.lang.String jobName)
JobState	getJobState(java.lang.String jobName)
JobState	getJobState(java.lang.String jobName, java.lang.String jobRunId)
int	getTaskRetries(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId) get the number of retries for the task.
java.util.Collection<java.lang.String>	getTransientBulks(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId)
java.util.Collection<java.lang.String>	getTriggeredJobs(java.lang.String bucketId)
AnyMap	getWorkerData(java.lang.String jobName, java.lang.String worker)
AnyMap	getWorkflowRunData(java.lang.String jobName, java.lang.String workflowRunId) get information about a workflow run.
boolean	hasTask(java.lang.String jobName, java.lang.String workflowRunId, java.lang.String taskId) Check if the given task with the given task id exists for the job and workflowrun.
boolean	hasTasks(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId)
boolean	hasWorkflowRun(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId)
void	obsoleteTask(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId, java.util.Map<java.lang.String,java.lang.String> properties) Called when a task finished as obsolete.
boolean	prepareToFinishWorkflowRun(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId) Called before a workflow run is finished and deleted.
void	registerJobRunListener(JobRunListener jobRunListener, java.lang.String jobName)
void	removeJobTrigger(java.lang.String bucketId, java.lang.String jobName)
void	retriedTask(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId, boolean retryByWorker, java.util.Map<java.lang.String,java.lang.String> properties) Called when a task is retried.
void	setJobState(java.lang.String jobName, java.lang.String jobRunId, JobState jobState) Sets the state for the given job run.
boolean	setJobState(java.lang.String jobName, java.lang.String jobRunId, JobState expectedState, JobState newState) Sets the state for the given job run if the given expected state matches the current state.
void	setZooKeeperService(ZooKeeperService zkService) method for DS to set a service reference.
java.lang.String	startCompletionWorkflowRun(java.lang.String jobName, java.lang.String jobRunId) Called when a new completing workflow run is started for the given job run.
void	startJobRun(java.lang.String jobName, java.lang.String jobRunId, JobRunMode jobRunMode, JobRunDefinitions jobRunDefs) Starts a job run.
void	startTask(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId) Called when a new task is started for the given step in the given workflow of the given job run.
void	startTask(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.lang.String stepId, java.lang.String taskId, int numberOfRetries) Called when a new task is started for the given step in the given workflow of the given job run.
void	startTasks(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId, java.util.Collection<Task> tasks) Start multiple new tasks for a given job/workflow run at once.
java.lang.String	startWorkflowRun(java.lang.String jobName, java.lang.String jobRunId) Called when a new workflow run is started for the given job run.
void	successfulWorkflowRun(java.lang.String jobName, java.lang.String jobRunId, java.lang.String workflowRunId) Called when a workflow run successfully finished.
void	unsetZooKeeperService(ZooKeeperService zkService) method for DS to unset a service reference.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

JOBMANAGER_PREFIX

public static final java.lang.String JOBMANAGER_PREFIX

prefix for jobmanager data.

See Also:

Constant Field Values

NODE_WORKFLOW_RUNS

public static final java.lang.String NODE_WORKFLOW_RUNS

workflow runs of a job run.

See Also:

Constant Field Values

NODE_BUCKETS

public static final java.lang.String NODE_BUCKETS

data of a job run.

See Also:

Constant Field Values

NODE_JOBS

public static final java.lang.String NODE_JOBS

data of a job run.

See Also:

Constant Field Values

NODE_DATA

public static final java.lang.String NODE_DATA

global data of a job run.

See Also:

Constant Field Values

NODE_WORKERDATA

public static final java.lang.String NODE_WORKERDATA

worker specific data of a job run.

See Also:

Constant Field Values

NODE_TASKS

public static final java.lang.String NODE_TASKS

tasks of a workflow step.

See Also:

Constant Field Values

NODE_TRANSIENT_BULKS

public static final java.lang.String NODE_TRANSIENT_BULKS

transient bulks of a workflow run.

See Also:

Constant Field Values

NODE_RUN_DEFINITIONS

public static final java.lang.String NODE_RUN_DEFINITIONS

root node for stored definitions that are used by a job run.

See Also:

Constant Field Values

NODE_RUN_DEFINITIONS_JOB

public static final java.lang.String NODE_RUN_DEFINITIONS_JOB

stores job definition used by a job run.

See Also:

Constant Field Values

NODE_RUN_DEFINITIONS_WORKFLOW

public static final java.lang.String NODE_RUN_DEFINITIONS_WORKFLOW

stores workflow definition used by a job run.

See Also:

Constant Field Values

NODE_RUN_DEFINITIONS_BUCKET

public static final java.lang.String NODE_RUN_DEFINITIONS_BUCKET

stores bucket definitions used by a job run.

See Also:

Constant Field Values

NODE_WORKFLOW_RUN_FINISHING

public static final java.lang.String NODE_WORKFLOW_RUN_FINISHING

indicates whether a workflow run is currently finished.

See Also:

Constant Field Values

Constructor Detail

RunStorageZk

public RunStorageZk()

Method Detail

getCurrentTimestamp

public java.lang.String getCurrentTimestamp()

Specified by:

getCurrentTimestamp in interface RunStorage

Returns:

string with current timestamp

activate

protected void activate(ComponentContext context)

OSGi Declarative Services service activation method.

Parameters:

context - OSGi service component context.

deactivate

protected void deactivate(ComponentContext context)

OSGi Declarative Services service deactivation method.

Parameters:

context - OSGi service component context.

setZooKeeperService

public void setZooKeeperService(ZooKeeperService zkService)

method for DS to set a service reference.

Parameters:

zkService - ZooKeeperService reference.

unsetZooKeeperService

public void unsetZooKeeperService(ZooKeeperService zkService)

method for DS to unset a service reference.

Parameters:

zkService - ZooKeeperService reference.

startJobRun

public void startJobRun(java.lang.String jobName,
               java.lang.String jobRunId,
               JobRunMode jobRunMode,
               JobRunDefinitions jobRunDefs)
                 throws RunStorageException

Starts a job run. If another instance already started the job the method will throw an exception.

Specified by:

startJobRun in interface RunStorage

Parameters:

jobName - The name of the job to start.

jobRunId - The job run id for the started job run.

jobRunMode - The job run mode for the started job run.

jobRunDefs - The job run definitions that are stored for this job run.

Throws:

RunStorageException - If there's an error while accessing the run storage.

finishJobRun

public boolean finishJobRun(java.lang.String jobName,
                   java.lang.String jobRunId)
                     throws RunStorageException

Called when a job run is finished. This does _not_ remove the job run data. Since it is possible that more instances of a job manager try to finish a job run concurrently the one that succeeded will get 'true' as return value, the others false but no exception will be thrown in that case, because this is not an exceptional behavior.

Specified by:

finishJobRun in interface RunStorage

Parameters:

jobName - The job name

jobRunId - The job run id for the finished job run.

Returns:

true if the caller was the first to finish the job, false if another caller finished the job in the meantime.

Throws:

RunStorageException - error

cancelJobRun

public java.util.List<java.lang.String> cancelJobRun(java.lang.String jobName,
                                            java.lang.String jobRunId)
                                              throws RunStorageException

Cancels the given job run data -> active workflow runs and current tasks will be canceled. (Does not set job run state!).

Specified by:

cancelJobRun in interface RunStorage

Parameters:

jobName - the job

jobRunId - the job run for which to remove the data

Returns:

the ids of the workflow runs that were canceled.

Throws:

RunStorageException - error

deleteJobRun

public void deleteJobRun(java.lang.String jobName,
                java.lang.String jobRunId)
                  throws RunStorageException

Removes the job run data of the given job run.

Specified by:

deleteJobRun in interface RunStorage

Parameters:

jobName - the job

jobRunId - the job run for which to remove the data

Throws:

RunStorageException - error

getCurrentJobs

public java.util.Collection<java.lang.String> getCurrentJobs()
                                                      throws RunStorageException

Specified by:

getCurrentJobs in interface RunStorage

Returns:

names of job that have currently run data stored in this RunStorage. Usually it means that they are JobState.PREPARING,JobState.RUNNING, or JobState.FINISHING, but while cleaning up jobs in states JobState.SUCCEEDED and JobState.FAILED, they are still in RunStorage, too.

Throws:

RunStorageException - error

getJobRunId

public java.lang.String getJobRunId(java.lang.String jobName)
                             throws RunStorageException

Specified by:

getJobRunId in interface RunStorage

Parameters:

jobName - the job

Returns:

the id of the currently active job run, or 'null' if no job run is active.

Throws:

RunStorageException - error

getJobRunData

public AnyMap getJobRunData(java.lang.String jobName,
                   boolean returnDetails)
                     throws RunStorageException

Specified by:

getJobRunData in interface RunStorage

Parameters:

jobName - job

returnDetails - whether to return details (e.g. workflow/bucket definition)

Returns:

the data for the currently active job run, or 'null' if no job run is active.

Throws:

RunStorageException - error

getWorkflowRunData

public AnyMap getWorkflowRunData(java.lang.String jobName,
                        java.lang.String workflowRunId)
                          throws RunStorageException

get information about a workflow run. The result contains these keys with integer values:

• JobManagerConstants.DATA_WORKFLOW_RUN_NO_OF_ACTIVE_TASKS
• JobManagerConstants.DATA_WORKFLOW_RUN_NO_OF_TRANSIENT_BULKS

The result may contain additional keys.

Specified by:

getWorkflowRunData in interface RunStorage

Returns:

workflow run information

Throws:

RunStorageException - jobrun or workflowrun are not active anymore.

getWorkerData

public AnyMap getWorkerData(java.lang.String jobName,
                   java.lang.String worker)

Returns:

worker data for given worker in given job. return empty AnyMap if error while reading.

getJobState

public JobState getJobState(java.lang.String jobName)
                     throws RunStorageException

Specified by:

getJobState in interface RunStorage

Parameters:

jobName - job

Returns:

the state of the currently active job run for the given job, or 'null' if no job run is active.

Throws:

RunStorageException - error

getJobState

public JobState getJobState(java.lang.String jobName,
                   java.lang.String jobRunId)
                     throws RunStorageException

Specified by:

getJobState in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

Returns:

the state of the given job run for the given job, or 'null' if this job run is not active

Throws:

RunStorageException - error

getJobRunInfo

public JobRunInfo getJobRunInfo(java.lang.String jobName)
                         throws RunStorageException

Specified by:

getJobRunInfo in interface RunStorage

Parameters:

jobName - job

Returns:

if job is defined and active: id and state of current run. Else null.

Throws:

RunStorageException - error

setJobState

public void setJobState(java.lang.String jobName,
               java.lang.String jobRunId,
               JobState jobState)
                 throws RunStorageException

Sets the state for the given job run. Do not invoke concurrently! This method does not check any preconditions and overwrites the state without further notice. Please check that you are the only one to set the state for a job run before invoking this method!

Specified by:

setJobState in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

jobState - the new state to set

Throws:

RunStorageException - error

setJobState

public boolean setJobState(java.lang.String jobName,
                  java.lang.String jobRunId,
                  JobState expectedState,
                  JobState newState)
                    throws RunStorageException

Sets the state for the given job run if the given expected state matches the current state.

Specified by:

setJobState in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

expectedState - the old state that is expected to be the current state

newState - the new state to set

Returns:

'true', if new state could be set, otherwise 'false'.

Throws:

RunStorageException - error

startWorkflowRun

public java.lang.String startWorkflowRun(java.lang.String jobName,
                                java.lang.String jobRunId)
                                  throws RunStorageException

Called when a new workflow run is started for the given job run.

Specified by:

startWorkflowRun in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

Returns:

the workflow run id of the new workflow run.

Throws:

RunStorageException - error

startCompletionWorkflowRun

public java.lang.String startCompletionWorkflowRun(java.lang.String jobName,
                                          java.lang.String jobRunId)
                                            throws RunStorageException

Called when a new completing workflow run is started for the given job run.

Specified by:

startCompletionWorkflowRun in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

Returns:

the workflow run id of the new workflow run.

Throws:

RunStorageException - error

prepareToFinishWorkflowRun

public boolean prepareToFinishWorkflowRun(java.lang.String jobName,
                                 java.lang.String jobRunId,
                                 java.lang.String workflowRunId)
                                   throws RunStorageException

Called before a workflow run is finished and deleted. Decides whether this thread will be allowed to finish the workflow run. Two flags are set here to avoid concurrent races: 1.) a temporary job run data map entry 'workflow-run-finish-' is set to a locally created id, to make sure that only one thread is doing the finishing. 2.) a temporary ephemeral 'finishing' flag is set to show that this workflow run is currently finished. This is needed because server could be crashed in between. If entry above is set and flag is missing, than entry above can be overwritten! Both entries are removed in RunStorage.deleteWorkflowRun(String, String, String).

Specified by:

prepareToFinishWorkflowRun in interface RunStorage

Returns:

'true' if the calling thread is the one to finish the workflow, false if not.

Throws:

RunStorageException

successfulWorkflowRun

public void successfulWorkflowRun(java.lang.String jobName,
                         java.lang.String jobRunId,
                         java.lang.String workflowRunId)
                           throws RunStorageException

Called when a workflow run successfully finished. The workflow run data is merged into the job run data. Does not delete the data of the workflow run.

Specified by:

successfulWorkflowRun in interface RunStorage

Throws:

RunStorageException

failedWorkflowRun

public void failedWorkflowRun(java.lang.String jobName,
                     java.lang.String jobRunId,
                     java.lang.String workflowRunId)
                       throws RunStorageException

Called when a workflow run failed. The workflow run data is merged into the job run data. Does not delete the data of the workflow run.

Specified by:

failedWorkflowRun in interface RunStorage

Throws:

RunStorageException

deleteWorkflowRunData

public void deleteWorkflowRunData(java.lang.String jobName,
                         java.lang.String jobRunId,
                         java.lang.String workflowRunId)
                           throws RunStorageException

Deletes the data of the given workflow run.

Specified by:

deleteWorkflowRunData in interface RunStorage

Throws:

RunStorageException

deleteWorkflowRun

public void deleteWorkflowRun(java.lang.String jobName,
                     java.lang.String jobRunId,
                     java.lang.String workflowRunId)
                       throws RunStorageException

Deletes the given workflow run.

Specified by:

deleteWorkflowRun in interface RunStorage

Throws:

RunStorageException

checkAndCleanupActiveWorkflowRuns

public boolean checkAndCleanupActiveWorkflowRuns(java.lang.String jobName,
                                        java.lang.String jobRunId)
                                          throws RunStorageException

Checks if there are active workflow runs. Also checks if there are completed workflow runs that someone else started finishing but couldn't succeed. Such completed workflow runs are cleaned up. (finished + removed)

Specified by:

checkAndCleanupActiveWorkflowRuns in interface RunStorage

Returns:

'true' if there are active workflow runs, otherwise 'false'.

Throws:

RunStorageException

hasWorkflowRun

public boolean hasWorkflowRun(java.lang.String jobName,
                     java.lang.String jobRunId,
                     java.lang.String workflowRunId)
                       throws RunStorageException

Specified by:

hasWorkflowRun in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

Returns:

'true' if the given workflow run exists.

Throws:

RunStorageException - error

hasTasks

public boolean hasTasks(java.lang.String jobName,
               java.lang.String jobRunId,
               java.lang.String workflowRunId)
                 throws RunStorageException

Specified by:

hasTasks in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

Returns:

'true' if there are active tasks in the given workflow run, otherwise 'false'

Throws:

RunStorageException - error

startTask

public void startTask(java.lang.String jobName,
             java.lang.String jobRunId,
             java.lang.String workflowRunId,
             java.lang.String stepId,
             java.lang.String taskId)
               throws RunStorageException

Called when a new task is started for the given step in the given workflow of the given job run.

Specified by:

startTask in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

stepId - step id

taskId - task id

Throws:

RunStorageException

startTask

public void startTask(java.lang.String jobName,
             java.lang.String jobRunId,
             java.lang.String workflowRunId,
             java.lang.String stepId,
             java.lang.String taskId,
             int numberOfRetries)
               throws RunStorageException

Called when a new task is started for the given step in the given workflow of the given job run. The numberOfRetries parameter adds the possibility to store the number of times this task has already been restarted.

Specified by:

startTask in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

stepId - step id

taskId - task id

numberOfRetries - current number of retries for this task

Throws:

RunStorageException - if task already exists, or other error while storing task

startTasks

public void startTasks(java.lang.String jobName,
              java.lang.String jobRunId,
              java.lang.String workflowRunId,
              java.util.Collection<Task> tasks)
                throws RunStorageException

Start multiple new tasks for a given job/workflow run at once.

Specified by:

startTasks in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

tasks - new tasks

Throws:

RunStorageException - error while storing task

finishTask

public void finishTask(java.lang.String jobName,
              java.lang.String jobRunId,
              java.lang.String workflowRunId,
              java.lang.String stepId,
              java.lang.String taskId,
              java.util.Map<java.lang.String,java.lang.Number> workerCounter,
              java.util.Map<java.lang.String,java.lang.String> properties)
                throws RunStorageException

Called when a task is successfully finished. Deletes the data of a given task for the given step in the given workflow of the given job run.

Specified by:

finishTask in interface RunStorage

Throws:

RunStorageException

obsoleteTask

public void obsoleteTask(java.lang.String jobName,
                java.lang.String jobRunId,
                java.lang.String workflowRunId,
                java.lang.String stepId,
                java.lang.String taskId,
                java.util.Map<java.lang.String,java.lang.String> properties)
                  throws RunStorageException

Called when a task finished as obsolete. Deletes the data of a given task for the given step in the given workflow of the given job run.

Specified by:

obsoleteTask in interface RunStorage

Throws:

RunStorageException

failedTask

public void failedTask(java.lang.String jobName,
              java.lang.String jobRunId,
              java.lang.String workflowRunId,
              java.lang.String stepId,
              java.lang.String taskId,
              boolean failedAfterRetry,
              java.util.Map<java.lang.String,java.lang.String> properties)
                throws RunStorageException

Called when a task is failed. Deletes the data of a given task for the given step in the given workflow of the given job run.

Specified by:

failedTask in interface RunStorage

Throws:

RunStorageException

retriedTask

public void retriedTask(java.lang.String jobName,
               java.lang.String jobRunId,
               java.lang.String workflowRunId,
               java.lang.String stepId,
               java.lang.String taskId,
               boolean retryByWorker,
               java.util.Map<java.lang.String,java.lang.String> properties)
                 throws RunStorageException

Called when a task is retried. Deletes the data of a given task for the given step in the given workflow of the given job run.

Specified by:

retriedTask in interface RunStorage

retryByWorker - whether the retry was triggered by a worker

Throws:

RunStorageException

hasTask

public boolean hasTask(java.lang.String jobName,
              java.lang.String workflowRunId,
              java.lang.String taskId)
                throws RunStorageException

Check if the given task with the given task id exists for the job and workflowrun.

Specified by:

hasTask in interface RunStorage

Parameters:

jobName - the name of the job.

workflowRunId - the id of the workflow run

taskId - the id of the task

Returns:

'true' if such a task exists, 'false' if not.

Throws:

RunStorageException - an exception occurred while checking for task existence.

getTaskRetries

public int getTaskRetries(java.lang.String jobName,
                 java.lang.String jobRunId,
                 java.lang.String workflowRunId,
                 java.lang.String stepId,
                 java.lang.String taskId)
                   throws RunStorageException

get the number of retries for the task.

Specified by:

getTaskRetries in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

stepId - step id

taskId - task id

Returns:

The number of retries for the task.

Throws:

RunStorageException - if task already exists, or other error while storing task

addTransientBulk

public void addTransientBulk(java.lang.String jobName,
                    java.lang.String jobRunId,
                    java.lang.String workflowRunId,
                    java.lang.String bulk)
                      throws RunStorageException

Stores a transient bulk for the given workflow run.

Specified by:

addTransientBulk in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

bulk - the bulk id

Throws:

RunStorageException - error

getTransientBulks

public java.util.Collection<java.lang.String> getTransientBulks(java.lang.String jobName,
                                                       java.lang.String jobRunId,
                                                       java.lang.String workflowRunId)
                                                         throws RunStorageException

Specified by:

getTransientBulks in interface RunStorage

Parameters:

jobName - job

jobRunId - job run

workflowRunId - workflow run

Returns:

the transient bulk ids for the given workflow run

Throws:

RunStorageException - error

clear

public void clear()
           throws RunStorageException

Remove all jobmanager run data. Can be called concurrently.

Specified by:

clear in interface RunStorage

Throws:

RunStorageException - error

getTriggeredJobs

public java.util.Collection<java.lang.String> getTriggeredJobs(java.lang.String bucketId)
                                                        throws RunStorageException

Specified by:

getTriggeredJobs in interface RunStorage

Parameters:

bucketId - a bucket ID

Returns:

names of jobs triggered by this bucket

Throws:

RunStorageException - error

addJobTrigger

public void addJobTrigger(java.lang.String bucketId,
                 java.lang.String jobName)
                   throws RunStorageException

Specified by:

addJobTrigger in interface RunStorage

Parameters:

bucketId - a bucket ID

jobName - job name

Throws:

RunStorageException - error

removeJobTrigger

public void removeJobTrigger(java.lang.String bucketId,
                    java.lang.String jobName)
                      throws RunStorageException

Specified by:

removeJobTrigger in interface RunStorage

Parameters:

bucketId - a bucket ID

jobName - job name

Throws:

RunStorageException - error

registerJobRunListener

public void registerJobRunListener(JobRunListener jobRunListener,
                          java.lang.String jobName)

Specified by:

registerJobRunListener in interface RunStorage

Parameters:

jobRunListener - listener to be notified about job run events.

jobName - job on which to listen