org.eclipse.smila.taskmanager.persistence.zk

Class TaskStorageZk

java.lang.Object

org.eclipse.smila.taskmanager.persistence.zk.TaskStorageZk

All Implemented Interfaces:

TaskStorage

public class TaskStorageZk
extends java.lang.Object
implements TaskStorage

Zookeeper task storage implementation. Zookeeper structure: /smila/taskmanager/hosts/ /smila/taskmanager/qualifierlocks// ... (also see ZkTaskQueue) A

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	HOST_COUNTER_MAP_ZK_PATH Zookeeper path to stored host counters.
static java.lang.String	QUALIFIER_LOCKS_ZK_PATH Zookeeper path to qualifier locks.
static java.lang.String	TASK_HANDLER_CONFIGURATION
static long	TASKSPERHOST_UNLIMITED default value for _maxNoOfTasksPerHost.

Constructor Summary

Constructors

Constructor and Description
TaskStorageZk()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	activate(ComponentContext context)
void	checkQualifierLockAge(long timeToLiveMs) check age of qualifier locks.
void	clear() remove all tasks from storage and reset all counters.
void	createTaskQueue(java.lang.String name)
protected void	deactivate(ComponentContext context)
void	deleteTask(java.lang.String workerName, java.lang.String taskId) Remove the task from the inprogress part of the workers queue.
java.util.List<Task>	filterDuplicates(java.util.List<Task> tasksToBeAdded) return new list of tasks with only those tasks from the given task list that do not have property Task.PROPERTY_UNIQUENESS_TAG set or for which currently no task is in the Todo queue that has the same uniqueness tag.
long	getFailSafetyLevel()
Task	getInProgressTask(java.lang.String workerName, java.lang.String taskId) read the task content of an in-progress task.
long	getMaxNoOfTasksPerHost()
java.util.Map<java.lang.String,java.lang.Integer>	getScaleUpCounters()
Task	getTask(java.lang.String workerName, java.lang.String workerHost, java.util.Collection<java.lang.String> qualifiers) Returns a task that's not "in process" from the given workerName for the given messageCondition.
java.util.Map<java.lang.String,TaskCounter>	getTaskCounters() Get number of tasks in all current queues.
Any	getTaskInfo(java.lang.String workerName, java.lang.String section, java.lang.String taskName) Prepares information on a task stored in the task storage.
TaskList	getTaskList(java.lang.String workerName, java.lang.String section, int maxCount) Get task list for current task pipe and sub-type.
protected java.util.Collection<ZkTaskQueue>	getTaskQueues() get task queues as unmodifyable collection.
boolean	hasTaskQueue(java.lang.String name)
boolean	isLockedQualifier(java.lang.String workerName, java.lang.String qualifier) check if a lock exists for given worker and qualifier.
void	keepAlive(java.lang.String workerName, java.lang.String taskId) Signals that the worker processing the given task is still alive.
void	lockQualifiers(java.lang.String workerName, java.util.Collection<java.lang.String> qualifiers) prevent new todo tasks from being added for the given worker and qualifiers.
void	purge(java.lang.String workerName) Remove all tasks from the given task pipe.
AnyMap	readConfiguration() Return stored taskmanager configuration.
void	removeTasks(AnyMap filterMap) Remove canceled tasks identified by the filter map from todo and in-progress queue.
void	setClusterConfigService(ClusterConfigService ccs) set new ClusterConfigService.
void	setMaxNoOfTasksPerHost(long maxTasks) set the maximum number of tasks that should be delivered to a host.
void	setZooKeeperService(ZooKeeperService zooKeeperService)
void	storeConfiguration(AnyMap config) Store given configuration.
void	storeInProgressTask(Task task) Stores a new task to the inprogress queue of the given task's worker.
void	storeTask(java.lang.String workerName, Task task) Stores a new task to the todo queue of the given workerName.
void	storeTask(Task task) Stores a new task to the todo queue of the given task's worker.
void	unsetClusterConfigService(ClusterConfigService ccs) remove an ClusterConfigService.
void	unsetZooKeeperService(ZooKeeperService zooKeeperService)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HOST_COUNTER_MAP_ZK_PATH

public static final java.lang.String HOST_COUNTER_MAP_ZK_PATH

Zookeeper path to stored host counters.

See Also:

Constant Field Values

QUALIFIER_LOCKS_ZK_PATH

public static final java.lang.String QUALIFIER_LOCKS_ZK_PATH

Zookeeper path to qualifier locks.

See Also:

Constant Field Values

TASK_HANDLER_CONFIGURATION

public static final java.lang.String TASK_HANDLER_CONFIGURATION

See Also:

Constant Field Values

TASKSPERHOST_UNLIMITED

public static final long TASKSPERHOST_UNLIMITED

default value for _maxNoOfTasksPerHost.

See Also:

Constant Field Values

Constructor Detail

TaskStorageZk

public TaskStorageZk()

Method Detail

activate

protected void activate(ComponentContext context)

Parameters:

context - OSGi service component context.

deactivate

protected void deactivate(ComponentContext context)

Parameters:

context - OSGi service component context.

createTaskQueue

public void createTaskQueue(java.lang.String name)

Specified by:

createTaskQueue in interface TaskStorage

Parameters:

name - the name of the task queue to create.

hasTaskQueue

public boolean hasTaskQueue(java.lang.String name)

Specified by:

hasTaskQueue in interface TaskStorage

Parameters:

name - the task queue name to check

Returns:

'true' if task for given name exists, 'false' otherwise.

getTaskQueues

protected java.util.Collection<ZkTaskQueue> getTaskQueues()

get task queues as unmodifyable collection.

getMaxNoOfTasksPerHost

public long getMaxNoOfTasksPerHost()

Specified by:

getMaxNoOfTasksPerHost in interface TaskStorage

Returns:

maximum number of tasks that should be delivered to a host. (scale up control)

setMaxNoOfTasksPerHost

public void setMaxNoOfTasksPerHost(long maxTasks)

set the maximum number of tasks that should be delivered to a host. (scale up control)

Specified by:

setMaxNoOfTasksPerHost in interface TaskStorage

Parameters:

maxTasks - maximum number of tasks that should be delivered to a host. (scale up control)

getScaleUpCounters

public java.util.Map<java.lang.String,java.lang.Integer> getScaleUpCounters()
                                                                     throws TaskmanagerException

Specified by:

getScaleUpCounters in interface TaskStorage

Returns:

number of tasks in progress on different hosts.

Throws:

TaskmanagerException - on error.

storeInProgressTask

public void storeInProgressTask(Task task)
                         throws TaskmanagerException

Stores a new task to the inprogress queue of the given task's worker.

Specified by:

storeInProgressTask in interface TaskStorage

Parameters:

task - the task to store, contains the worker name

Throws:

TaskmanagerException - if send fails

storeTask

public void storeTask(Task task)
               throws TaskmanagerException

Stores a new task to the todo queue of the given task's worker.

Specified by:

storeTask in interface TaskStorage

Parameters:

task - the task to store, contains the worker name

Throws:

TaskmanagerException - if send fails

storeTask

public void storeTask(java.lang.String workerName,
             Task task)
               throws TaskmanagerException

Stores a new task to the todo queue of the given workerName. The worker name stored in the task will be ignored.

Specified by:

storeTask in interface TaskStorage

Parameters:

workerName - the worker for which to store the task.

task - the task to store.

Throws:

TaskmanagerException - if send fails

getTask

public Task getTask(java.lang.String workerName,
           java.lang.String workerHost,
           java.util.Collection<java.lang.String> qualifiers)
             throws TaskmanagerException

Returns a task that's not "in process" from the given workerName for the given messageCondition. The returned task will be marked as "in process".

Specified by:

getTask in interface TaskStorage

Parameters:

workerName - the workerName where to get the task from.

workerHost - the host name where the worker is running that requested the task

qualifiers - if not null, allowed qualifiers of returned task.

Returns:

a task from the workerName for the given messageCondition (if any) or null if there is none available.

Throws:

TaskmanagerException - if receive fails

getInProgressTask

public Task getInProgressTask(java.lang.String workerName,
                     java.lang.String taskId)
                       throws TaskmanagerException

read the task content of an in-progress task.

Specified by:

getInProgressTask in interface TaskStorage

Parameters:

workerName - name of worker

taskId - task Id

Returns:

task contents or null if no such task exists.

Throws:

TaskmanagerException - if task is not in-progress or other errors.

deleteTask

public void deleteTask(java.lang.String workerName,
              java.lang.String taskId)
                throws TaskmanagerException

Remove the task from the inprogress part of the workers queue.

Specified by:

deleteTask in interface TaskStorage

Parameters:

workerName - the workerName from where the task was originally get.

taskId - the task identifier.

Throws:

TaskmanagerException - if taskId is invalid or not marked as finishing

keepAlive

public void keepAlive(java.lang.String workerName,
             java.lang.String taskId)
               throws TaskmanagerException

Signals that the worker processing the given task is still alive.

Specified by:

keepAlive in interface TaskStorage

Parameters:

workerName - name/id as string

taskId - task id that is currently processed as string

Throws:

TaskmanagerException - if any error occurs

purge

public void purge(java.lang.String workerName)
           throws TaskmanagerException

Remove all tasks from the given task pipe.

Specified by:

purge in interface TaskStorage

Parameters:

workerName - name/id as string

Throws:

TaskmanagerException - if any error occurs

lockQualifiers

public void lockQualifiers(java.lang.String workerName,
                  java.util.Collection<java.lang.String> qualifiers)

Description copied from interface: TaskStorage

prevent new todo tasks from being added for the given worker and qualifiers. Used by org.eclipse.smila.taskmanager.TaskManager#finishTasks to prevent tasks (re-)entering the cleared qualifiers for the some time. The locks should be kept active by the TaskStorage implementation for at least some minutes, afterwards they may be cleaned up.

Specified by:

lockQualifiers in interface TaskStorage

Parameters:

workerName - a worker's name

qualifiers - a list of qualifiers to lock

isLockedQualifier

public boolean isLockedQualifier(java.lang.String workerName,
                        java.lang.String qualifier)
                          throws TaskmanagerException

check if a lock exists for given worker and qualifier.

Specified by:

isLockedQualifier in interface TaskStorage

Throws:

TaskmanagerException

checkQualifierLockAge

public void checkQualifierLockAge(long timeToLiveMs)

check age of qualifier locks.

Parameters:

timeToLiveMs -

getTaskCounters

public java.util.Map<java.lang.String,TaskCounter> getTaskCounters()
                                                            throws TaskmanagerException

Get number of tasks in all current queues.

Specified by:

getTaskCounters in interface TaskStorage

Returns:

task counters

Throws:

TaskmanagerException - counting fails

getTaskList

public TaskList getTaskList(java.lang.String workerName,
                   java.lang.String section,
                   int maxCount)
                     throws TaskmanagerException

Get task list for current task pipe and sub-type.

Specified by:

getTaskList in interface TaskStorage

Parameters:

workerName - name/id as string

section - task pipe sub-type: "inprogress" or "todo".

maxCount - max. number of tasks in return list

Returns:

task list

Throws:

TaskmanagerException - if accessing task list fails

getTaskInfo

public Any getTaskInfo(java.lang.String workerName,
              java.lang.String section,
              java.lang.String taskName)
                throws TaskmanagerException

Prepares information on a task stored in the task storage.

Specified by:

getTaskInfo in interface TaskStorage

Parameters:

workerName - pipe in which the task is stored

section - queue section ("todo" or "inprogress")

taskName - task identifier

Returns:

task information

Throws:

TaskmanagerException - on error

getFailSafetyLevel

public long getFailSafetyLevel()

Specified by:

getFailSafetyLevel in interface TaskStorage

Returns:

number of nodes that may fail before tasks get lost.

clear

public void clear()
           throws TaskmanagerException

remove all tasks from storage and reset all counters.

Specified by:

clear in interface TaskStorage

Throws:

TaskmanagerException - error

filterDuplicates

public java.util.List<Task> filterDuplicates(java.util.List<Task> tasksToBeAdded)
                                      throws TaskmanagerException

Description copied from interface: TaskStorage

return new list of tasks with only those tasks from the given task list

• that do not have property Task.PROPERTY_UNIQUENESS_TAG set or
• for which currently no task is in the Todo queue that has the same uniqueness tag.

Specified by:

filterDuplicates in interface TaskStorage

Parameters:

tasksToBeAdded - tasks created as follow-up tasks but not yet added to the task manager

Returns:

tasks that are not already contained in its task queue's todo list

Throws:

TaskmanagerException

storeConfiguration

public void storeConfiguration(AnyMap config)
                        throws TaskmanagerException

Description copied from interface: TaskStorage

Store given configuration.

Specified by:

storeConfiguration in interface TaskStorage

Throws:

TaskmanagerException

readConfiguration

public AnyMap readConfiguration()
                         throws TaskmanagerException

Description copied from interface: TaskStorage

Return stored taskmanager configuration.

Specified by:

readConfiguration in interface TaskStorage

Throws:

TaskmanagerException

setZooKeeperService

public void setZooKeeperService(ZooKeeperService zooKeeperService)

Parameters:

zooKeeperService - referenced service

unsetZooKeeperService

public void unsetZooKeeperService(ZooKeeperService zooKeeperService)

Parameters:

zooKeeperService - referenced service

setClusterConfigService

public void setClusterConfigService(ClusterConfigService ccs)

set new ClusterConfigService. To be called by DS runtime before activation.

Parameters:

ccs - new ClusterConfigService

unsetClusterConfigService

public void unsetClusterConfigService(ClusterConfigService ccs)

remove an ClusterConfigService. To be called by DS runtime after deactivation.

Parameters:

ccs - new ClusterConfigService

removeTasks

public void removeTasks(AnyMap filterMap)
                 throws TaskmanagerException

Remove canceled tasks identified by the filter map from todo and in-progress queue.

Specified by:

removeTasks in interface TaskStorage

Parameters:

filterMap - map to identify tasks to be removed

Throws:

TaskmanagerException - an exception if something went wrong