Task Scheduler

Task scheduling follows these principles:

  • each task is assigned a *user*
  • users belong to *groups*
  • workers fetch tasks for *groups*, with a given priority

Mapping a user to a group can be dynamic: the broker can be configured with a function which at runtime assigns a task to a group given the original userid.

Load can be assigned to different workers and the administrator can decide to give more priority to specific group of users.

An example can be useful in order to better understand:

  • there are two task types, for example "sendemail" and "processhits"
  • there are two groups of users, group "gold" and group "poor"
  • there are many users
  • there are 10 workers
  • workers are configure in order to process at most 10 tasks of type "sendemail" and at most 2 tasks of type "processhits" (which is a very CPU intensive task), but in total each JVM will be working to at most 10 tasks
  • 8 workers are configured to take tasks for group "gold" and then for group "poor"
  • 2 workers are configured to take tasks for group "poor" and then for group "gold"

This kind of configuration will give more working time to the "gold" users.
When a task is submitted to the broker it is assigner to an user so, if we suppose that is a "poor" user, the tasks could remain in the system for some time because "poor" users have less resources and this impose a little bit of "latency" to the execution of the task submitted by them if the system is also working on tasks submitted by "gold" users.
If the user is promoted to the "gold" group his tasks are automatically assigned to the "gold" group and the waiting tasks will be processed immediately.