Scheduling workflow tasks with unknown task execution time by combining machine-learning and greedy-optimization

Workflow tasks are time-sensitive and their task completion utility, i.e., value of task completion, is inversely proportional to their completion time. Existing solutions to the NP-hard problem of utility-maximization task scheduling were achieved under the assumptions of linear Time Utility Function (TUF), i.e., utility is inversely proportional to completion time following a linear function, and prior knowledge of task execution time, which is unrealistic for many applications and dynamic systems. This paper proposes a novel model of combining greedy optimization with machine learning for scheduling time-sensitive tasks with convex TUF and unknown task execution time on heterogeneous cloud servers offline nonpreemptively to maximize the total utility of input tasks. For a set of time-sensitive tasks with data dependencies, we first employ multi-layer perceptron neural networks to predict task execution time by utilizing historical data. Then, by solving a linear program after relaxing the disjunctive constraint introduced by the nonpreemption requirement to calculate maximum utility increment, we propose a novel greedy algorithm of marginal incremental utility maximization that jointly determines the task-to-processor allocation plan and tasks' execution sequence on each processor. We then show that our algorithm has an expected approximation ratio of (e−1)(τ−2)eτ for convex TUF and e−13e≈0.21 for linear TUF, where τ is the ratio of total completion utility over total delay cost under optimal scheduling. Our result presents the first polynomial-time approximation solution for this problem that achieves a performance guarantee of bounded ratio for convex TUF and constant ratio for linear TUF respectively. Extensive experiment results through both simulation and real cloud implementation demonstrate significant performance improvement of our algorithm over the known results.