Keynote by Robert I. Davis
Posted: Mon Jul 04, 2016
Title: On the Evaluation of Schedulability Tests for Real-Time Scheduling Algorithms
Speaker: Robert I. Davis (University of York, UK)
Abstract: This talk discusses criteria and methods that can be used to evaluate the performance of schedulability tests for real-time scheduling algorithms. We briefly review theoretical methods such as utilisation bounds, dominance relations, and speed-up factors, as well as empirical methods such as simulation and case studies. The talk then focusses on empirical evaluation and the generation of parameters for synthetic task sets. We discuss the need for a systematic approach, issues of bias, confounding variables and statistical confidence. A simple evaluation framework is outlined, covering how to generate task utilisation values, periods and other parameters, as well as how task set parameters can be grounded in data from benchmarks. A simple systematic approach to covering the parameter space is proposed, and different ways of presenting results considered. Finally, the talk ends with an open discussion of the benefits of having a de-facto standard approach, and how we might improve the quality of empirical evaluation in the real-time community.
Attached slides: Attached paper:
Speaker: Robert I. Davis (University of York, UK)
Abstract: This talk discusses criteria and methods that can be used to evaluate the performance of schedulability tests for real-time scheduling algorithms. We briefly review theoretical methods such as utilisation bounds, dominance relations, and speed-up factors, as well as empirical methods such as simulation and case studies. The talk then focusses on empirical evaluation and the generation of parameters for synthetic task sets. We discuss the need for a systematic approach, issues of bias, confounding variables and statistical confidence. A simple evaluation framework is outlined, covering how to generate task utilisation values, periods and other parameters, as well as how task set parameters can be grounded in data from benchmarks. A simple systematic approach to covering the parameter space is proposed, and different ways of presenting results considered. Finally, the talk ends with an open discussion of the benefits of having a de-facto standard approach, and how we might improve the quality of empirical evaluation in the real-time community.
Attached slides: Attached paper: