Premium
Timing trials, or the trials of timing: experiments with scripting and user‐interface languages
Author(s) -
Kernighan Brian W.,
Wyk Christopher J. Van
Publication year - 1998
Publication title -
software: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.437
H-Index - 70
eISSN - 1097-024X
pISSN - 0038-0644
DOI - 10.1002/(sici)1097-024x(19980710)28:8<819::aid-spe184>3.0.co;2-o
Subject(s) - computer science , garbage collection , scripting language , interface (matter) , programming language , benchmark (surveying) , code (set theory) , operating system , garbage , geography , geodesy , bubble , set (abstract data type) , maximum bubble pressure method
This paper describes some basic experiments to see how fast various popular scripting and user‐interface languages run on a spectrum of representative tasks. We found enormous variation in performance, depending on many factors, some uncontrollable and even unknowable. There seems to be little hope of predicting performance in other than a most general way; if there is a single clear conclusion, it is that no benchmark result should ever be taken at face value. A few general principles hold: Compiled code usually runs faster than interpreted code: the more a program has been ‘compiled’ before it is executed, the faster it will run. Memory‐related issues and the effects of memory hierarchies are pervasive: how memory is managed, from hardware caches to garbage collection, can change runtimes dramatically. Yet users have no direct control over most aspects of memory management. The timing services provided by programs and operating systems are woefully inadequate. It is difficult to measure runtimes reliably and repeatably even for small, purely computational kernels, and it becomes significantly harder when a program does much I/O or graphics. Although each language shines in some situations, there are visible and sometimes surprising deficiencies even in what should be mainstream applications. We encountered more than a few bugs, size limitations, maladroit features, and total mysteries. © 1998 John Wiley & Sons, Ltd.