Open AccessStochastic rounding: implementation, error analysis and applications
Author(s) -
Matteo Croci,
Massimiliano Fasi,
Nicholas J. Higham,
Théo Mary,
Mantas Mikaitis
Publication year - 2022
Publication title -
royal society open science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.84
H-Index - 51
ISSN - 2054-5703
DOI - 10.1098/rsos.211631
Subject(s) - rounding , round off error , probabilistic logic , upper and lower bounds , sequence (biology) , algorithm , constant (computer programming) , computer science , expected value , mathematics , statistics , mathematical analysis , biology , genetics , programming language , operating system
Stochastic rounding (SR) randomly maps a real numberx to one of the two nearest values in a finite precision number system. The probability of choosing either of these two numbers is 1 minus their relative distance tox . This rounding mode was first proposed for use in computer arithmetic in the 1950s and it is currently experiencing a resurgence of interest. If used to compute the inner product of two vectors of lengthn in floating-point arithmetic, it yields an error bound with constantn u with high probability, whereu is the unit round-off. This is not necessarily the case for round to nearest (RN), for which the worst-case error bound has constantnu . A particular attraction of SR is that, unlike RN, it is immune to the phenomenon of stagnation, whereby a sequence of tiny updates to a relatively large quantity is lost. We survey SR by discussing its mathematical properties and probabilistic error analysis, its implementation, and its use in applications, with a focus on machine learning and the numerical solution of differential equations.