Premium
The Keys to Unlocking Public Payments Data
Author(s) -
Rahal Charles
Publication year - 2018
Publication title -
kyklos
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.766
H-Index - 58
eISSN - 1467-6435
pISSN - 0023-5962
DOI - 10.1111/kykl.12171
Subject(s) - payment , computer science , public domain , politics , data science , procurement , transparency (behavior) , merge (version control) , big society , computer security , business , world wide web , political science , marketing , information retrieval , law , philosophy , theology
Summary We mechanize some of the richest yet significantly under‐utilized data resources within developed, ‘Open Data' economies. We show how it is possible to scrape, parse, clean and merge tens of thousands of disaggregated public payments datasets in an attempt to bridge the methodological gap between newly available data from the administrative sphere and applications in empirical social science research. We outline techniques to unambiguously link records to various freely available institutional registers. In particular, we offer guidance on overcoming the substantial challenges of heterogeneous provision and administrative recording errors in the absence of Uniform Resource Identifiers, namely in the form of an approximate, domain‐specific ‘record‐linkage' type matching algorithm. As an illuminating example, we construct a cleaned database of 24,581,192 local government payments subject to the Local Transparency Codes which total £169.87bn in value. We overcome various challenges in a detailed examination of the procurement of services by local government from the voluntary sector: an important contemporary issue due to the rise of the ‘Big Society’ political ideology of the early 21st century. Finally, we motivate future work in this area and discuss potential international applications and practical advancements.