Open AccessHARVESTING, INTEGRATING AND DISTRIBUTING LARGE OPEN GEOSPATIAL DATASETS USING FREE AND OPEN-SOURCE SOFTWARE
Author(s) -
Ricardo C. L. F. Oliveira,
Rafael Pastor Moreno
Publication year - 2016
Publication title -
the international archives of the photogrammetry, remote sensing and spatial information sciences/international archives of the photogrammetry, remote sensing and spatial information sciences
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.264
H-Index - 71
eISSN - 1682-1777
pISSN - 1682-1750
DOI - 10.5194/isprsarchives-xli-b7-939-2016
Subject(s) - open data , python (programming language) , geospatial analysis , upload , cloud computing , computer science , scripting language , database , download , software , world wide web , transparency (behavior) , open government , open source , metadata , interoperability , data science , computer security , remote sensing , operating system , geography
Federal, State and Local government agencies in the USA are investing heavily on the dissemination of Open Data sets produced by each of them. The main driver behind this thrust is to increase agencies’ transparency and accountability, as well as to improve citizens’ awareness. However, not all Open Data sets are easy to access and integrate with other Open Data sets available even from the same agency. The City and County of Denver Open Data Portal distributes several types of geospatial datasets, one of them is the city parcels information containing 224,256 records. Although this data layer contains many pieces of information it is incomplete for some custom purposes. Open-Source Software were used to first collect data from diverse City of Denver Open Data sets, then upload them to a repository in the Cloud where they were processed using a PostgreSQL installation on the Cloud and Python scripts. Our method was able to extract non-spatial information from a ‘not-ready-to-download’ source that could then be combined with the initial data set to enhance its potential use.