Kepler Science Operations Center pipeline framework

The Kepler Mission is designed to continuously monitor up to 170,000 stars at a 30-minute cadence for 3.5 years searching for Earth-size planets. The data are processed at the Science Operations Center at NASA Ames Research Center. Because of the large volume of data and the memory needed, as well as the CPU-intensive nature of the analyses, significant computing hardware is required. We have developed generic pipeline framework software that is used to distribute and synchronize processing across a cluster of CPUs and provide data accountability for the resulting products. The framework is written in Java and is, therefore, platform-independent. The framework scales from a single, standalone workstation (for development and research on small data sets) to a full cluster of homogeneous or heterogeneous hardware with minimal configuration changes. A plug-in architecture provides customized, dynamic control of the unit of work without the need to modify the framework. Distributed transaction services provide for atomic storage of pipeline products for a unit of work across a relational database and the custom Kepler DB. Generic parameter management and data accountability services record parameter values, software versions, and other metadata used for each pipeline execution. A graphical user interface allows for configuration, execution, and monitoring of pipelines. The framework was developed for the Kepler Mission based on Kepler requirements, but the framework itself is generic and could be used for a variety of applications where these features are needed.