English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

This paper presents an approach for data-driven modeling of hidden, stationary temporal dynamics in sequential images or videos using deep learning and Bayesian non-parametric techniques. In particular, a deep Convolutional Neural Network (CNN) is used to extract spatial features in an unsupervised fashion from individual images and then, a Gaussian process is used to model the temporal dynamics of the spatial features extracted by the deep CNN. By decomposing the spatial and temporal components and utilizing the strengths of deep learning and Gaussian processes for the respective sub-problems, we are able to construct a model that is able to capture complex spatio-temporal phenomena while using relatively small number of free parameters. The proposed approach is tested on high-speed grey-scale video data obtained of combustion flames in a swirl-stabilized combustor, where certain protocols are used to induce instability in combustion process. The proposed approach is then used to detect and predict the transition of the combustion process from stable to unstable regime. It is demonstrated that the proposed approach is able to detect unstable flame conditions using very few frames from high-speed video. This is useful as early detection of unstable combustion can lead to better control strategies to mitigate instability. Results from the proposed approach are compared and contrasted with several baselines and recent work in this area. The performance of the proposed approach is found to be significantly better in terms of detection accuracy, model complexity and lead-time to detection. Devesh Jha et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. 1. MOTIVATION AND INTRODUCTION Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstractions (LeCun, Bengio, & Hinton, 2015; Bengio, Courville, & Vincent, 2013; Hinton & Salakhutdinov, 2006). Deep learning methods are representation-learning techniques obtained by composition of non-linear modules or layers that each transform the representation at the previous level into a higher and slightly more abstract level in a hierarchical manner. The main idea is that by cascading a large number of such transformations, very complex functions can be learned in a data-driven manner. Convolutional deep neural nets (Krizhevsky, Sutskever, & Hinton, 2012; Lee, Grosse, Ranganath, & Ng, 2009) are designed to process data that come in the form of multiple arrays, for example images. In this technique, a discrete spatial convolution filter is used for detecting highly correlated distinctive motifs across an image. The same process can be extended to find temporal features by using a convolution filter over time, and using a stack of images as input. However, this will lead to an increase in the number of the hyperparameters used to train such a spatio-temporal convolutional network. Instead, we propose a framework, where a convolutional network is used to extract features from images and a Bayesian nonparametric model like the Gaussian process (Rasmussen & Williams, 2006) is used to model the temporal dynamics of the features; the objective is to reduce the number of parameters used to train such a network while modeling the spatiotemporal dynamics in an image sequence or videos. The proposed algorithm is used to detect stable and unstable combustion flame dynamics in a swirl-stabilized combustor. Another motivation is that adding a Gaussian process-based filter might allow to reduce over-fitting of the deep CNN to some extent as it allows to relate to the causal dynamics present in the data in a much lower dimensional space. International Journal of Prognostics and Health Management, ISSN2153-2648, 2016 022 1 INTERNATIONAL JOURNAL OF PROGNOSTICS AND HEALTH MANAGEMENT Combustion instability is a highly nonlinear coupled thermoacoustic phenomenon that results in self-sustained oscillations in a combustor. These oscillations may result in severe structural degradation in gas turbine engines. Some good surveys on the current understanding of the mechanisms for combustion instability phenomena can be found in (O’Connor, Acharya, & Lieuwen, 2015; Sé et al., 2003; Candel, Durox, Schuller, Bourgouin, & Moeck, 2014; Huang & Yang, 2009; Moeck, Bourgouin, Durox, Schuller, & Candel, 2012). The current state-of-the-art techniques rely heavily on model-based approaches for analysis of the process. The current technical literature lacks rigorous statistical analysis of the combustion instability phenomenon so that the predictive power of the data has been largely overlooked. Active combustion instability control (ACIC) with fuel modulation has proven to be an effective approach for reducing pressure oscillations in combustors (Banaszuk, Mehta, Jacobson, & Khibnik, 2006; Banaszuk, Mehta, & Hagen, 2007). Based on the work available in literature, one can conclude that the performance of ACIC is primarily limited by large delays in the feedback loop and limited actuator bandwidth (Banaszuk et al., 2006, 2007). Model-based approaches for active control are infeasible as complexity of the models and uncertain measurements make real-time estimates difficult. On the other hand, use of machine learning techniques for information extraction remain unexplored for this problem. From the perspective of active control of the unstable phenomena, it is necessary to accurately detect and, desirably, predict the future states of the combustion process. The goal of this paper is to present a statistical model for the instability phenomenon during combustion which could be used to design a statistical filter to accurately predict the system states. This can potentially alleviate the problems with delay in the ACIC feedback loop and thus possibly improve the performance. Some recent work on statistical analysis of combustion instability using pressure time-series data could be found in (Jha, Virani, & Ray, 2016; Virani, Jha, & Ray, 2016). The work presented in (Jha et al., 2016) shows the change in the underlying Markov model for pressure data as the system approaches the unstable regime resulting in self-sustained oscillations of the flame. Some other popular methods for detection of coherent structures include proper orthogonal decomposition (POD) (Berkooz, Holmes, & Lumley, 1993) and dynamic mode decomposition (DMD) (Schmid, 2010), which use tools of spectral theory to derive spatial coherent structure modes. Specifically, DMD has been used to estimate growth rates and frequencies from experimental data and also for stability analysis of experimental data. Recently, some analyses were also presented using deep learning for detection of combustion instabilities (Sarkar, Lore, Sarkar, Ramanan, et al., 2015; Sarkar, Jha, Lore, Sarkar, & Ray, 2016). More recently the work done in (Sarkar, Lore, & Sarkar, 2015) presents a neuro-symbolic approach where the output a deep convolutional network are analyzed by a Markov modeling module to construct an anomaly measure. However, as we demonstrate later, the advantages of using a deep neural network is unclear. Another analysis is presented in (Hauser, Li, Li, & Ray, 2016) using various image analysis techniques like histogram of oriented gradients (HOG) and Wavelets; however, the final decision is made by making a Markov model out of the features and thus requires long sequences to arrive at a decision. Moreover, perfect separability between stable and unstable classes is not achieved. This paper presents further insights into the combustion instability phenomena from a data-driven perspective (Darema, 2005) and presents a framework which allows temporal learning in video data in a lower dimensional subspace of features produced by a deep learning module. We show that flames have different spatial structures at unstable behavior when compared to stable behavior and with an appropriate architecture of neural network, we can perfectly capture the associated distinguishing features. Contributions: This paper presents a framework for modeling of spatio-temporal dynamics in sequential images using deep learning and Gaussian processes. We use the proposed algorithm to present a statistical analysis of the complex combustion instability phenomena and show that we are able to capture the change in the model as the system moves from a stable regime to an unstable regime. We show that the deep CNN is able to achieve fairly good classification performance; however the use of a Gaussian process filter to model temporality of extracted features results in perfect detection with very low false alarm rates and much shorter lead time to detection. The advantage of the proposed method is that it can be used for making early predictions of the transition from a stable regime to quasi-periodic unstable oscillations in combustion system. 2. PRELIMINARIES AND PROBLEM FORMULATION In this section, we will very briefly describe the convolutional neural nets and Gaussian processes. Interested readers are referred to (Lee et al., 2009; Rasmussen & Williams, 2006; Bengio et al., 2013) for an in-depth discussion on these topics. After the brief review, we will state the problem considered in this paper. The idea to use Gaussian processes to model the features obtained from deep CNN is to be able to add memory to the estimation filter without going to the architecture of a recurrent neural network. 2.1. Convolutional Neural Networks In this section, we briefly introduce the deep convolutional neural networks (CNN) for the completeness of the paper and motivation for use in this work. Deep convolutional neural networks have a long history (Le Cun Y. et al., 1990) but

Temporal Learning in Video Data Using Deep Learning and Gaussian Processes