F. Serdio, E. Lughofer, K. Pichler, T. Buchegger, M. Pichler and H. Efendic, Multivariate Fault Detection using Vector Autoregressive Moving Average and Orthogonal Transformation in the residual Space, Annual Conference of the Prognostics and Health Management Society, PHM 2013, New Orleans, LA, USA, 2013, pp. 548-555.

1. Multivariate Fault Detection using
Vector Autoregressive Moving Average
and Orthogonal Transformation
in Residual Space
Francisco Serdio Fernández
Department of Knowledge-Based
Francisco Serdio, Edwin Lughofer, Kurt Pichler,
Thomas Buchegger, Markus Pichler, Hajrudin Efendic
francisco.serdio@jku.at
Mathematical Systems
Johannes Kepler University
http://www.flll.jku.at/staff/francisco
Linz - Austria

2. Index
• IFAC Technical Committee SAFEPROCESS
» Fault detection
» Fault
• Residual Based Approach
» Recall Main Idea
» Graphical Explanation
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

3. Index
• Orthogonal transformations
» Principal Components Analysis (PCA)
» Mathematical Formulation
» Meaning
» How to use
» Partial Least Squares (PLS)
» Mathematical Formulation
» Meaning
» How to use
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

4. Index
• Vector Autoregressive Moving Average (VARMA)
» Motivation  ARMA
» Differences with ARMA
» How to use
• Soft Computing: Sparse Fuzzy Inference Systems
(SparseFIS)
• Overall picture: PCA/PLS + SparseFIS + VARMA
• Dynamic Residual Analysis
• Results as ROC curves
• Conclusions & Outlook
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

5. IFAC Technical Committee
SAFEPROCESS
• Fault detection
» Determination of faults present in a system and
the time of detection
• Fault
» Unpermitted deviation of at least one
characteristic property or variable of the system
from acceptable / usual / standard behaviour
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

6. Main Idea of Residual-Based Approach
Increasing the dimensionality of the joint channel space decreases the likelihood that a
fault is affected in all channels with same intensity and direction!
Fault No Fault!, but non-smooth
pattern of signal
Joint Channel Space
(smooth dependency)
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

7. Orthogonal Transformations
• Principal Components Analysis (PCA)
» Vector space transformation
» Identifies the most meaningful basis to re-express the
original space
» Preserves maximum variance in minimum number of
dimensions  filter out the noise / irrelevant information
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

8. Orthogonal Transformations
• Partial Least Squares (PLS)
» Also known as Projection to Latent Structures
» As PCA, also a vector space transformation
» Reduces the dimensionality of the input and target
variables by projecting them to the directions
maximizing the covariance between target and input
variables  filter out the noise / irrelevant information
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

9. Orthogonal Transformations
• What is the deal ?
» Apply the orthogonal transformation
» Train a model on top of the new (transformed)
space
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

10. Vector Autoregressive Moving Average
(VARMA)
• Motivation: Arma
» AutoRegressive Moving Average model
» Predicts a channel using its own history
» Autoregressive
» Own history means some (chosen) past values
» Lag operator, also known as Backshift operator
• Differences with Arma
» The lags belong to other channels
» Predicts a channel using other channels and other
channels’ history
» Vector Autoregressive
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

11. Vector Autoregressive Moving Average
(VARMA)
• What is the deal ?
» Span the dataset introducing lags
» Model over the spanned dataset including lags
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

12. Soft Computing: SparseFIS
• Sparse Fuzzy Inference System (SparseFIS)
» Top down fuzzy modeling approach applying numerical
sparsity constraints optimization, out-weighting
unimportant rules and parameters
» Employs iterative VQ, projected gradient descent and
Semi-Smooth Newton
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

13. Orthogonal Transformations, Soft Computing and
Vector Autoregressive Moving Average
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

14. Orthogonal Transformations, Soft Computing and
Vector Autoregressive Moving Average
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

15. Dynamic Residual Analysis (On-line)
Normalized Residual for
ith model: Confidence Band
Incremental/Decremental Tolerance Band
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

16. ROC curves: LR & SPF, with PCA & PLS
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

17. ROC curves: LR vs PCR vs PCR+Lags
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

18. Conclusions
• PCA before training
» The expansion of the datasets with the lags produces
no clear improvement in fault detection capabilities, and
the VARMA models can be ignored in this case
• PLS before training
» When the datasets are transformed using PLS, VARMA
models help to improve the ROC curves, and therefore
the fault detection capabilities
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

19. Outlook
• Deeper analysis of results
» Enforce the results with statistical tests
• Work in the Fault Identification and Fault
Isolation domain
» Create a confidence measure accompanying
the ROC curve
» Use the deformation of the model when a fault
appears
» Analyze the gradients of the inputs
» Compare gradients with detections
»Goal: Determine how true a detection is
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}

20. Thanks a lot for your attention!
{francisco.serdio,edwin.lughofer}@jku.at
http://www.flll.jku.at/staff/{Francisco Serdio, Dr. Edwin Lughofer francisco,lughofer}