【異常検知】Deep Learning for Anomaly Detection: A Survey を読んだ (後編)

Deep Learning for Anomaly Detection: A Surveyを読んだので備忘録を残しておきます。 前半は 深層異常検知 (Deep Anomaly Detection; DAD) のアーキテクチャの分類や長所・短所の紹介でした。後編は DAD の様々な領域への応用についてですが, 膨大な参考文献のため実質リンク集となっています。文献を徐々に読み進めて内容を追記する予定です。

Deep Anomaly Detection (DAD) の応用領域

• Intrusion Detection
• Fraud Detection
• Malware Detection
• Medical Anomaly Detection
• Deep learning for Anomaly detection in Social Networks
• Log Anomaly Detection
• Internet of things (IoT) Big Data Anomaly Detection
• Industrial Anomalies Detection
• Anomaly Detection in Time Series
• Video Surveillance

Intrusion Detection

侵入検知システム (The intrusion detection system; IDS) はコンピュータシステムにおける悪意のある活動を検知するシステムである。
IDS は単一のホストにインストールされる Host Intrusion Detection (HIDS) と大規模ネットワークを監視する Network Intrusion Detection (NIDS) に分類される。

Host-Based Intrusion Detection Systems (HIDS)

ホスト上のシステムコールのログはプログラムやユーザの操作によって生成される。マルウェア等による悪意のある操作は通常の操作とは異なるシーケンスでシステムコールが実行されることに繋がる。
HIDS は主にシグネチャベース (signature-based) と異常ベース (anomaly based) の2種類がある。異常ベースは見たことのない攻撃を検出できる可能性がある一方で誤検知が多くなるが, シグネチャベースと組み合わせることで強力な防衛手段となる。 HIDS に適用される DAD は可変長かつ系列データを処理することが必要となる。
Chawla et al. [2018] では, システムコールのデータセット ADFA-LD に対して, システムコールの入力系列を32次元の固定ベクトルに埋め込み, 系列を短くする前処理と局所特徴の抽出器として 1D-CNN を利用し GRU に渡すモデル CNN-GRU を提案し LSTM に比べ訓練時間を短くしながら SOTA の性能を維持したことを報告している。

• Lstm-based system-call language modeling and robust ensemble method for designing host-based intrusion detection systems (Kim et al. [2016])
• Host based intrusion detection system with combined cnn/rnn model (Chawla et al. [2018])
• Henet: A deep learning approach on intelR processor trace for effective exploit detection (Chen et al. [2018])
• Recurrent neural networks for enhancement of signaturebased network intrusion detection systems (Sohi et al. [2018])
• Applying convolutional neural network for network intrusion detection (Vinayakumar et al. [2017])
• Detecting deceptive reviews using generative adversarial networks (Aghakhani et al. [2018])
• Anomaly detection with generative adversarial networks for multivariate time series ( Li et al. [2018])
• An intrusion detection model based on deep belief networks (Gao et al. [2014])
• Probabilistic deep learning using random sum-product networks (Peharz et al.[2018])
• A two-stage flow-based intrusion detection model for next-generation networks (Umer et al. [2018])

Network Intrusion Detection Systems (NIDS)

ネットワークトラフィックは volume, velocity, variety (3V) の点でビッグデータである。リアルタイム検出が必要な点や, 攻撃者が IDS に適応することで攻撃の性質が変化し続ける点が難しい。深層学習は非線形の性質を持つネットワークトラフィックの異常検知に有効であるが, 1D-CNN は LSTM や VAE より良い性能が得られず, また 1D-CNN の層を深くしても性能改善には至らなかった。 (Kwon et al. [2018]) または [1]

• Network intrusion detection through stacking dilated convolutional autoencoders (Yu et al. [2017])
• Ieee 802.11 network anomaly detection and attack classification: A deep learning approach (Thing [2017])
• Increasing web service availability by detecting application-layer ddos attacks in encrypted traffic (Zolotukhin et al. [2016])
• Analyzing flow-based anomaly intrusion detection using replicator neural networks (Cordero et al. [2016])
• Toward an online anomaly intrusion detection system based on deep learning (Alrawashdeh and Purdy [2016])
• Deep learning approach for network intrusion detection in software defined networking (Tang et al. [2016])
• Conditional variational autoencoder for prediction and feature recovery applied to intrusion detection in iot (Lopez-Martin et al. [2017])
• Deep learning approach combining sparse autoen-coder with svm for network intrusion detection (Al-Qatf et al. [2018])
• Kitsune: an ensemble of autoencoders for online network intrusion detection (Mirsky et al. [2018])
• Network anomaly detection with stochastically improved autoencoder based models (Aygun and Yavuz [2017])
• Idsgan: Generative adversarial networks for attack generation against intrusion detection (Lin et al. [2018])
• An enhancing framework for botnet detection using generative adversarial networks (Yin et al. [2018])
• Flow-based network traffic generation using generative adversarial networks (Ring et al. [2018])
• When deep learning meets security (Latah [2018])
• Boosting anomaly detection using multi-discriminator generative adversarial networks (Intrator et al. [2018])
• Anomaly machine component detection by deep generative model with unregularized score (Matsubara et al. [2018])
• A hybrid autoencoder and density estimation model for anomaly detection (Nicolau et al. [2016])
• Adversarial deep learning against intrusion detection classifiers (Rigaki [2017])
• An empirical evaluation of deep learning for network anomaly detection (Malaiya et al. [2018])
• An empirical study on network anomaly detection using convolutional neural networks (Kwon et al. [2018])
• Applying long short-term memory recurrent neural networks to intrusion detection (Staudemeyer [2015])
• Enhanced network anomaly detection based on deep neural networks (Naseer et al. [2018])

情報セキュリティの分野では, ラベル付きデータの欠如や絶えず進化し続ける攻撃, セキュリティアナリストが攻撃の調査に使える時間には限界があるという課題がある。Veeramachaneni et al. [2016] は AI squared というアナリストのフィードバックを取り入れたセキュリティシステムを提案している。AI squared では, まず教師なし学習ベースの外れ値検出 (行列分解ベース, 密度ベース, Replicator Neural Network (i.e. AE)) のアンサンブルから異常スコアに基づいたランク付けを行う。次に, この結果の上位のイベントに対してアナリストがフィードバックを行い, そのラベルを用いて教師ありモデルを構築し教師なしモデルと組み合わせ異常検知を行う。この手順を継続的に繰り返すことで SOTA の教師なし異常検知モデルに対して FP を 1/5 に減らすことができたと報告している。

Fraud Detection

Fraud Detection (不正検知) は様々な産業における違法な行為の検出を指す。

Banking fraud

ここ数年, クレジットカード不正検知のための技術が複数発表されている (Zhou et al. [2018], Suganya and Kamalraj [2015])。課題は不正に一貫したパターンがないことである。一般的なアプローチでは各ユーザのプロファイルや利用状況を監視するが数十億のユーザがいるため, このアプローチはスケールが難しい。従って DAD が広く使われている。

• Detection of anomalies in large scale accounting data using deep autoencoder networks (Schreyer et al. [2017])
• Solving the false positives problem in fraud prediction (Wedge et al. [2017])
• Deep learning anomaly detection as support fraud investigation in brazilian exports and anti-money laundering (Paula et al. [2016])
• Fraud detection on unlabeled data with unsupervised machine learning (Renstrom and Holmsten [2018])
• Using deep networks for fraud detection in the credit card transactions (Kazemi and Zarrabi [2017])
• One-class adversarial nets for fraud detection (Zheng et al. [2018a])
• Credit card fraud detection using deep learning based on auto-encoder and restricted boltzmann machine (Pumsirirat and Yan [2018])
• Fraudminer: A novel credit card fraud detection model based on frequent itemset mining (Seeja and Zareapoor [2014])
• Autoencoding credit card fraud (Sweers et al. [2018])
• Using generative adversarial networks for improving classification effectiveness in credit card fraud detection (Fiore et al. [2017])
• Generative ensembles for robust anomaly detection (Choi and Jang [2018])
• Neural fraud detection in credit card operations (Dorronsoro et al. [1997])
• End-to-end neural network architecture for fraud scoring in card payments (Gomez et al. [2018])
• Credit card transactions, fraud detection, and machine learning: Modelling time with lstm recurrent neural networks (Wiese and Omlin [2009])
• Sequence classification for credit-card fraud detection (Jurgovsky et al. [2018])
• Learning temporal representation of transaction amount for fraudulent transaction recognition using cnn (Heryadi and Warnars [2017])
• Detecting fraudulent behavior using recurrent neural networks (Ando et al. [2016])
• Session-based fraud detection in online ecommerce transactions using recurrent neural networks (Wang et al. [2017a])
• Credit card fraud detection: Personalized or aggregated model (Alowais and Soon [2012])
• Critical analysis of machine learning based approaches for fraud detection in financial transactions (Amarasinghe et al. [2018a])
• Transaction fraud detection using gru-centered sandwich-structured model (Lp et al. [2018])
• Application of classification models on credit card fraud detection (Shen et al. [2007])
• Convnets for fraud detection analysis (Chouiekh and Haj [2018])
• Convolutional and recurrent neural networks for real-time data classification (Abroyan [2017b])
• Credit card fraud detection using convolutional neural networks (Fu et al. [2016])
• Deep neural networks and fraud detection (Lu [2017])
• Credit card fraud detection based on whale algorithm optimized bp neural network (Wang et al. [2018a])
• Neural networks for financial market risk classification (Abroyan [2017a])
• A model based on convolutional neural network for online transaction fraud detection (Zhang et al. [2018a])

Mobile cellular network fraud

音声やメッセージによる振り込め詐欺などのモバイルネットワークの不正検知は, ネットワークトラフィックの量と速度のため簡単ではなく, 従来の特徴エンジニアリングでは日々進化する不正に適応することが難しい。

• Mobile big data analytics using deep learning and apache spark (Alsheikh et al. [2016])
• Click fraud detection in mobile ads served in programmatic inventory (Badhe [2017])
• Detecting telecommunication fraud using neural networks through data mining (Akhter and Ahamad [2012])
• Perspective analysis of telecommunication fraud detection using data stream analytics and neural network classification based data mining (Jain [2017])
• Generative adversarial network based telecom fraud detection at the receiving bank (Zheng et al. [2018b])

Insurance fraud

保険詐欺 (保険金の偽装請求) は従来の機械学習手法を適用したアプローチである程度成功している。課題は専門家によるロバストな特徴抽出を行う必要性がある点や, 全体の保険金請求に対して詐欺が占める割合が非常に低い点である。

• Auto claim fraud detection using bayesian learning neural networks (Viaene et al. [2005])
• Vos: a method for variational oversampling of imbalanced data (Fajardo et al. [2018])
• Using generative adversarial networks for improving classification effectiveness in credit card fraud detection (Fiore et al. [2017])
• Neural networks for insurance fraud detection (Keung et al. [2009])
• Application of classification models on credit card fraud detection (Shen et al. [2007])
• A model based on convolutional neural network for online transaction fraud detection (Zhang et al. [2018a])

Healthcare fraud

医療詐欺, 特に医療保険請求詐欺 (療養費詐欺) は医療費の増加に大きく影響している。 詐欺を検出することでこの影響を軽減できるため機械学習モデルが使われてきた（Bauder and Khoshgoftaar [2017]）

• Deep learning to detect medical treatment fraud (Lasaga and Santhana [2018])
• Semi-supervised generative adversarial network for gene expression inference (Ghasedi Dizaji et al. [2018])
• Adversarial attacks against medical deep learning systems (Finlayson et al. [2018])
• Dermatologist-level classification of skin cancer with deep neural networks (Esteva et al. [2017])

Malware Detection

従来のマルウェア検出 (Malicious Software Detection) は特徴抽出とクラスタリング/分類の二段階に構成される。データが膨大で攻撃者の適応性が高く高度なテクニックでマルウェアの振る舞いを隠そうとする点が課題である。
マルウェアはデータセンター内で急速に拡散するため IaaS の提供においてマルウェア検知は重要な課題である。 機械学習によるマルウェア検知では, マルウェアがリモートの攻撃者から指示があるまでアイドル状態で待機し訓練段階では活動を開始しないミスラベリング問題 (mislabeling problem) が課題となっている。Abdelsalam et al. [2018]) では RNN よりシンプルで訓練が高速なことから CNN を選択し, PIDとプロセスごとの性能メトリクス に時間 (time-window で移動) を加えた 3D-CNN による二値分類で 90% の accuracy を達成したと報告している。

• Autoencoder-based feature learning for cyber security applications (Yousefi-Azar et al. [2017])
• Dl4md: A deep learning framework for intelligent malware detection (Hardy et al. [2016])
• An investigation of a deep learning based malware detection system (Sewak et al. [2018])
• Classification of malware programs using autoencoders based deep learning architecture and its application to the microsoft malware classification challenge (big 2015) dataset (Kebede et al. [2017])
• Malware detection through low-level features and stacked denoising autoencoders (De Paola et al. [2018])
• Improving malware detection accuracy by extracting icon information (Silva et al. [2018])
• Adversarial malware binaries: Evading deep learning for malware detection in executables (Kolosnjaji et al. [2018])
• Exploring adversarial examples in malware detection (Suciu et al. [2018])
• Muldef: Multi-model-based defense against adversarial examples for neural networks (Srisakaokul et al. [2018])
• Artificial intelligence crime: An interdisciplinary analysis of foreseeable threats and solutions (King et al. [2018])
• R2-d2: color-inspired convolutional neural network (cnn)-based android
  malware detections (Huang and Kao [2017])
• Malware detection with convolutional neural network using hardware events (Guo et al. [2017])
• Malware detection in cloud infrastructures using convolutional neural networks (Abdelsalam et al. [2018])
• Malware detection by eating a whole exe (Raff et al. [2017])
• Maldozer: Automatic framework for android malware detection using deep learning. Digital Investigation (Karbab et al. [2018])
• Evaluating convolutional neural network for effective mobile malware detection (Martinelli et al. [2017])
• Deep android malware detection (McLaughlin et al. [2017])
• Using convolutional neural networks for classification of malware represented as images (Gibert et al. [2018])
• Empowering convolutional networks for malware classification and analysis (Kolosnjaji et al. [2017])
• End-to-end deep neural networks and transfer learning for automatic analysis of nation-state malware (Rosenberg et al. [2018])
• Deepsign: Deep learning for automatic malware signature generation and classification (David and Netanyahu [2015])
• Application of deep belief networks for android malware detection (YANG et al. [2016])
• Application of deep belief networks for opcode based malware detection (Ding et al. [2016])
• Deep belief network based detection and categorization of malicious urls (Selvaganapathy et al. [2018])
• Malware detection based on deep learning algorithm (Yuxin and Siyi [2017])
• Deep neural networks for automatic android malware detection (Hou et al. [2017])
• Malware detection with deep neural network using process behavior (Tobiyama et al. [2016])
• Black-box attacks against rnn based malware detection algorithms (Hu and Tan [2017])
• A method for estimating process maliciousness with seq2seq model (Tobiyama et al. [2018])
• Long-term temporal averaging for stochastic optimization of deep neural networks (Passalis and Tefas)
• Deep learning at the shallow end: Malware classification for non-domain experts (Le et al. [2018])
• Adversary resistant deep neural networks with an application to malware detection (Wang et al. [2017b])
• Zero-day malware detection using transferred generative adversarial networks based on deep autoencoders (Kim et al. [2018])
• Effective android malware detection with a hybrid model based on deep autoencoder and convolutional neural network (Wang et al. [2018b])
• A hybrid malicious code detection method based on deep learning. methods (Li et al. [2015])
• A deep recurrent neural network based approach for internet of things malware threat hunting (HaddadPajouh et al. [2018])

Medical Anomaly Detection

医学および生命情報科学 (bio-informatics) における深層学習の理論的および実用のためにいくつかの研究がある。(Min et al. [2017], Cao et al. [2018a], Zhao et al. [2016], Khan and Yairi [2018]).   医療分野の異常検出において不均衡データは大きな課題である。深層学習はブラックボックスの技術として考えられてきたが, 最近では優れた解釈性を持つモデルが提案され SOTA を達成することが示されている。 (Gugulothu et al., Amarasinghe et al. [2018b], Choi [2018])
Amarasinghe et al. [2018b] は DNN の予測に対して入力特徴の関連性スコア (relevance scores), 推定値の信頼度, テキストによる説明 (事前に設定された異常タイプの説明) を提示する explainable DNN based anomaly detection を提案した。NSL-KDD データセットで実験を行い, 関連性スコアを伴った上で accuracy 97% を達成している。人間が意思決定に関わるミッションクリティカルなシステムの監視において DNN ベースの異常検知は高い予測性能の一方で, 不透明で非線形な構造を持つことが導入の制限となる場合があるが, これにより DNN の予測結果を人間が直感的に評価する際の手助けとなる。

• Research on healthy anomaly detection model based on deep learning from multiple time-series physiological signals (Wang et al. [2016])
• A primitive study on unsupervised anomaly detection with an autoencoder in emergency head ct volumes (Sato et al. [2018])
• Deep belief networks used on high resolution multichannel electroencephalography data for seizure detection (Turner et al. [2014])
• Abnormality detecting deep belief network (Sharma et al. [2016])
• Semi-supervised anomaly detection for eeg waveforms using deep belief nets (Wulsin et al. [2010])
• An unsupervised deep hyperspectral anomaly detector (Ma et al. [2018])
• Automatic sleep stage classification based on sparse deep belief net and combination of multiple classifiers (Zhang et al. [2016])
• Modeling electroencephalography waveforms with semisupervised deep belief nets (Wulsin et al. [2011])
• Adaptive anomalies detection with deep network (Wu et al. [2015a])
• Enhanced restricted boltzmann machine with prognosability regularization for prognostics and health assessment (Liao et al. [2016])
• Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications (Xu et al. [2018])
• Anomaly detection for skin disease images using variational autoencoder (Lu and Xu [2018])
• Unsupervised detection of lesions in brain mri using constrained adversarial auto-encoders (Chen and Konukoglu [2018])
• Toward sustainable virtualized healthcare: Extracting medical entities from chinese online health consultations using deep neural networks (Yang and Gao [2018])
• Bidirectional rnn for medical event detection in electronic health records (Jagannatha and Yu [2016])
• A combined deep learning gru-autoencoder for the early detection of respiratory disease in pigs using multiple environmental sensors (Cowton et al. [2018])
• Recurrent neural radio anomaly detection (O’Shea et al. [2016])
• Phonocardiographic sensing using deep learning for abnormal heartbeat detection (Latif et al. [2018])
• Time series prediction and anomaly detection of light curve using lstm neural network (Zhang and Zou [2018])
• Anomaly detection in ecg time signals via deep long short-term memory networks (Chauhan and Vig [2015])
• Sparse neural networks for anomaly detection in high-dimensional time series (Gugulothu et al.)
• Toward explainable deep neural network based anomaly detection (Amarasinghe et al. [2018b])
• Aricial intelligence in retina (Schmidt-Erfurth et al. [2018])
• Detecting and locating gastrointestinal anomalies using deep learning and iterative cluster unification (Iakovidis et al. [2018])
• A hybrid semi-supervised anomaly detection model for high-dimensional data (Song et al. [2017])

Deep learning for Anomaly detection in Social Networks

SNS 内の異常は, スパマーや偽のユーザなどによる不規則な振る舞いパターンとして現れる。ソーシャルネットワークにおける異常検知と課題のサーベイ論文は複数ある。(Liu and Chawla [2017], Savage et al. [2014], Anand et al. [2017], Yu et al. [2016], Cao et al. [2018b])
Castellini et al. [2017] は Twitter 上で経済や政治, 社会への影響力を高める目的で作成された偽のフォロワー (Fake followers) の検出に denoising autoencoder を用いる手法を提案した。本物のユーザのプロファイルを特徴量として denoising autoencoder を訓練し再構成誤差の大きいユーザを検出することで, 教師ありモデルよりも偽のフォロワーの変化に対してロバストであると報告している。
知識ベース (e.g. Wikipedia) などのオンラインプラットフォームではスパムや捏造などの荒らし行為を行うユーザが問題となっている。Wikipedia では ClueBot NG という荒らし行為を検出するツールが使われている。Zheng et al. [2018a] は荒らし行為を検出する one-class adversarial nets (OCAN) を提案した。OCAN は最初に LSTM-AE で良性ユーザ (benign users) のオンライン活動を潜在空間にエンコードしユーザ表現とする。次に, ユーザ表現を complementary GAN の識別器に渡す。識別器は通常の GAN の目的関数を修正し, 良性ユーザと悪意のあるユーザを表す complementary samples を分離するように訓練される。

OCAN は one-class 手法のため良性ユーザのみから訓練でき, LSTM-AE がユーザーの活動を逐次的に捉えることで早期検出を可能にする。実験では, 良性ユーザと悪意のあるユーザの両方を必要とする M-LSTM と比較し競争力のある F1-Score を示した。
データの不均質で動的な性質は DAD にとって課題であるが, 以下の手法は SOTA となることが示されている。

• Detecting rumors on online social networks using multi-layer autoencoder (Zhang et al. [2017])
• Fake twitter followers detection by denoising autoencoder (Castellini et al. [2017])
• Detecting anomalous emotion through big data from social networks based on a deep learning method (Sun et al. [2018])
• Doc: Deep open classification of text documents (Shu et al. [2017])
• Anomaly detection in moving crowds through spatiotemporal autoencoding and additional attention (Yang et al. [2018])
• Detecting event-related changes in organizational networks using optimized neural network models (Li et al. [2017a])

Log Anomaly Detection

ログファイルの異常検知はシステム障害の原因を示すテキストを見つけることが目的である。過去の経験から構築されるドメイン特有の正規表現 (regular-expression) のパターンマッチングを用いて障害を見つけるアプローチでは新しい障害の検出が難しい。(Memon [2008])
ログデータのフォーマットとセマンティクスの両方の非構造化と多様性は異常検知の大きな課題であり, さらにリアルタイム検出が求められる。
以下の DAD はログデータを系列として扱い異常検出に有効であることが示されている。

• Long short-term memory (Hochreiter and Schmidhuber [1997])
• Recurrent neural network attention mechanisms for interpretable system log anomaly detection (Brown et al. [2018])
• Deep learning for unsupervised insider threat detection in structured cybersecurity data streams (Tuor et al. [2017])
• Desh: deep learning for system health prediction of lead times to failure in hpc (Das et al. [2018])
• Long short term memory networks for anomaly detection in time series (Malhotra et al. [2015])
• Deeplog: Anomaly detection and diagnosis from system logs through deep learning (Du et al. [2017])
• Detecting anomalous data using auto-encoders (Andrews et al. [2016a])
• Anomaly detection using autoencoders with nonlinear dimensionality reduction (Sakurada and Yairi [2014])
• Analyzing Business Process Anomalies Using Autoencoders (Nolle et al. [2018a])
• Anomaly detection for application log data (Grover [2018])
• Anomaly detection in unstructured time series datausing an lstm autoencoder (Wolpher [2018])
• Role-based log analysis applying deep learning for insider threat detection (Zhang et al. [2018b])
• Anomaly detection in aircraft data using recurrent neural networks (Nanduri and Sherry [2016])
• Anomaly detection in smart grid based on encoder-decoder framework with recurrent neural network (Fengming et al. [2017])
• Non-linear prediction with lstm recurrent neural networks for acoustic novelty detection (Marchi et al. [2015])
• Unsupervised anomaly detection in noisy business process event logs using denoising autoencoders (Nolle et al. [2016])
• Detecting anomaly in big data system logs using convolutional neural network (Lu et al. [2018])
• Insider threat detection with deep neural network (Yuan et al. [2018a])
• A compact convolutional neural network for textured surface anomaly detection (Racki et al. [2018])
• Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes (Zhou et al. [2016])
• Convolutional neural networks for unsupervised anomaly detection in text data (Gorokhov et al. [2017])
• Deep convolutional player modeling on log and level data (Liao et al. [2017])
• Deep convolutional neural networks for anomaly event classification on distributed systems (Cheng et al. [2017])
• Alphamex: A smarter global pooling method for convolutional neural networks (Zhang et al. [2018c])

Internet of things (IoT) Big Data Anomaly Detection

IoT は相互接続されたデバイスによるネットワークであり, データは時系列データである。IoT ネットワークでの異常検知の課題は異種のデバイスが相互接続されているためにシステムが複雑になることである。 IoT における深層学習の利用は、Mohammadi et al. [2018] によって提示された。

• Distributed anomaly detection using autoencoder neural networks in wsn for iot (Luo and Nagarajany [2018])
• Neural network cognitive engine for autonomous and distributed underlay dynamic spectrum access (Mohammadi and Kwasinski [2018])
• In Proceedings of the 18th International Conference on Computer Systems and Technologies (Kakanakova and Stoyanov [2017])
• Lstm-based analysis of industrial iot equipment (Zhang et al. [2018d])
• An unsupervised anomalous event detection framework with class aware source separation (Mudassar et al. [2018])

Industrial Anomalies Detection

産業システムを構成する部品の故障・劣化・損傷を検知するために機械学習が使用されている。 (Ramotsoela et al. [2018], Mart´ı et al. [2015]) 
故障検知は異常検知の問題として定式化できる。課題はデータの動的な性質と量の双方の様々な要因によって異常が引き起こされる点である。  

• Anomaly detection for a water treatment system using unsupervised machine learning (Inoue et al. [2017])
• One-class collective anomaly detection based on lstm-rnns (Thi et al. [2017])
• Detecting cyber attacks in industrial control systems using convolutional neural networks (Kravchik and Shabtai [2018])
• Lired: A light-weight real-time fault detection system for edge computing using lstm recurrent neural networks (Park et al. [2018a])
• A distributed anomaly detection method of operation energy consumption using smart meter data (Yuan and Jia [2015])
• An ensemble learning framework for anomaly detection in building energy consumption (Araya et al. [2017])
• Detection of pitting in gears using a deep sparse autoencoder (Qu et al. [2017])
• Detecting anomalous faces with no peeking autoencoders (Bhattad et al. [2018])
• Power profiling of microcontroller’s instruction set for runtime hardware trojans detection without golden circuit models (Lodhi et al. [2017])
• Deep convolutional neural networks for detection of rail surface defects (Faghih-Roohi et al. [2016])
• Deepanomaly: Combining background subtraction and deep learning for detecting obstacles and anomalies in an agricultural field. Sensors (Christiansen et al. [2016])
• Convolutional neural net and bearing fault analysis (Lee et al. [2016])
• Camera anomaly detection based on morphological analysis and deep learning (Dong et al. [2016])
• A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition (Fuentes et al. [2017])
• A deep learning-based method for sleep stage classification using physiological signal (Huang et al. [2018])
• A review of artificial intelligence algorithms used for smart machine tools (Chang et al. [2018])
• On accurate and reliable anomaly detection for gas turbine combustors: A deep learning approach (Yan and Yu [2015])
• Gas turbine engine gas path anomaly detection using deep learning with gaussian distribution (Luo and Zhong [2017])
• Cleaning method for status monitoring data of power equipment based on stacked denoising autoencoders (Dai et al. [2017])
• Neural network-based data-driven modelling of anomaly detection in thermal power plant (Banjanovic-Mehmedovic et al. [2017])

Anomaly Detection in Time Series

時系列データは単変量と多変量時系列に分類することができる。単変量時系列は単一の変数が時間とともに変化する (e.g. 室内の温度センサ) 。多変量時系列は複数の変数で構成される (e.g. 3軸加速度センサ)。
時系列データにおける異常検知の課題は以下である。

• ノイズがアルゴリズムの性能に与える影響
• 時系列データの長さと共に増加する計算量
• 時系列データの非定常性, 非線形性, 動的な変化への対処
  

時系列データの様々な異常検知のツール及びデータセットは rob-med/awesome-TS-anomaly-detection で管理されている。

深層学習は豊富な階層的な特徴を抽出できるため, 単変量時系列データの異常検知性能を大幅に改善できる。一方, 多変量時系列データの異常検知は難しいタスクである。Buda et al. [2018] により多変量時系列データの深層学習に基づく一般的なフレームワーク DeepAD が提案されている。RNN/LSTM ベースの手法 (pnnl/safekit) は多変量時系列データセット内の解釈可能な異常を検出できることが示されている。注意機構 (attention mechanism) を用いた異常検知システムは検出された異常の説明に効果的である。(Yuan et al. [2018b], Guo and Lin [2018])

単変量時系列に対する DAD は以下がある。

• Time series anomaly detection; detection of anomalous drops with limited features and sparse examples in noisy highly periodic data (Shipmon et al. [2017a])
• Deep and confident prediction for time series at uber (Zhu and Laptev [2017])
• Unsupervised real-time anomaly detection for streaming data (Ahmad et al. [2017])
• Lstm-based encoder-decoder for multi-sensor anomaly detection (Malhotra et al. [2016a])
• Collective anomaly detection based on long short-term memory recurrent neural networks (Bontemps et al. [2016])
• Anomaly detection in automobile control network data with long short-term memory networks (Taylor et al. [2016])
• Ms-lstm: A multi-scale lstm model for bgp anomaly detection (Cheng et al. [2016])
• Sequence to sequence pattern learning algorithm for real-time anomaly detection in network traffic (Loganathan et al. [2018])
• Anomaly detection in ecg time signals via deep long short-term memory networks (Chauhan and Vig [2015])
• Deepant: A deep learning approach for unsupervised anomaly detection in time series (Munir et al. [2018])
• Time series anomaly detection: Detection of anomalous drops with limited features and sparse examples in noisy periodic data (Shipmon et al. [2017b])
• Multi-sensor prognostics using an unsupervised health index based on lstm encoder-decoder (Malhotra et al. [2016b])
• Multivariate industrial time series with cyber-attack simulation: Fault detection using an lstm-based predictive data model (Filonov et al. [2016])
• Deep learning-based detection of periodic abnormal waves in ecg data (Sugimoto et al. [2018])
• Residual error based anomaly detection using auto-encoder in smd machine sound (Oh and Yun [2018])
• Multi-scale change point detection in multivariate time series (Ebrahimzadeh and Kleinberg)
• AI squared: training a big data machine to defend (Veeramachaneni et al. [2016])
• Anomaly detection using replicator neural networks trained on examples of one class (Dau et al. [2014])
• Recurrent neural networks for anomaly detection in the post-mortem time series of lhc superconducting magnets (Wielgosz et al. [2017])
• Online anomaly detection with concept drift adaptation using recurrent neural networks (Saurav et al. [2018])
• The model of an anomaly detector for hilumi lhc magnets based on recurrent neural networks and adaptive quantization (Wielgosz et al. [2018])
• Robust online time series prediction with recurrent neural networks (Guo et al. [2016])
• Rnn-based early cyber-attack detection for the tennessee eastman process (Filonov et al. [2017])
• Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and hilbert transform (Kanarachos et al. [2017])
• Modeling approaches for time series forecasting and anomaly detection (Du et al.)
• Anomaly detection in nanofibrous materials by cnn-based self-similarity (Napoletano et al. [2018])
• Anomaly detection in video using predictive convolutional long short-term memory networks (Medel and Savakis [2016])
• Variational inference for on-line anomaly detection in high-dimensional time series (Solch et al. [2016])
• Efficient gan-based anomaly detection (Zenati et al. [2018])
• Doping: Generative data augmentation for unsupervised anomaly detection with gan (Lim et al. [2018])
• Unsupervised anomaly detection for traffic surveillance based on background modeling (Laptev,Wei et al. [2018])

多変量時系列に対する DAD は以下がある。

• Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding (Hundman et al. [2018])
• Deep learning for anomaly detectionin multivariate time series data (Assendorp et al.[2017])
• Binet: Multivariate business process anomaly detection using deep learning (Nolle et al. [2018b])
• A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data (Zhang et al. [2018e])
• Multidimensional time series anomaly detection: A gru-based gaussian mixture variational autoencoder approach (Guo et al. [2018])
• Aircraft engine fault detection based on grouped convolutional denoising autoencoders (Fu et al. [2019])
• Outlier detection for multidimensional time series using deep neural networks (Kieu et al. [2018])
• Packet-data anomaly detection in pmu-based state estimator using convolutional neural network (Basumallik et al. [2019])
• Jiffy: A convolutional approach to learning time series similarity (Shanmugam et al. [2018])
• Estimation of dimensions contributing to detected anomalies with variational autoencoders (Ikeda et al. [2018])
• A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder (Park et al. [2018b])
• Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks (Li et al. [2019])
• A combined deep learning gru-autoencoder for the early detection of respiratory disease in pigs using multiple environmental sensors (Cowton et al. [2018])
• Recurrent neural network language models for open vocabulary event-level cyber anomaly detection (Tuor et al. [2018])

Video Surveillance

映像監視システムではラベルのないデータが大量にある点や異常の明確な定義がない点が, 教師付き機械学習や深層学習にとって課題となる。Kiran et al. [2018] や Chong and Tay [2015] は映像の異常検知のための最先端の深層モデルを研究し, モデルのタイプ と検出基準に基づいて分類を行った。

• Anomaly detection for security imaging (Andrewsa et al.)
• Fully convolutional neural network for fast anomaly detection in crowded scenes (Sabokrou et al. [2016a])
• Deep-cascade: cascading 3d deep neural networks for fast anomaly detection and localization in crowded scenes (Sabokrou et al. [2017])
• Transferred deep learning for anomaly detection in hyperspectral imagery (Li et al. [2017b])
• Convolutional neural networks for crowd behaviour analysis: a survey (Tripathi et al. [2018])
• Deepfall–non-invasive fall detection with deep spatio-temporal convolutional autoencoders (Nogas et al. [2018])
• Abnormal event detection in videos using spatiotemporal autoencoder (Chong and Tay [2017])
• Improved anomaly detection in surveillance videos based on a deep learning method (Khaleghi and Moin [2018])
• Abnormal event detection based on deep autoencoder fusing optical flow (Qiao et al. [2017])
• Detection of video anomalies using convolutional autoencoders and one-class support vector machines (Gutoski et al.)
• Detecting abnormal behaviors in surveillance videos based on fuzzy clustering and multiple auto-encoders (Chen et al. [2015])
• Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder (Sabokrou et al. [2016b])
• Anomaly detection using a convolutional winner-take-all autoencoder (Tran and Hogg [2017])
• Learning temporal regularity in video sequences (Hasan et al. [2016])
• Unsupervised extraction of video highlights via robust recurrent auto-encoders (Yang et al. [2015])
• ANOMALY DETECTION IN SURVEILLANCE VIDEOS USING DEEP RESIDUAL NETWORKS (Cinelli [2017])
• Real-world anomaly detection in surveillance videos (Sultani et al. [2018])
• Unsupervised discovery of normal and abnormal activity patterns in indoor and outdoor environments (Dotti et al. [2017])
• Autoencoder with recurrent neural networks for video forgery detection (D’Avino et al. [2017])
• Unsupervised change detection using spatial transformer networks (Chianucci and Savakis [2016])
• Spatio-temporal anomaly detection for industrial robots through prediction in unsupervised feature space (Munawar et al. [2017])
• Remembering history with convolutional lstm for anomaly detection (Luo et al. [2017a])
• Attentioned convolutional lstm inpaintingnetwork for anomaly detection in videos (Ben-Ari and Shwartz-Ziv [2018])
• Anomaly detection for temporal data using long short-term memory (lstm) (Singh [2017])
• A revisit of sparse coding based anomaly detection in stacked rnn framework (Luo et al. [2017b])
• Abnormal event detection using recurrent neural network (Zhou and Zhang [2015])
• Video anomaly detection using deep incremental slow feature analysis network (Hu et al. [2016])
• Modeling representation of videos for anomaly detection using deep learning: A review (Chong and Tay [2015])
• Training adversarial discriminators for crosschannel abnormal event detection in crowds (Ravanbakhsh et al. [2017a])

[1] An Empirical Study on Network Anomaly Detection using Convolutional Neural Networks July 2, 2018 (Presented at SNTA 2018)