3.10 Anomaly Detection

3.10.1 Anomaly Detection Model Types

3.10.2 Training an Anomaly Detection Model

3.10.3 Using an Anomaly Detection Model

HeatWave AutoML includes anomaly detection, which is also known as outlier detection. Anomaly detection is the data mining task that finds unusual patterns in data. It is particularly useful for the following applications:

• Financial fraud detection.

• Network intrusion detection for cyber security.

• Detecting life-threatening medical conditions.

For anomaly detection, HeatWave AutoML uses Generalized kth Nearest Neighbors, GkNN, which is a model developed at Oracle. It is a single ensemble algorithm that outperforms state-of-the-art models on public benchmarks. It can identify common anomaly types, such as local, global, and clustered anomalies, and can achieve an AUC score that is similar to, or better than, when identifying the following:

• Global anomalies compared to KNN, with an optimal k hyperparameter value.

• Local anomalies compared to LOF, with an optimal k hyperparameter value.

• Clustered anomalies.

Optimal k hyperparameter values would be extremely difficult to set without labels and knowledge of the use-case.

Other algorithms would require training and comparing scores from at least three algorithms to address global and local anomalies, ignoring clustered anomalies: LOF for local, KNN for global, and another generic method to establish a 2/3 voting mechanism.

MySQL 8.4.0 introduces support for two additional models:

• Principal Component Analysis (PCA)

• The internally developed Generalized Local Outlier Factor (GLOF)