Scaling the kmax criterion in the DDCAM methodology

Resumo

Outlier detection in multivariate data is a critical challenge with applications in fields such as finance, medicine, and industrial process monitoring. This study investigates the Data-Driven Cluster Analysis Method, designed to improve the identification of atypical observations through adaptive clustering strategies. Specifically, the research examines the role of the stopping criterion kmax —– the maximum number of clusters considered —– in determining the method’s efficiency and accuracy. Using Monte Carlo simulations with contaminated normal, exponential, and point mass distributions, the study evaluates whether excessively large kmax values contribute meaningfully to model performance or merely increase computational cost. Results demonstrate that the optimal number of clusters, selected via the Bayesian Information Criterion (BIC), consistently falls well below the imposed kmax threshold, regardless of dimensionality, or contamination level. Furthermore, as sample size increases, the gap between the selected k and the kmax limit widens, while runtime grows proportionally. These findings suggest that overly conservative settings for kmax are unnecessary and can be replaced by more parsimonious values without compromising detection accuracy. The study reinforces DDCAM’s robustness and stability while highlighting opportunities for computational optimization.