Machine Learning-Based Anomaly Detection on a Labelled Cyber-Crime Forensic Dataset

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Objective: Digital traces are an increasingly common form of evidence, yet the manual review of large activity logs is slow and observer-dependent. This study evaluated whether conventional supervised classification algorithms can separate suspicious from normal user activity in a structured, labelled cyber-crime forensic dataset after the removal of every identified channel of label leakage.Methods: A publicly available, synthetically generated cyber-crime forensic dataset of 7,400 activity records with 11 attributes was used. The anomaly type field, which is recorded only for suspicious records and therefore encodes the outcome, was removed from the data frame. All 1,233 suspicious records and a simple random sample of 1,235 normal records, drawn without any completeness restriction, formed a near-balanced working dataset of 2,468 records. Missing values were imputed with constants computed on the pooled data without reference to the label. After label encoding, the data were split 80/20 and 27 classification algorithms were compared at their default settings; a logistic regression reference model was additionally assessed by five-fold stratified cross-validation, its confusion matrix, and the magnitudes of its coefficients.Results: None of the algorithms performed better than random guessing in any meaningful way. The highest test-set accuracy was 0.581 (perceptron; balanced accuracy 0.569; ROC AUC 0.562), against a chance baseline of 0.462 accuracy and 0.500 balanced accuracy; only five of the 26 algorithms exceeded a balanced accuracy of 0.500, and no ROC AUC exceeded 0.562. Cross-validation of the reference model gave a mean ROC AUC of 0.503 (SD 0.049). No predictor correlated with the label beyond an absolute value of 0.05. By contrast, the same pipeline had produced accuracies up to 0.862 in the balanced design, and perfect scores in 19 of 26 algorithms in an imbalanced diagnostic run, while the leakage channels were open.Conclusion: The classification performance of approximately 86% reported for this dataset was an artefact of label leakage introduced during preprocessing, not evidence of genuine separability. After remediation, the predictors carry no detectable information about the label. The study is reported as a documented case of how leakage can fabricate convincing forensic classifiers.