Small Effect Sizes in Malware Detection? Make Harder Train/Test Splits!
원문 초록
Industry practitioners care about small improvements in malware detection accuracy because their models are deployed to hundreds of millions of machines, meaning a 0.1\% change can cause an overwhelming number of false positives. However, academic research is often restrained to public datasets on the order of ten thousand samples and is too small to detect improvements that may be relevant to industry. Working within these constraints, we devise an approach to generate a benchmark of configurable difficulty from a pool of available samples. This is done by leveraging malware family information from tools like AVClass to construct training/test splits that have different generalization rates, as measured by a secondary model. Our experiments will demonstrate that using a less accurate secondary model with disparate features is effective at producing benchmarks for a more sophisticated target model that is under evaluation. We also ablate against alternative designs to show the need for our approach.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
EMBER2024 -- A Benchmark Dataset for Holistic Evaluation of Malware Classifiers
LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis
Evaluating the Robustness of Adversarial Defenses in Malware Detection Systems
Cyber Security Data Science: Machine Learning Methods and their Performance on Imbalanced Datasets
Imbalanced malware classification: an approach based on dynamic classifier selection
CyberLLMInstruct: A New Dataset for Analysing Safety of Fine-Tuned LLMs Using Cyber Security Data
토론
이 논문과 연결된 포럼 글이 아직 없습니다.