Research Article
Measuring Accuracy of Automated Parsing and Categorization Tools and Processes in Digital Investigations
@INPROCEEDINGS{10.1007/978-3-319-14289-0_11, author={Joshua James and Alejandra Lopez-Fernandez and Pavel Gladyhsev}, title={Measuring Accuracy of Automated Parsing and Categorization Tools and Processes in Digital Investigations}, proceedings={Digital Forensics and Cyber Crime. Fifth International Conference, ICDF2C 2013, Moscow, Russia, September 26-27, 2013, Revised Selected Papers}, proceedings_a={ICDF2C}, year={2015}, month={2}, keywords={Digital forensic investigation Investigation accuracy Information retrieval Precision and recall Digital investigation measurement Digital investigation verification}, doi={10.1007/978-3-319-14289-0_11} }- Joshua James
Alejandra Lopez-Fernandez
Pavel Gladyhsev
Year: 2015
Measuring Accuracy of Automated Parsing and Categorization Tools and Processes in Digital Investigations
ICDF2C
Springer
DOI: 10.1007/978-3-319-14289-0_11
Abstract
This work presents a method for the measurement of the accuracy of evidential artifact extraction and categorization tasks in digital forensic investigations. Instead of focusing on the measurement of accuracy and errors in the functions of digital forensic tools, this work proposes the application of information retrieval measurement techniques that allow the incorporation of errors introduced by tools and analysis processes. This method uses a ‘gold standard’ that is the collection of evidential objects determined by a digital investigator from suspect data with an unknown ground truth. This work proposes that the accuracy of tools and investigation processes can be evaluated compared to the derived gold standard using common precision and recall values. Two example case studies are presented showing the measurement of the accuracy of automated analysis tools as compared to an in-depth analysis by an expert. It is shown that such measurement can allow investigators to determine changes in accuracy of their processes over time, and determine if such a change is caused by their tools or knowledge.