Paper: A Comparative Evaluation of Automated Vulnerability Scans Versus Manual Penetration Tests on False-negative Errors

Posted on November 18, 2018 in Cybersecurity

I am co-author of an academic paper, which is the result of the course "Independent R&D Studies", which was rewarded with 6 credit points. This is my first scientific publication.

The paper was presented at CYBER 2018 (The Third International Conference on Cyber-Technologies and Cyber-Systems) and published as a regular paper. You can download the paper via the free-access ThinkMind digital library. So, there is no need to use Sci-Hub this time :)


Security analysis can be done through different types of methods, which include manual penetration testing and automated vulnerability scans. These two different approaches are often confused and believed to result in the same value. To evaluate this, we have build a lab with several prepared vulnerabilities to simulate a typical small and medium-sized enterprise. Then, we performed a real penetration test on the lab, and a vulnerability scan as well, and then compared the results. Our conclusion shows, that the results obtained through both types of security analysis are highly distinct. They differ in time expenditure and false-positive rate. Most importantly, we have seen a remarkable higher false-negative rate in the vulnerability scan, which suggests that automated methods cannot replace manual penetration testing. However, the combination of both methods is a conceivable approach.