As one of the most common techniques to assess information system security, penetration testing legally attempts to break into the target system by utilizing tools and techniques similar to those used by real hackers. The main objective of such technique is to effectively call to light potential vulnerabilities existing in the system, and then come up with pragmatic solutions to address such weaknesses; thus, enhancing the security of the system as a whole.

Similar to every profession, penetration testing processes are efficiently aided by collections of automated tools. Nevertheless, due to the large number of tools available, penetration testing practitioners might encounter difficulties in choosing the most suitable tools for the task. As a result, this thesis firstly aims to provide the security community more reliable references regarding the effectiveness of penetration testing tools. Groups of service fingerprinting tools including Nmap, Dmitry, Unicornscan, and vulnerability scanning tools including Nessus, OpenVAS, and GFI Languard, were selected for performance evaluation. Results of the study suggest that Nmap and Nessus are more powerful than others owing to their quick response time and fair coverage.

In parallel, the research introduces an unorthodox use of attack tree model for post-attack analysis activities. Attacks demonstrated on the experimental system were gathered and organized into various attack tree diagrams. By analyzing the diagrams, most effective attack surfaces can be easily outlined. The outcomes of the research have confirmed that outdated operating systems and un-patched services might contain the most critical vulnerabilities that allow attackers to seize a system’s administrative access without spending too much time and effort. It is also pointed out that weak passwords and user’s gullibility can be taken advantage of to gain initial access to the system, followed by further malicious activities for privilege escalation.