Behaviour Anomaly on Linux Systems to Detect Zero-day Malware Attacks

Ahmed, Ovais
Nisbet, Alastair
Item type
Degree name
Master of Information Security and Digital Forensics
Journal Title
Journal ISSN
Volume Title
Auckland University of Technology

Internet-connected devices have been the subject of cyber threats due to the gain malicious actors can get by compromising these systems. Endpoint protection is available on these systems, protecting if the malware signature is available for the malicious software. The challenge is that if the signature is not available on the endpoint protection, as in the case of zero-day malware, the endpoint will not detect or protect the system. The system follows the file analysis of zero-day malware in the sandbox environment for file identification, creating the signature and updating the endpoint database. The process of zero-day can generate a delay which can result in substantial damage to the systems by the time signature is updated. The research examines the abnormal behaviour on a Linux-based operating system and evaluates the method to explore the zero-day malware build for the platform. Malware samples are sourced from the available public repositories. The sample files used include known malicious and known non-malicious files. The known malicious files have the signatures available on the antivirus tool. Therefore, the setup removes the necessary signatures for the known malware sample files to treat them as zero-day malware. Total twenty-two malware has been used to test the method to detect the zero-day malware, out of which few have been tried without signature information on endpoint antivirus to determine the consistency of the test results. The research examines the malware behaviour on the Linux based system. It monitors the process in the two different situations where non-malicious and known malware is executed at different intervals. The abnormal process behaviour detects the malicious file. The second phase of the research explores the methods to act on the file after the detection. It discusses YARA rules and programable interface integration across the platform to automate the file quarantine feature.

Malware detection , Anomaly Behaviour , Linux System , Zero-day Malware
Publisher's version
Rights statement