Phishing Email Detection Using Subject and Body Text: A Comparative Study of Transformers, Bi-LSTM, and Traditional Classifiers

Abstract

Phishing emails continue to bypass blacklist, signature, and rule-based defenses, motivating the development of detectors that can learn robust patterns from message content. This Research-in-Progress paper reports completed results for the subject+body text module of a planned multi-segment phishing email detector covering headers, URLs, and attachments. We compare Transformer-based models (BERT, DistilBERT, DeBERTa-v3-base, DeBERTa-v3-small) with recurrent neural models (LSTM, Bi-LSTM) and traditional classifiers (Logistic Regression, SVM, Random Forest, Decision Tree, Naïve Bayes) under consistent preprocessing and evaluation settings. DeBERTa-v3-small achieves the strongest overall performance on the text module (F1 = 0.9972, Matthews correlation coefficient (MCC) = 0.9950), followed closely by BERT (F1 = 0.9969, MCC = 0.9944). DistilBERT provides a strong efficiency–effectiveness trade-off (F1 = 0.9957, MCC = 0.9922), and a linear SVM remains competitive among traditional classifiers (F1 = 0.9963, MCC = 0.9933). At batch size 1, the model-only inference time is 4.84 ms/message for DistilBERT, compared with 8.98 ms/message for BERT and 12.27 ms/message for DeBERTa-v3-small. These results provide a strong baseline for the text module, and future work will implement the remaining modules and measure their added value through controlled ablation and fusion studies.