Can AI Detect Malicious Intent in an Email?
Phishing emails are becoming harder to detect, even for humans. A recent study tested various large language models (LLMs) for their ability to recognize malicious intent in emails, revealing significant differences in performance.
One standout was Claude 3.5 Sonnet, which scored over 90% at low false positive rates and even flagged suspicious emails that humans overlooked. When explicitly asked to assess suspicion, it correctly classified all phishing emails while avoiding false alarms on legitimate messages. However, it struggled with conventional phishing emails, achieving only an 81% true-positive rate in that category…