Every vendor pitch deck now has a slide about “AI-powered threats.” Strip away the marketing, and there’s a real, specific shift in attacker tradecraft worth understanding — and a real, specific set of defensive changes it calls for.
LLMs haven’t given attackers new categories of attack — phishing, malware, social…
Phishing and business email compromise — AI-generated messages are harder for both humans and…
Signature and pattern-based detection alone is no longer sufficient against content generated…
LLMs haven’t given attackers new categories of attack — phishing, malware, social engineering and reconnaissance are all decades old. What’s changed is cost and scale. Generating a convincing, grammatically fluent, context-aware phishing email in a target’s own language and tone used to take a skilled human minutes; generating a thousand variants, each personalized from scraped LinkedIn and company data, now takes an automated pipeline seconds. The skill floor for convincing social engineering has dropped sharply.
Signature and pattern-based detection alone is no longer sufficient against content generated specifically to evade it. The practical shift is toward behavioral detection (what is this email/process/login actually trying to do, regardless of how it’s worded or packaged), stronger out-of-band verification for high-risk actions (a phone call to a known number before wiring funds, not a reply to the email that requested it), and security awareness training that specifically covers AI-generated phishing’s tells — which are different from the broken-English tells security training has taught for twenty years.
The same capability curve applies defensively — AI-assisted log analysis, anomaly detection, and automated triage of security alerts reduce the time between an intrusion and its detection. The organizations keeping pace aren’t the ones avoiding AI; they’re the ones applying it on both sides of the equation, as part of a continuously tested security posture rather than a one-time control.
Increasingly, yes — because fluency and personalization, which filters previously used as positive signals, are no longer reliable tells. Filters are adapting toward behavioral and sender-reputation signals rather than content-quality signals alone.
SMBs are frequently more exposed — they typically have less mature email authentication (SPF/DKIM/DMARC), less security awareness training, and less out-of-band verification process for financial requests, all of which AI-generated social engineering specifically exploits.
Out-of-band verification for any financial or credential-related request — a callback to a known number, not a reply to the message itself. It’s low-cost, defeats the large majority of AI-assisted social engineering regardless of how convincing the message is, and doesn’t depend on employees correctly spotting a well-crafted fake.