Cybersecurity · Unit 1 · Lesson A.1

Deep Dive: AI-Based Attacks

🔬 Deep Dive
This is your textbook for this topic. Take your time. Read it more than once.
1.4.AAttack Technique

AI-Powered Attack Types

Attackers have always had a toolkit — phishing emails, phone pretexting, reconnaissance scraped by hand. AI doesn't create a brand-new category of crime; it removes the friction from every step attackers already used, and it makes a handful of new tricks possible for the first time. Five techniques matter most right now:

🎭 Deepfakes
📧 AI-generated phishing
🧩 LLM prompt injection
🔍 AI-powered reconnaissance
🦠 AI-written malware

Deepfakes

A deepfake is synthetic audio or video generated to convincingly impersonate a real person — cloning a voice from a few seconds of a public speech, or generating a video call where the "person" is entirely AI-rendered. The classic case is CEO fraud: a finance employee receives a call that sounds exactly like their CEO, urgently requesting a wire transfer. The voice is real-sounding because, in a sense, it is real audio — just recombined by AI rather than spoken live.

AI-generated phishing

For years, one of the most reliable tells of a phishing email was bad grammar or an awkward, obviously-translated tone. Large language models erase that tell. An attacker can now generate a fluent, well-punctuated, perfectly localized phishing email in any language, personalized with details pulled from a target's public profile — no longer limited by the attacker's own writing skill or fluency.

⚠️"It was well-written and specific to me" is no longer evidence that a message is legitimate. AI has made fluency and personalization cheap for attackers to produce at scale.

LLM prompt injection

This one is different from the others — it doesn't target a human at all. Prompt injection attacks the AI system itself: an attacker hides malicious instructions inside content an AI assistant will process — a webpage, an email, a document — hoping the AI follows the hidden instructions instead of (or in addition to) the user's actual request. As more tools give AI assistants the ability to browse, read files, or take actions on a user's behalf, this becomes a real way to hijack what the assistant does.

AI-powered reconnaissance

Before any attack, a skilled attacker researches the target — names, job titles, reporting structure, travel schedules, writing style. That research used to take hours of manual digging. AI tools can now scrape and synthesize a target's public digital footprint into a usable profile in seconds, dramatically shortening the time between "choosing a target" and "launching a convincing attack."

AI-written malware

AI can also assist on the purely technical side — helping generate or obfuscate malicious code, or producing polymorphic malware that rewrites its own code slightly with every copy so that signature-based antivirus tools struggle to recognize it.

🔑Notice the pattern: AI attacks mostly exploit the same human trust signals people have always relied on — a familiar voice, fluent writing, a message that "knows" personal details. AI just makes those signals cheap and fast to fake.
Attack TechniqueAttack Chain — Voice-Cloned Executive Fraud

A finance employee at a mid-size company receives an urgent call from someone who sounds exactly like the company's CFO, asking for an emergency wire transfer to a new vendor account before end of day.

Step 1Recon
The attacker scrapes a 90-second video of the real CFO speaking at a public conference, plus their name, title, and typical vendor relationships from LinkedIn and press releases.
1.4.A — AI-Powered Attack Types  ·  ~1 min
1.4.BDefense

Defenses Against AI Attacks

Here's the reassuring part: the defenses that stop AI-powered attacks aren't new or exotic. They're the same boring, reliable habits that have always stopped social engineering — they just matter more now that the "tells" attackers used to give off are gone.

Shared secrets and verification phrases

A shared secret is a pre-agreed code word or phrase that only legitimate parties know — established before any suspicious contact happens, never during it. Families use this to defeat "grandparent scam" voice clones; companies use it for high-value requests like wire transfers. If a caller claiming to be the CFO can't produce the agreed phrase, the call is not trusted, no matter how convincing the voice sounds.

Verify through a second, independent channel

If an urgent request arrives by phone, confirm it by text or in person on a number you already had on file — never one the caller gives you during the call. This single habit defeats deepfakes and AI phishing alike, because it doesn't depend on detecting anything about the message itself.

MFA as defense

Multi-factor authentication stops AI attacks at a different point in the chain. Even if an AI-written phishing email is flawless and a user is fully fooled into typing their password into a fake site, MFA means the attacker still can't log in without the second factor — something the target has (a phone, a hardware key) or something they are (a fingerprint). AI can perfect the lie; it can't fabricate a factor it doesn't possess.

💡Notice that shared secrets, second-channel verification, and MFA all share one property: none of them require a human to correctly judge whether a message "feels real." That's exactly why they still work once AI has made messages feel real by default.

Explore the interactive below — toggle the defense layer on and off to see exactly where in an attack chain a shared secret or MFA prompt actually stops the attacker.

This is a typical AI-assisted social-engineering chain. Toggle the defense layer to see where a single habit — verifying through a second channel — breaks the chain, no matter how convincing the deepfake is.

1AI Recon2Pretext Built3Deepfake Contact4Request Made
1. AI ReconScrapes LinkedIn, press releases, and social posts to learn names, titles, and travel schedules.
2. Pretext BuiltAI drafts a fluent, personalized message impersonating a real executive or vendor.
3. Deepfake ContactA cloned voice or video calls or messages the target, creating urgency.
4. Request Made"Wire the funds" / "reset my password" / "send me the file" — the actual ask.
Result: attack succeeds. Nothing in this chain requires the attacker to be caught — every stage relies on the target trusting what they see and hear.
DefenseCase File — Stopping the CFO Wire-Transfer Scam

Same scenario as the attack chain above: a voice-cloned 'CFO' calls a finance employee requesting an urgent wire transfer. Here's how the company's defenses actually stopped it.

1.4.B — Defenses Against AI Attacks  ·  ~1 min
← Back to Lesson A.1Try the Activity →