Open with a live-scale statistic: Google reports its ML filters block roughly 100 million phishing messages from Gmail every day, and Microsoft Defender's cloud ML processes trillions of signals per day. Ask students: 'If a human analyst can review maybe 100 alerts a shift, and your enterprise generates 10 million events a day, what has to be true about defense?' Take 2–3 responses. Land the punchline: AI is not optional at scale — but it is not a magic filter either. Tell students that today they will look at how those filters actually decide, why they still miss things (and false-alarm), and where a human must stay in the loop. Note this is the last content day before the unit test, and the closing block will double as unit review.

1. 7m

   How ML learns a behavioral baseline and flags anomalies

   Content

   Anomaly detection in cyber defense works by first learning a behavioral baseline — a statistical picture of what 'normal' looks like for a user, host, or network segment. The model is trained on historical logs: typical login hours, common source IPs, average bytes-out per session, usual processes launched, protocols used. Once trained, every new event is scored against that baseline; events that fall far outside the learned distribution are flagged as anomalies. This is fundamentally different from signature vs behavior detection: a signature engine matches a known-bad hash or string (fast, precise, but blind to novel attacks), while behavior/anomaly detection can catch never-before-seen activity but does not know why the event is unusual — only that it is statistically unusual. Worked example: a finance-team account normally logs in from Dallas between 08:00–18:00, transfers <50 MB/day, and uses Outlook and Excel. At 03:14 the same account logs in from a Bucharest IP, spawns PowerShell, and pushes 4 GB to an external host. Every one of those features sits far outside the baseline; the anomaly score spikes and the SIEM raises an alert. Note what the model does NOT know: it does not know the user is on vacation, does not know PowerShell was authorized by IT, does not know intent. It only knows the distance from normal.

   Delivery

   Emphasize the phrase 'distance from normal, not knowledge of intent' — this is the misconception to pre-empt right now. Ask: 'If the same finance user genuinely traveled to Bucharest for a conference and worked at night, what would the model do?' (Flag it — that's a false positive waiting to happen.) Then ask: 'What if an attacker mimicked the user's exact baseline behavior?' (Model misses it — false negative.) These two questions set up beat 2. Keep pace tight; don't get into algorithm math — AP scope is conceptual.

2. 7m

   The confusion matrix: false positives vs false negatives in security

   Content

   Every detector's outputs sort into four cells. True positive (TP): attack occurred, model flagged it — win. True negative (TN): benign, model stayed quiet — win. False positive (FP): benign activity flagged as attack — wastes analyst hours, causes alert fatigue, and if tied to automated response can block legitimate business (a shipping firm auto-blocking its own CFO's login on quarter-close). False negative (FN): real attack, model missed it — the intrusion proceeds. Tuning a model always trades these off: lower the anomaly-score threshold and you catch more attacks (fewer FNs) but drown analysts in FPs; raise the threshold and FPs drop but FNs rise. Concrete numbers: suppose a spam/phishing filter processes 1,000,000 emails/day, 1% (10,000) are truly malicious. At 99% recall and 0.1% FP rate, the filter catches 9,900 phishes (100 FNs slip through) but also flags 990 clean emails as spam (990 FPs). Which error is worse depends on context — for phishing an FN can mean credential theft; for an IDS auto-block an FP can shut down a hospital's EHR. This is why the confusion matrix is a risk-analysis tool, not just a math object.

   Delivery

   Walk the four cells in order and hammer that FP and FN have DIFFERENT costs — this is the AP Analyze Risk skill in action. Ask: 'For a hospital intrusion detection system that auto-quarantines endpoints, which error is more dangerous — FP or FN?' Push students to argue both sides (an FN lets ransomware run; an FP quarantines an ICU workstation). The point: the answer depends on impact, and that's the analyst's judgment call, not the model's. Correct the misconception directly: 'AI does NOT catch everything — every model has an FN rate, and the FP rate is what makes analyst triage necessary.'

3. 6m

   Human-in-the-loop and model drift — why analysts don't go away

   Content

   A mature SOC pipeline looks like this: raw telemetry → ML detection → automated enrichment (WHOIS, threat intel lookups) → alert triage queue → analyst decision → response (block, isolate, ignore, escalate) → feedback label back into training data. The AI is a force multiplier that surfaces the top-k suspicious events; the human decides. Two reasons humans stay in the loop. First, models do not understand intent or business context — an analyst knows that the 'anomalous' 3 AM database dump is the quarterly backup job IT scheduled last week. Second, models drift: attackers evolve tactics (living-off-the-land binaries, low-and-slow exfiltration), and 'normal' itself changes (COVID sent every user to residential IPs overnight and broke every geolocation baseline). Without model retraining on fresh, labeled data, accuracy decays. A reasonable design rule for automated response: auto-block only when confidence is very high AND the action is low-blast-radius (quarantine a single endpoint, drop a single email). High-blast-radius actions (disable an account, shut down a subnet) require analyst review. This is the AP Mitigate Risk skill: layered controls with humans placed where judgment matters most.

   Delivery

   This beat closes today's new content and pivots into unit review. Tie the workflow back to layered defense from earlier in the unit — AI detection is one control layer, not the whole stack. Ask: 'What went wrong in COVID for geolocation-based anomaly models?' (Baseline invalidated overnight — everyone became 'anomalous'.) Then set up the transition: 'The alert-triage activity next asks you to be the analyst — the AI has surfaced candidates, you decide.' Correct the two remaining misconceptions explicitly: models are NOT permanently accurate (they drift and need retraining), and AI does NOT replace analysts (it prioritizes their queue).

4. 4m

   Unit test preview — power ideas across Introduction to Security

   Content

   Tomorrow's unit test covers Introduction to Security as a whole. The power ideas to review: (1) the CIA triad — confidentiality, integrity, availability — and matching each to a real breach type; (2) threat actors and motivations (nation-state, cybercriminal, insider, hacktivist, script kiddie); (3) the vulnerability–threat–risk relationship (risk = likelihood × impact, and a vulnerability with no threat is still a weakness but not yet a risk); (4) defense-in-depth / layered controls (preventive, detective, corrective); (5) today's material — AI as a detective control with FP/FN trade-offs and human-in-the-loop. Highest-leverage vocabulary to lock in: CIA triad, vulnerability, threat, risk, exploit, control, defense-in-depth, false positive, false negative. Misconceptions students most often miss on the exam: confusing threat with risk (a threat is a potential actor/event; risk is the exposure that results), treating antivirus/AI as complete solutions rather than layers, and forgetting that availability attacks (DDoS, ransomware) count as CIA violations too.

   Delivery

   Keep this beat brisk — it is a signpost, not a re-teach. Tell students the final activity is a mixed-review triage that pulls from all these ideas, and the exit ticket is written in AP exam style. Ask a fast retrieval round: 'Threat vs risk in one sentence each?' Cold-call two students. Then transition to the activity.

1. 18m

   SOC Alert Triage Lab — you are the analystLab

   Targets AP Skill Category 3 (Detect Attacks) and Skill Category 1 (Analyze Risk). Students work in pairs at their computers. Give each pair the six-alert handout below; they classify each alert as likely TP, likely FP, or Need More Info, then decide the response (auto-block, escalate to Tier-2 analyst, or dismiss with note). They must justify each call in one sentence citing the baseline deviation, business context, or missing evidence. Circulate and pressure-test their reasoning — especially any pair that marks every alert 'escalate' (that is the alert-fatigue failure mode) or every alert 'auto-block' (that is the blast-radius failure mode). Debrief last 4 minutes: cold-call pairs on alerts 3 and 5, which are the two designed to split the room. Student handout — SOC Alert Triage You are a Tier-1 SOC analyst at MedNorth Regional Hospital. Your ML detection platform produced the six alerts below in the last hour. For EACH alert, decide: - Classification: likely True Positive · likely False Positive · Need More Info - Action: Auto-block · Escalate to Tier-2 · Dismiss (with note) - Justification (1 sentence): cite the baseline deviation, business context, or the missing evidence that drove your call. You must justify every call. 'It looks suspicious' is not a justification. Alert 1. User jbanks (billing clerk) logged in at 02:47 from a residential IP in Manila. Baseline: logins 07:30–17:00, Dallas-area IPs only. No prior travel ticket filed. Alert 2. Domain controller DC-01 initiated an outbound TLS connection to 185.220.101.47 (known Tor exit node from threat intel feed). DCs have zero baseline outbound internet traffic. Alert 3. Radiology workstation RAD-14 spawned powershell.exe with an encoded command. Baseline: RAD-14 runs DICOM viewer only. IT ticket #4471 (opened 40 minutes ago) shows a vendor is remotely patching radiology software tonight. Alert 4. 312 failed SSH logins to jump-host bastion-2 from 47 distinct IPs in 8 minutes. Bastion is public-facing and rate-limited; MFA is required for success. Alert 5. CFO account transferred 2.1 GB to Dropbox at 16:52. Baseline: CFO averages 180 MB/day to Dropbox; quarter-close is this week and finance routinely uploads audit packages then. Alert 6. Nursing tablet NUR-08 beaconing every 60.0 seconds (± 0.2 s) to 91.234.55.12 for the last 3 hours. Destination has no reputation data. Baseline: no such pattern. Debrief questions (answer with your partner): - Which alert would be most costly as a false negative if you dismissed it? Why? - Which alert would be most costly as a false positive if you auto-blocked it? Why? - For which alerts did business context (not the model) change your decision?

   Materials

   • Student computers with a spreadsheet or text editor
   • Printed or on-screen handout (content below)
   • Timer visible to the room

   Example outputs

   • Alert 2 — True Positive, Escalate to Tier-2 (or auto-isolate DC): a domain controller has no baseline outbound internet traffic and Tor exit destinations are strong threat-intel indicators; even if FP, DC isolation blast radius is high so escalate first rather than auto-block.
   • Alert 3 — Need More Info, Escalate with IT ticket attached: encoded PowerShell on a DICOM-only workstation is a classic living-off-the-land pattern (TP-shaped), BUT ticket #4471 provides business context that could make this a FP; analyst must verify vendor identity before dismissing.
   • Alert 5 — likely False Positive, Dismiss with note: 2.1 GB is ~12× baseline but quarter-close context explains it; auto-blocking the CFO during audit uploads is high-blast-radius — analyst should verify with finance before any action.
   • Alert 6 — likely True Positive, Escalate: 60.0 s ± 0.2 s beaconing is a signature of C2 heartbeat traffic; low jitter is not a business pattern.

   No-equipment fallback

   Distribute the handout on paper; pairs write classification/action/justification in the margin. Same debrief.

2. 12m

   Unit Test Rapid-Review Retrieval Round

   Targets AP Skill Categories 1 (Analyze Risk) and 2 (Mitigate Risk). Whole-class retrieval practice for tomorrow's Introduction to Security unit test. Students work the 8 items solo for 6 minutes, then the class goes over them together for 6 minutes. Cold-call or use randomizer. For each item, after revealing the answer, ask ONE follow-up 'why is the wrong answer wrong' — that's where AP points are lost. Student handout — Unit Review Retrieval (8 items) 1. Define threat, vulnerability, and risk in one sentence each, then give one concrete hospital example that includes all three. 2. A ransomware attack encrypts a hospital's EHR database and demands payment for the decryption key. Which parts of the CIA triad are violated? Name all that apply and justify each. 3. Classify each control as preventive, detective, or corrective: - multi-factor authentication - SIEM anomaly alerts - restoring from backup after ransomware - firewall egress rules - forensic disk imaging after an incident 4. A vendor sells an 'AI firewall that stops 100% of attacks.' Give TWO reasons, using unit vocabulary, that this claim is not credible. 5. Match the threat actor to the most likely motivation: nation-state, cybercriminal, insider, hacktivist, script kiddie — motivations: financial gain, ideology, espionage, notoriety/curiosity, grievance/access misuse. 6. An IDS flags 940 alerts per day; investigation shows 12 are true attacks. Compute the approximate false positive rate among alerts and explain in one sentence why this number matters for analyst staffing. 7. Explain why a false negative in an anti-phishing filter can be more damaging than a false positive, using a specific attack scenario. 8. Your CISO proposes letting the AI auto-disable any user account whose login is flagged as anomalous. Argue for or against, referencing blast radius and human-in-the-loop.

   Materials

   • Whiteboard or projector for scoring
   • Handout (content below) — one per student

   Example outputs

   • Item 1: Threat = a potential actor/event that could cause harm (ransomware gang); Vulnerability = a weakness (unpatched EHR server); Risk = the exposure created when a threat can exploit the vulnerability (likelihood the gang exploits the unpatched EHR × impact of encrypted patient records).
   • Item 2: All three — Confidentiality (attackers may have exfiltrated records before encryption — double extortion), Integrity (data has been altered/encrypted without authorization), Availability (clinicians cannot access EHR). Students who name only Availability lose points.
   • Item 6: 928/940 ≈ 98.7% FP rate among alerts; at this rate analysts develop alert fatigue and true positives get missed — this is why triage and tuning matter.
   • Item 8: Argue AGAINST auto-disable for all accounts — blast radius is high (disabling a physician mid-code-blue endangers patients); reserve auto-response for low-blast-radius actions (quarantine one endpoint, drop one email) and keep human-in-the-loop for account-level actions.

1. A university deploys an ML-based intrusion detection system that has a 95% true-positive rate and a 2% false-positive rate. The network generates 500,000 events per day, of which 100 are truly malicious. Approximately how many alerts will analysts see per day, and roughly what fraction of those alerts will be false positives? Show your reasoning.

   calculationTrue positives ≈ 0.95 × 100 = 95. False positives ≈ 0.02 × 499,900 ≈ 9,998. Total alerts ≈ 10,093/day. Fraction FP ≈ 9,998 / 10,093 ≈ 99.1%. Reasoning point: even a 2% FP rate on a huge benign population overwhelms the 95% recall on the tiny malicious population — this is why triage and human-in-the-loop are non-negotiable. (Targets Skill 1 Analyze Risk and Skill 3 Detect Attacks.)

2. An anomaly detection model was trained on pre-2020 office-network traffic. In March 2020 the workforce shifts to home offices overnight, and the model begins flagging nearly every user as anomalous. Which cybersecurity concept best explains this failure, and what specific mitigation should the security team implement? A) Signature evasion — deploy a new signature feed B) Model drift / stale baseline — retrain the model on current traffic and schedule periodic retraining C) False negative rate — lower the alert threshold D) Insider threat — investigate the users being flagged

   multiple choiceB. The behavioral baseline no longer reflects current normal behavior; retraining on fresh labeled data restores accuracy, and periodic retraining prevents recurrence. A is wrong because this is a behavior model, not signature. C makes the problem worse (more FPs). D confuses a modeling failure with a human threat. (Targets Skill 2 Mitigate Risk.)

3. Free response — AP style. A hospital CISO proposes deploying an AI system that automatically disables any user account whose login is flagged as anomalous. In 3–5 sentences, evaluate this proposal. Your response must (a) identify at least one risk of automating this decision, (b) reference the concepts of false positive and blast radius, and (c) recommend where a human-in-the-loop review must remain.

   short answerStrong response identifies that anomaly detection flags statistical deviation, not intent, so false positives are guaranteed — auto-disabling on FP has HIGH blast radius (a physician locked out during patient care, a payroll admin locked out on payday). The proposal should be modified: auto-response is acceptable for LOW-blast-radius actions (quarantine one endpoint, drop one email, step-up MFA challenge), while account disable/lockout must require Tier-2 analyst review with business-context verification. Also acceptable: mention model drift requiring retraining, and the need to log/label decisions to feed back into training data. (Targets Skill 1 Analyze Risk and Skill 2 Mitigate Risk.)

4. Unit-test-style item. Match each scenario to the CIA-triad element(s) most directly violated: (i) attacker exfiltrates a database of patient SSNs; (ii) attacker modifies lab-result values in the EHR; (iii) DDoS attack knocks the patient portal offline for 6 hours; (iv) ransomware encrypts imaging archives.

   short answer(i) Confidentiality — data disclosed to unauthorized party. (ii) Integrity — data altered without authorization; note clinical harm potential. (iii) Availability — legitimate users cannot access the service. (iv) Availability AND Integrity (data altered/encrypted); also Confidentiality if double-extortion exfiltration occurred first. Common student error: naming only Availability for ransomware. (Unit review — targets Skill 1 Analyze Risk.)

• ☐Student computers (1 per pair) with spreadsheet or text editor
• ☐Printed handouts: (1) SOC Alert Triage — six alerts, (2) Unit Review Retrieval — eight items
• ☐Projector for slide deck
• ☐Whiteboard for confusion-matrix worked numbers and review scoring
• ☐Timer visible to the room