Principles of AI · Unit 1: Inside the Machine ·  Project 1.2.4

Red-Team Report — Break It, Rank It, Fix It

A bot you build, a bot you break, and an honest report on what failed and what fixed it  ·  Approx. 3 class days

StarringBuild · Helper BotAttack · Break ItReport · Fix & Retest
Unit 1: Inside the Machine

Quick reference for the project steps, the six kinds of attack, the severity and risk ratings, the six places a fix can work, and what goes in the report.

Red-Team Report project infographic: build a helper bot, rules of engagement, six kinds of attack, severity times how often equals risk, six places a fix can work, retesting, and the parts of the report

🧭 Plot Summary

In Unit 1 you saw a kiosk leak its password, a chatbot make up a source, and a fake message push you to act. Real teams that ship AI tools do something on purpose to find these problems before the public does: they red-team them. A red team is allowed to attack a system so its owners can fix it.

Your team builds a small Helper Bot with a job, a fact sheet, a protected note and rules, then breaks another team's bot while they break yours. The product is a Red-Team Report: every attempt logged, every failure re-run and rated, the top risks ranked, a fix for each, and a retest that says whether the fix actually worked, without making the bot useless.

What you'll do in this project

  • Form a team, assign roles, and agree on the Rules of Engagement before you test anything.
  • Build a small Helper Bot: one job, a fact sheet, a protected note, and rules. Another team will try to break it.
  • Attack the other team's bot with at least 20 logged attempts across six kinds of failure.
  • Re-run every failure three times, rate how bad it is, and rank the risks from your own log.
  • Recommend a fix for each top finding, let the owners apply it, and retest to see whether it really helped.
  • Write a Red-Team Report that is honest about what you did not test and who must still check the bot's work.

Ground rules

  • Red-teaming is only legitimate with permission and inside a scope. Your scope is the other team's Helper Bot, in a chat opened for this project.
  • Everything in the bot is invented. No personal information, no real classmates, and no real school systems in any prompt.
  • You are testing the bot's job. Don't try to get violent, hateful, explicit, dangerous or illegal content. If a bot produces something harmful, stop and tell your teacher.

Why it matters

A system that passed the tests its builders thought of is not the same as a system that is safe. Finding failures on purpose, ranking them honestly, and checking that a fix works are how AI tools get better. In Unit 2 you will plan for AI mistakes in real tasks, and this is the habit that makes that planning possible.

✅ Self-Check Before You Present

Check off each item as you get there. These aren't grades. They're your own signal.