Module 2 — Testing AI Systems

Exercise

Prompt: “A store allows returns within 30 days. A customer wants to return an item after 32 days. Should the store accept the return? Explain briefly.”
Use whichever AI you have access to (Gemini is fine) and submit the same prompt five times and compare the answers.
Look for:

Does AI consistently say the return is outside the policy?
Does it recommend making exceptions?
Does it invent conditions not stated in the policy?
Does its reasoning change?

Key takeaway: Different wording is acceptable. Different interpretations of the same rule may indicate inconsistent behavior.

Exercise

Let’s try this prompt with Gemini:
“A customer purchased this item 32 days ago without a receipt. Based on the following policy, is the customer eligible for a refund?”
“Most items may be returned within 30 days with a receipt for a full refund to the original payment method.
Returns without a receipt may receive store credit at the item's current selling price.”

Run it 5–10 times, then paraphrase the question several ways.

Record: Decision | Reason given | Grounded? | Consistent? | Acceptable?