Part 9e — Question Bank: Domain 5, Governance, Safety & Risk Management (Q99–Q119)
Original practice questions; not official or leaked. Objectives quoted from Exam Guide v1.0 §6 D5. Compliance items test architectural reasoning, not legal advice.
Q99 · D5 · Objective: Implement guardrails and safety controls · Difficulty: Advanced
Scenario: An email-triage agent reads inbound customer emails and can send external emails. A penetration test shows an email with hidden instructions made the agent forward internal data. Question: Which response is best? A. Lengthen the system prompt with stronger warnings. B. Switch to a larger model. C. Ask users to avoid suspicious emails. D. Remove the unconstrained send capability (use draft-only plus approval), authorize tool calls outside the model, restrict egress, treat email content as untrusted data, and add screens and monitoring. Correct answer: D Explanation: Layered controls with preventive capability reduction and enforcement outside the model address indirect injection. Why not the others: A and B rely on model behavior; C shifts burden to users. Architectural principle: Don't assume prompting alone is sufficient. Ref: https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks
Q100 · D5 · Objective: Implement guardrails and safety controls · Difficulty: Intermediate
Scenario: An assistant fetches web pages and passes their text to Claude. Question: How should the third-party content be delivered? A. Appended to the system prompt. B. In tool-result content with its origin labeled, with a policy stating such content is untrusted data that must not override instructions. C. As a message from the user role containing the text verbatim, unlabeled. D. Hidden in the first few tokens of the conversation. Correct answer: B Explanation: Treating retrieved content as tool results (data) with provenance, and stating the policy, reduces the chance it's treated as instructions. Why not the others: A and C elevate untrusted text to instruction channels; D obscures provenance. Architectural principle: Separate trusted instructions from untrusted data. Ref: Mitigate jailbreaks and prompt injections docs
Q101 · D5 · Objective: Implement guardrails and safety controls · Difficulty: Intermediate
Scenario: A public chatbot sees attempts to jailbreak it. You want an inexpensive pre-filter. Question: Which design aligns with Anthropic's guardrail guidance? A. A lightweight model screens input and returns a constrained classification (e.g., structured output) before the main call, alongside input validation and a hardened system prompt. B. Rely only on the main model to refuse. C. Block all users who type unusual words. D. Disable logging. Correct answer: A Explanation: Layered screening with a small model and constrained output is a recommended pattern. Why not the others: B is single-layer; C is blunt and error-prone; D removes monitoring. Architectural principle: Layered defenses. Ref: Mitigate jailbreaks docs
Q102 · D5 · Objective: Implement guardrails and safety controls · Difficulty: Advanced
Scenario: An agent can write files and call external URLs. A review flags exfiltration risk. Question: Which control provides the strongest preventive protection? A. A system-prompt rule not to exfiltrate data. B. A weekly review of logs. C. A network egress allow-list and tool-level path/URL restrictions enforced outside the model, with deny-by-default behavior. D. A user warning banner. Correct answer: C Explanation: Technical enforcement stops the action regardless of model behavior. Why not the others: A relies on compliance; B is detective and late; D isn't a control. Architectural principle: Enforce at the boundary, deny by default. Ref: Prep Lesson 3
Q103 · D5 · Objective: Implement guardrails and safety controls · Difficulty: Intermediate
Question (Select TWO): Which are preventive controls? A. Removing tools the workflow does not need. B. Immutable audit logs. C. Server-side authorization of every tool call with deny-by-default. D. Anomaly alerts on tool usage. E. Post-incident reviews. Correct answers: A, C Explanation: They stop or limit actions before they occur. Why not the others: B, D, and E detect or learn after the fact. Architectural principle: Prefer preventive controls; add detective ones. Ref: Prep Lesson 3
Q104 · D5 · Objective: Implement guardrails and safety controls · Difficulty: Advanced
Scenario: A support bot occasionally echoes customers' full card numbers from tool results into replies. Question: Which fix best matches a safe design? A. Add an apology after leaks. B. Hope the model learns. C. Warn users not to paste cards. D. Minimize and mask sensitive fields before they enter the prompt, add an output screen for PII patterns, and block or route violations to review; fail closed if the screen is unavailable. Correct answer: D Explanation: Data minimization plus output screening limits exposure; failing closed avoids leaks during outages. Why not the others: A and B don't prevent; C shifts burden. Architectural principle: Minimize data and screen outputs. Ref: Prep Lesson 3
Q105 · D5 · Objective: Identify risks, limitations, and failure modes of LLM systems · Difficulty: Intermediate
Scenario: An assistant has a tool that can delete customer accounts. A prompt injection triggers it. Question: Which risk category is this, and what is the primary mitigation? A. Hallucination; add citations. B. Excessive agency; apply least privilege by removing or tightly gating destructive capability and authorizing outside the model. C. Latency; add caching. D. Bias; add examples. Correct answer: B Explanation: The agent had more authority than needed. Why not the others: The symptoms don't match those categories. Architectural principle: Limit agency. Ref: Prep Lesson 3
Q106 · D5 · Objective: Identify risks, limitations, and failure modes of LLM systems · Difficulty: Advanced
Scenario: A crafted document causes an agent to loop through thousands of tool calls, creating a surprise bill. Question: What is the best control? A. Step, token, time, and spend budgets with hard stops and alerts at the orchestrator. B. Ask the model to be efficient. C. Increase quotas. D. Remove monitoring. Correct answer: A Explanation: Budgets enforce bounds regardless of model behavior. Why not the others: B isn't enforcement; C increases exposure; D hides it. Architectural principle: Bound autonomy and cost. Ref: Building effective agents
Q107 · D5 · Objective: Identify risks, limitations, and failure modes of LLM systems · Difficulty: Advanced
Scenario: Reviewers accept AI recommendations 99% of the time and rarely read evidence. Question: What risk is this, and what design response fits? A. Hallucination; increase temperature. B. Latency; stream. C. Automation bias; show evidence and alternatives, measure override rates, sample and audit decisions. D. Token bloat; compress. Correct answer: C Explanation: Over-reliance degrades oversight; design the review experience and measure it. Why not the others: Unrelated causes. Architectural principle: Design for human oversight effectiveness. Ref: Prep Lesson 3
Q108 · D5 · Objective: Apply human-in-the-loop validation strategies · Difficulty: Advanced
Scenario: A procurement agent has low confidence on a $40,000 vendor change that cannot be easily reversed. Question: What should happen? A. Auto-execute and log. B. Auto-execute and notify later. C. Retry until confidence rises. D. Route to a human decision-maker with the evidence and alternatives; do not execute without approval. Correct answer: D Explanation: Low confidence plus irreversibility and high cost warrants human decision. Why not the others: A–C act without oversight. Architectural principle: Route by confidence, reversibility, and cost. Ref: Prep Lesson 3
Q109 · D5 · Objective: Apply human-in-the-loop validation strategies · Difficulty: Expert
Scenario: A design auto-approves actions when Claude states "confidence: 95%." Question: What is the flaw, and what is better? A. No flaw; self-reported confidence is reliable. B. Self-reported confidence is not a calibrated probability; derive confidence from validators, retrieval coverage, agreement across runs, and classifier scores, and calibrate against labeled outcomes. C. Use 99% instead. D. Ask the model twice and average the numbers. Correct answer: B Explanation: Verbalized confidence can be miscalibrated; use measurable signals and calibration. Why not the others: A ignores calibration; C and D keep the same flaw. Architectural principle: Calibrated, signal-based confidence. Ref: Prep Lesson 3
Q110 · D5 · Objective: Apply human-in-the-loop validation strategies · Difficulty: Intermediate
Scenario: The safety-screening service times out during a traffic spike. Question: What should the system do for outputs that require screening? A. Fail closed: hold or route to review and alert, rather than releasing unscreened outputs. B. Fail open to avoid user delays. C. Retry forever. D. Skip screening for premium users. Correct answer: A Explanation: Safety controls default to deny when unavailable. Why not the others: B and D bypass safety; C blocks resources. Architectural principle: Fail closed for safety and security controls. Ref: Prep Lesson 3
Q111 · D5 · Objective: Apply human-in-the-loop validation strategies · Difficulty: Advanced
Question (Select TWO): Which changes best reduce rubber-stamp approvals? A. Show the evidence and alternatives, not just the recommendation. B. Remove the review step entirely. C. Sample and audit approvals and track reviewer override and error rates. D. Hide the underlying data from reviewers. E. Require approvals within five seconds. Correct answers: A, C Explanation: Evidence-rich review and oversight metrics improve review quality. Why not the others: B removes oversight; D worsens it; E pressures speed. Architectural principle: Make human review effective, not ceremonial. Ref: Prep Lesson 3
Q112 · D5 · Objective: Ensure compliance with regulations (e.g., GDPR, HIPAA, FedRAMP) · Difficulty: Advanced
Scenario: A payer wants an assistant to summarize member calls containing PHI. Design freeze is next week. Question: What must happen before freezing the design? A. Build first and check compliance after launch. B. Assume the vendor's general certifications automatically cover your use. C. Confirm the deployment path and vendor terms support PHI (e.g., a business associate agreement and eligible services), apply minimum-necessary data flows, and map controls to evidence. D. Remove logging so PHI isn't stored. Correct answer: C Explanation: Eligibility and data flows are architectural constraints; verify them early and with the proper documentation. Why not the others: A creates rework; B assumes coverage; D removes audit capability and doesn't address PHI in prompts. Architectural principle: Compliance by design with verified eligibility. Ref: Exam Guide D5; vendor compliance documentation
Q113 · D5 · Objective: Ensure compliance with regulations (e.g., GDPR, HIPAA, FedRAMP) · Difficulty: Advanced
Scenario: A user in the EU requests erasure of their personal data. The system stores it in a CRM, a vector index, application logs, and a cache. Question: What must the architecture support? A. Deleting only the CRM record. B. Telling the user it can't be done. C. Waiting until logs rotate in a year, with no process. D. A tested deletion process covering source systems, embeddings/indexes, logs, caches, and backups per policy, with evidence of completion. Correct answer: D Explanation: Erasure must reach every store holding the data. Why not the others: A–C leave data behind or are non-compliant. Architectural principle: Design data lifecycle and deletion across all stores. Ref: Exam Guide D5
Q114 · D5 · Objective: Ensure compliance with regulations (e.g., GDPR, HIPAA, FedRAMP) · Difficulty: Advanced
Scenario: A federal agency requires services authorized at a specific FedRAMP impact level. Question: What is the appropriate architect action? A. Use any region; the cloud provider handles it. B. Verify that each service in the proposed deployment path holds the required authorization and that boundaries and data flows are within it; select the path accordingly. C. Use a consumer app. D. Ask the model whether it complies. Correct answer: B Explanation: Authorization applies to specific services and boundaries and must be verified. Why not the others: A and C assume; D is not authoritative. Architectural principle: Verify authorization status for the exact deployment path. Ref: Exam Guide D5
Q115 · D5 · Objective: Ensure compliance with regulations (e.g., GDPR, HIPAA, FedRAMP) · Difficulty: Intermediate
Question (Select TWO): Which implement the principle of data minimization? A. Filtering records to only the fields needed before building the prompt. B. Sending entire customer records "for context." C. Redacting identifiers not needed for the task in logs and indexes. D. Keeping all prompts forever. E. Adding more examples to the prompt. Correct answers: A, C Explanation: Limit data in prompts, logs, and indexes to what's necessary. Why not the others: B and D expand exposure; E is unrelated. Architectural principle: Minimize data by design. Ref: Exam Guide D5
Q116 · D5 · Objective: Ensure compliance with regulations (e.g., GDPR, HIPAA, FedRAMP) · Difficulty: Intermediate
Scenario: An auditor asks you to demonstrate that access controls work. Question: Which evidence is most convincing? A. Control test results (e.g., low-privilege and cross-tenant attempts denied), access logs, and approval records mapped to the control owner. B. A policy document stating the intent. C. A verbal assurance. D. A screenshot of the prompt. Correct answer: A Explanation: Evidence shows controls operate, not just that they're intended. Why not the others: B–D don't demonstrate operation. Architectural principle: Control → test → evidence. Ref: Prep Lesson 3
Q117 · D5 · Objective: Address ethical AI considerations (bias, fairness, transparency) · Difficulty: Advanced
Scenario: A résumé-screening assistant ranks identical resumes differently when applicant names change. Question: What is the best response? A. Declare it neutral. B. Hide the ranking. C. Remove unnecessary identity attributes from inputs, test with counterfactual pairs and segment metrics, keep human decision-makers, and document limits. D. Only add a disclaimer. Correct answer: C Explanation: Measure, mitigate, and preserve oversight for consequential decisions. Why not the others: A ignores evidence; B hides the issue; D doesn't fix harm. Architectural principle: Fairness testing and human oversight. Ref: Exam Guide D5
Q118 · D5 · Objective: Address ethical AI considerations (bias, fairness, transparency) · Difficulty: Intermediate
Scenario: A benefits assistant influences eligibility guidance. Question: Which transparency measures are most appropriate? A. Hide that AI is used. B. Provide only the final answer. C. Explain the system in a long technical paper. D. Disclose AI use, provide sources, state limitations, and offer a path to human review or appeal. Correct answer: D Explanation: Transparency builds appropriate trust and recourse. Why not the others: A and B obscure; C isn't user-facing. Architectural principle: Transparency and recourse. Ref: Exam Guide D5
Q119 · D5 · Objective: Address ethical AI considerations (bias, fairness, transparency) · Difficulty: Advanced
Scenario: Overall accuracy is 95%, but it is 82% for one language group; the acceptance policy requires no segment below 90%. Question: What is the right decision? A. Ship; overall accuracy is high. B. Treat it as a gate failure: investigate causes (data, prompts, retrieval, model), mitigate, and re-test before release. C. Remove that segment from reporting. D. Add a disclaimer. Correct answer: B Explanation: Segment thresholds exist to prevent disparate quality. Why not the others: A, C, and D conceal or ignore the gap. Architectural principle: Gate on fairness metrics by segment. Ref: Exam Guide D4/D5