Part 2 — Domain 2: Claude Models, Prompting & Context Engineering (13%)
Prep-course lesson: Claude Platform & Solution Design. Official tasks: select models by trade-offs · system prompts, templates, guardrails · zero-shot/few-shot/chain-of-thought · optimize context windows and token usage · prompt reuse (caching, modular prompts, Skills).
Topics: 2.1 Model selection · 2.2 System prompts, templates, techniques · 2.3 Context-window and token management · 2.4 Prompt reuse: caching, modular prompts, Skills
Verify-before-you-memorize: model names, context sizes, prices, and caching minimums change. The exam tests selection logic. Check https://docs.claude.com → Models overview and Pricing for current numbers.
2.1 Selecting Claude models on trade-offs
1. Objectives. Choose a model tier for a task using capability, latency, cost, context needs, and risk; know when to route across tiers.
2. Prerequisites. Tokens, input vs. output pricing, latency basics.
3. Beginner. Claude comes in tiers. Smaller/faster tiers (Haiku) are cheap and quick for simple, high-volume work. Mid-tier (Sonnet) balances intelligence and cost for most workloads. Larger tiers (Opus; and the top Mythos-class tier, e.g., Fable) handle the hardest reasoning and long-horizon work at higher cost and latency. At the time of writing the current API strings are claude-haiku-5-5, claude-sonnet-5-5, claude-opus-5-5, claude-fable-5-1.
4. Advanced. Selection is an empirical decision against your eval set, not a brand preference. Considerations: (a) task difficulty (classification vs. multi-step reasoning); (b) latency budget (time-to-first-token vs. total); (c) cost = input + output tokens × price, plus cache and batch effects; (d) context size and output length needs; (e) feature support (tools, vision, structured outputs, thinking / effort controls — verify per model); (f) deprecation/lifecycle (pin versions, plan migrations); (g) risk (higher-stakes → more capable model plus controls, not instead of controls). Tiered routing (cheap model for easy cases, stronger model for hard ones, or a small model as a screen/classifier) commonly beats a single model on cost at equal quality.
5. Steps. (1) Build a representative eval set (D4). (2) Run candidate models with the same prompt. (3) Plot quality vs. cost vs. p95 latency. (4) Pick the cheapest model that clears acceptance criteria with margin. (5) Add a fallback/escalation tier. (6) Pin the model ID; schedule re-evaluation when new models ship.
6. Diagram.
flowchart LR R[Request] --> S[Small-model router / screen] S -- easy --> H[Haiku-class answer] S -- standard --> N[Sonnet-class answer] S -- hard / low confidence --> O[Opus-class answer] H & N & O --> V[Validate output]
7. Scenario. A bank classifies 2M support messages/day into 40 intents (latency 500 ms) and also drafts replies for 3% flagged as complex. Use a small model for classification (batch for back-office tagging), a mid-tier for replies, and escalate only low-confidence cases.
8. Recommended/alternatives. Tiered routing. Alternative: one top-tier model everywhere (simple, 5–20× cost); smallest model everywhere (quality risk). Change if eval shows the small model misses accuracy thresholds, or a new model changes the Pareto frontier.
9. Security/governance. Model upgrades are changes: run regression + safety evals before switching; keep version pinning and rollback.
10. Performance/cost. Model cost dominates at volume; cache static prefixes; use batch for non-interactive work; cap max_tokens sensibly.
11. Mistakes. Picking the biggest model "to be safe"; comparing models with prompts tuned for only one; ignoring output tokens' price; auto-upgrading in production.
12. Scenario question. Latency SLA is 800 ms for single-label classification; the top-tier model scores 97%, the smallest scores 96% on your eval. → The small model (meets quality with margin, meets SLA, far cheaper).
13. Revision. Empirical selection; Pareto of quality/cost/latency; tiered routing; pin and re-eval.
14. Docs. Models overview and Pricing (docs.claude.com); Prep Lesson 1.
2.2 System prompts, templates, guardrails, and prompting techniques
1. Objectives. Write system prompts and templates that are clear and testable; apply zero-shot, few-shot, and chain-of-thought appropriately; understand what prompt-level guardrails can and cannot do.
2. Prerequisites. Messages API roles (system, user, assistant).
3. Beginner. The system prompt sets role, rules, tone, and output format. A template has fixed instructions plus variable slots. Zero-shot = instructions only. Few-shot = include 2–5 examples. Chain-of-thought = let the model reason before the final answer (and/or use thinking features on supported models).
4. Advanced. Principles that hold across versions: be explicit about task, audience, constraints, and output format; separate instructions from data with delimiters or XML-style tags; place long documents before the question; give examples that are diverse and representative (and include edge cases); specify what to do when information is missing ("say you don't know", cite sources); use structured outputs/JSON schema or tool schemas for machine-consumed output. Few-shot helps with format and nuanced judgment; CoT helps multi-step reasoning but adds tokens/latency. Guardrails in prompts are guidance, not enforcement. Anthropic's guardrail docs recommend layered defenses (input screening, output validation, monitoring); authorization and data access must be enforced in code (D3, D5).
5. Steps. (1) Draft the prompt from success criteria. (2) Parameterize variable inputs. (3) Add 3–5 examples including a hard negative. (4) Specify output schema and refusal/uncertainty behavior. (5) Test against the eval set; iterate one change at a time. (6) Version the prompt (D7).
6. Executable example (illustrative structure; check current SDK/model IDs):
# EXECUTABLE — requires ANTHROPIC_API_KEY
import anthropic
client = anthropic.Anthropic()
SYSTEM = """You are a claims-summary assistant for adjusters.
Use ONLY the provided <policy> and <notes>. If coverage is unclear, say "Needs review" and list what is missing.
Return JSON: {"coverage_summary": str, "citations": [str], "needs_review": bool}."""
def summarize(policy: str, notes: str) -> str:
msg = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=800,
system=SYSTEM,
messages=[{"role": "user",
"content": f"<policy>\n{policy}\n</policy>\n<notes>\n{notes}\n</notes>\nSummarize coverage."}],
)
return msg.content[0].text
7. Scenario. The support bot sometimes invents refund policies. Fix: constrain to retrieved policy text, require citations, return "needs_review" when absent, validate citations in code, and test with adversarial cases.
8. Recommended/alternatives. Explicit instructions + schema + few-shot + code validation. Alternatives: fine-tuning (not the default path for most architects — assess availability/need); longer prose instructions (diminishing returns). Change if few-shot examples bloat cost → cache them or fold into a skill.
9. Security. "Never reveal the system prompt" or "don't call delete" in a prompt is not a control. Pair prompts with deterministic checks.
10. Cost/latency. Few-shot and long prompts increase input tokens (mitigate with caching); CoT increases output tokens.
11. Mistakes. Vague prompts; mixing instructions with untrusted data; examples that all look alike; changing several things per iteration; treating prompts as security.
12. Scenario question. Output JSON sometimes has trailing prose that breaks the parser. → Use structured outputs / tool-schema output and validate; don't just add "ONLY output JSON" in capitals.
13. Revision. Explicit, structured, exemplified, schema-bound, tested; prompts guide, code enforces.
14. Docs. Prompt engineering overview; Mitigate jailbreaks and prompt injections: https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks
2.3 Context windows and token management
1. Objectives. Budget context; choose between stuffing, retrieval, summarization, and progressive loading; control token cost.
2. Prerequisites. Tokens; context window = input + conversation + output.
3. Beginner. The context window is everything the model can see in one request. Bigger windows allow more, but more tokens cost more and can dilute attention. Output tokens are limited by max_tokens.
4. Advanced. Strategies: (a) put only relevant context in (retrieval rather than "dump everything"); (b) progressive disclosure — give the model an index/summary and let it fetch details on demand (the same idea behind Skills and agentic search); (c) summarize/compact old conversation turns while preserving decisions and open items; (d) external memory/notes for long-running agents; (e) trim tool results (return only needed fields); (f) use token counting to pre-check size and costs. Long-context models still show quality variance across positions and distractors: measure with your own evals rather than assuming perfect recall. Rate limits are expressed in RPM, input and output tokens per minute, so token discipline is also a throughput issue.
5. Steps. (1) Measure token breakdown per request (system, tools, retrieved docs, history, output). (2) Set per-component budgets. (3) Replace broad retrieval with targeted retrieval/rerank. (4) Add compaction thresholds. (5) Monitor tokens per request in production (D4).
6. Diagram.
flowchart TD
Q[Task] --> I[Small index / summary in context]
I --> D{Need detail?}
D -- yes --> T[Tool: fetch section / search]
T --> I
D -- no --> A[Answer]
H[Long history] --> C[Compaction: decisions + open items]
C --> I
7. Scenario. A 300-page policy manual: instead of sending all pages per query, keep a table of contents + per-section summaries in the cached prefix and fetch only relevant sections.
8. Recommended/alternatives. Progressive disclosure + retrieval. Alternative: monolithic context (simple, works when total content is small and static and you cache it). Change if corpus grows beyond budget or cost/latency SLO fails.
9. Security. Retrieval must apply the user's entitlements before content enters context; a model cannot "un-see" data it was given.
10. Cost/latency. Tokens drive both. Prefix caching rewards stable, front-loaded content.
11. Mistakes. Stuffing the window "because it fits"; letting tool outputs balloon; forgetting output tokens count toward limits and cost; summaries that drop constraints.
12. Scenario question. An agent's performance degrades after 40 turns; logs show 90% of tokens are old tool outputs. → Clear/trim stale tool results and compact history; don't just switch to a bigger window.
13. Revision. Budget by component; retrieve, don't dump; disclose progressively; compact safely; entitlement-filter before context.
14. Docs. Context windows, token counting, rate limits (docs.claude.com/en/api/rate-limits).
2.4 Prompt reuse: prompt caching, modular prompts, and Skills
1. Objectives. Apply prompt caching correctly; structure prompts as reusable modules; explain what Agent Skills are and when to use them.
2. Prerequisites. 2.2–2.3.
3. Beginner. If many requests share the same long prefix (system prompt, policy, tools, examples), prompt caching lets the platform reuse it — lowering cost and time-to-first-token. Modular prompts are assembled from versioned parts. Skills are folders of instructions/resources that Claude loads when relevant.
4. Advanced — caching. You mark cache breakpoints with cache_control (type ephemeral) on content blocks; the cached prefix follows the request order of tools → system → messages, so stable content must come first and dynamic content last. Reads are billed at a fraction of normal input price; writes cost a premium; default lifetime is short (minutes) with a longer option available; there is a minimum cacheable length that varies by model, so short prompts silently don't cache. A cache hit requires an identical prefix — a changing timestamp or reordered tools at the top invalidates it. Track cache_creation_input_tokens and cache_read_input_tokens in usage to verify hits. (Confirm TTL options, multipliers, and minimums in the current caching docs.)
Advanced — Skills. Agent Skills package instructions, scripts, and reference files; Claude sees only lightweight metadata (name + description) up front and loads the full skill and its files on demand (progressive disclosure), keeping context small. Skills are portable across Claude surfaces (API, Claude Code, apps) and are versionable and shareable via source control. Use skills for repeatable procedures/standards; use caching for large static context you send every call; use retrieval for large, changing knowledge.
5. Steps. (1) Order prompt: tools, static system, static docs/examples, then dynamic user content. (2) Add cache_control at the end of the static block. (3) Verify via usage fields. (4) Keep volatile values (time, IDs) out of the prefix. (5) For shared procedures, create a skill with a precise description.
6. Executable example:
# EXECUTABLE — caching a long static policy (must exceed the model's minimum cacheable length)
resp = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=500,
system=[
{"type": "text", "text": SHORT_ROLE_INSTRUCTIONS},
{"type": "text", "text": LONG_POLICY_TEXT, "cache_control": {"type": "ephemeral"}},
],
messages=[{"role": "user", "content": user_question}],
)
print(resp.usage.cache_creation_input_tokens, resp.usage.cache_read_input_tokens)
7. Scenario (mirrors official Sample 2's logic). 8,000-token static prompt + short user turn; cost and latency both matter → static first + caching. Truncation loses policy; smallest-model-regardless risks quality; few-shot relocation doesn't create a cacheable prefix.
8. Recommended/alternatives. Cache static prefix; skills for procedures. Alternative: batch API (50% cheaper but async, up to 24 h — wrong for interactive); fine-tune (not about reuse). Change if traffic is too sparse to keep the cache warm (cost of writes > savings) → consider longer TTL or drop it.
9. Security. Cached content is still sensitive data — treat per your data policy. Skills can contain executable scripts: review, version, and restrict sources like any code dependency.
10. Cost/latency. Highest-leverage optimization for repeated large prefixes; monitor hit rate.
11. Mistakes. Dynamic content at the top; below-minimum length; expecting cache across different prefixes/models; huge skill descriptions that bloat startup context; vague skill descriptions that never trigger.
12. Scenario question. Cache read tokens are always 0. → Check minimum length, identical prefix (no timestamps up front), cache_control placement, and TTL expiry.
13. Revision. Static-first; breakpoints; verify via usage; skills = progressive disclosure; modules versioned.
14. Docs. Prompt caching: https://docs.claude.com/en/docs/build-with-claude/prompt-caching · Agent Skills overview (Claude API docs) · Batch processing: https://platform.claude.com/docs/build-with-claude/batch-processing