Part 7 — Domain 7: Developer Productivity & Operational Enablement (7%)
Prep-course lesson: Team Enablement & Operational Productivity (45 min). Official tasks: configure Claude tools and environments for teams (e.g., Claude Code) · improve developer workflows using AI-assisted tooling · support debugging and operational issue resolution.
Topics: 7.1 Configuring Claude tools for teams · 7.2 Developer workflows and AI-assisted tooling · 7.3 Operations: runbooks, troubleshooting, incident ownership · 7.4 Onboarding, versioning, documentation standards, and continuous improvement
Verify Claude Code specifics (file names, settings keys, hook events) against the current Claude Code docs at https://code.claude.com/docs — they evolve. The exam tests what to standardize and why.
7.1 Configuring Claude tools and environments for teams
1. Objectives. Standardize shared configuration (project memory/instructions, permissions, skills, MCP servers, hooks) so behavior is consistent, reviewable, and secure across a team.
2. Prerequisites. Claude Code in Action course; D3 tool/MCP concepts.
3. Beginner. Instead of every developer configuring Claude differently, the team keeps shared settings in the repository (version controlled), so everyone gets the same rules, tools, and conventions.
4. Advanced. Typical layers in Claude Code (names per current docs; verify): project instructions file (CLAUDE.md, with user-level and directory-level variants forming a hierarchy) holding build/test commands, architecture notes, code style, and "do/don't" rules; settings with permission rules (allow/ask/deny for tools and commands — e.g., deny reading secrets, require approval for destructive commands); MCP server configuration shared at project scope (reviewed and pinned; credentials via environment, never committed); skills (project-scoped skills shared via version control; personal skills for individual use); hooks (deterministic scripts at lifecycle points — e.g., run formatter/linter, block dangerous commands — because hooks enforce where instructions only request); subagents/commands for repeatable roles and workflows; plugins to distribute bundles. Governance: managed/organization-level settings for security-critical policy; least-privilege defaults; allow-list MCP servers and skill sources; code-review changes to shared config like code.
5. Steps. (1) Create a project instructions file with commands, conventions, and architecture overview (keep it concise; link details). (2) Define permission policy (deny secrets, ask for writes/network). (3) Add team skills for recurring procedures (code review checklist, release notes). (4) Add hooks for enforced checks. (5) Pin and review MCP servers. (6) PR-review all config changes; document how to override locally.
6. Example instructions file (illustrative):
# Project: claims-assistant
## Commands
- Test: `pytest -q` · Lint: `ruff check .` · Evals: `make eval` (must pass before PR)
## Architecture
- app/ (API), pipelines/ (RAG ingest), prompts/ (versioned), evals/ (datasets + graders)
## Rules
- Never edit `prompts/` without running `make eval`. Prompts are versioned (see CHANGELOG).
- Do not read `.env` or `secrets/`. Use the tool broker for all backend calls.
- Write tests for every new tool; validate all tool arguments server-side.
7. Scenario. Five developers get inconsistent code quality and one accidentally runs a destructive command. Fix: shared permission rules (deny/ask), a lint/format hook, and a team skill for PR review.
8. Recommended/alternatives. Version-controlled shared config + enforced hooks + least-privilege permissions. Alternative: each dev self-configures (drift, unsafe defaults); giant instruction files (bloat; tokens; ignored rules).
9. Security. Secrets via env/secret manager; permission denies for sensitive paths; vet third-party MCP servers/skills/plugins (supply chain); don't give agents prod credentials.
10. Cost/productivity. Concise instructions reduce tokens; skills load on demand; shared config reduces re-explaining.
11. Mistakes. Putting enforcement in prose instead of hooks/permissions; committing credentials; unreviewed MCP servers; instruction files that grow without pruning.
12. Scenario question. Team wants to guarantee tests run after every edit. → Use a hook (deterministic) rather than asking in the instructions file.
13. Revision. Shared, versioned, least-privilege config; hooks enforce; skills for procedures; vet dependencies.
14. Docs. Claude Code docs (settings, memory/CLAUDE.md, hooks, MCP, skills, plugins); Claude Code in Action course.
7.2 Improving developer workflows with AI-assisted tooling
1. Objectives. Identify high-leverage workflows (exploration, test generation, refactoring, review, docs, CI); define guardrails and measure productivity.
2. Prerequisites. 7.1.
3. Beginner. Use Claude where it removes toil — understanding unfamiliar code, writing tests, drafting changes — while humans review and own outcomes.
4. Advanced. Good patterns: explore → plan → implement → verify (ask for a plan first on non-trivial changes; run tests and linters to verify); tests as the objective (give Claude a runnable success check); small, reviewable diffs; context hygiene (point to specific files, keep sessions focused, reset when drifting); parallel sessions/subagents for independent tasks; CI integration (automated review comments, test-failure triage) with least-privilege tokens; headless/non-interactive modes for scripted tasks. Guardrails: human review of all merged code; no secrets in prompts; license/IP policy; protected branches; evals for AI-assisted pipelines. Measure: cycle time, PR size/review time, defect rate, rework, adoption, developer satisfaction — avoid vanity metrics (lines generated).
5. Steps. (1) Interview devs for top toil. (2) Pilot 2–3 workflows with baseline metrics. (3) Write skills/commands for each. (4) Add verification (tests/hooks). (5) Review metrics; expand or drop.
6. Diagram.
flowchart LR I[Issue] --> X[Explore code] --> P[Plan] --> IM[Implement] IM --> V[Verify: tests, lint, hooks] V -- fail --> IM V -- pass --> PR[PR + human review] --> M[Merge via CI gates]
7. Scenario. Legacy module with no tests: Claude generates characterization tests first, then refactors against them; reviewers check test intent.
8. Recommended/alternatives. Verify-driven workflows. Alternative: accept large unreviewed diffs.
9. Security/governance. Treat AI-authored code as untrusted until reviewed; run SAST/dependency checks; restrict agent credentials.
10. Cost. Track token spend per team; use cheaper tiers for mechanical tasks.
11. Mistakes. No verification; vague prompts; giant sessions; measuring volume not outcomes.
12. Scenario question. Developers merge Claude-generated changes with failing edge cases. → Require tests/CI gates and human review; give Claude the failing test as the objective.
13. Revision. Plan→implement→verify; tests as objective; small diffs; measure outcomes.
14. Docs. Claude Code docs (workflows, headless mode, CI integration).
7.3 Operations: runbooks, troubleshooting, and incident ownership
1. Objectives. Create runbooks; troubleshoot LLM-system incidents; assign ownership and escalation.
2. Prerequisites. D3.7, D4.4.
3. Beginner. When something breaks at 2 a.m., on-call needs a step-by-step guide and to know who owns what.
4. Advanced. Incident classes: quality regression (bad answers), safety/security event (injection success, data exposure), availability/latency (API errors, rate limits), cost spike, integration failure (tool/schema drift, expired tokens), data/index issue (stale retrieval). Each has triage steps: check recent changes → dashboards → traces → isolate component → mitigate (rollback prompt/model/config, disable tool, switch fallback, throttle) → communicate → root-cause → regression test → postmortem (blameless). Severity levels tied to impact; ownership matrix (RACI) covering prompts/config, retrieval/data, tools/integrations, platform/infra, safety/security, business owner. Kill switches for tools/features. Keep break-glass access audited.
5. Runbook template.
Runbook: Elevated wrong-answer reports
Owner: AI platform on-call | Severity guide: Sev2 if >2% flagged in 1h
1. Check changelog: any prompt/model/index/config change in last 24h? If yes → roll back; confirm.
2. Dashboards: retrieval hit rate, tool error rate, cache hit rate, token/request.
3. Pull 10 flagged traces: was correct context retrieved? tool args correct? output truncated?
4. Index issue? Run index health job; compare doc counts/versions; re-index affected sources.
5. Mitigation options: roll back prompt (cmd), switch to fallback model (flag), disable feature X (flag), route to human.
6. Communicate: status page + stakeholders every 30 min.
7. After: add failing cases to eval set; postmortem within 5 business days.
Escalate: security lead if data exposure suspected; business owner if customer impact > threshold.
6. Diagram.
flowchart TD A[Alert] --> T[Triage: severity + recent changes] T --> I[Isolate component via traces] I --> M[Mitigate: rollback / flag off / fallback] M --> C[Communicate] C --> R[Root cause + regression test] R --> P[Blameless postmortem + actions]
7. Scenario. Costs triple overnight: a prompt change moved examples above a cached prefix, destroying cache hits and increasing tokens. Runbook step "check changelog + cache hit rate" finds it in minutes.
8. Recommended/alternatives. Practiced runbooks with kill switches. Alternative: ad-hoc debugging in prod.
9. Governance. Security incidents follow the security incident-response process; preserve evidence.
10. Reliability. Measure MTTD/MTTR; run game days.
11. Mistakes. No named owner; no rollback path; fixing without adding a regression test; skipping postmortems.
12. Scenario question. Support reports wrong answers started yesterday; nothing was "deployed". → Check data/index refreshes, config flags, provider changes, and upstream data — "no deploy" isn't "no change".
13. Revision. Classify → triage → isolate → mitigate → communicate → root-cause → regress-test.
14. Docs. Prep Lesson 5; Anthropic status/service pages; your incident-management policy.
7.4 Onboarding, versioning, documentation standards, and continuous improvement
1. Objectives. Onboard teams, version prompts/config, set documentation standards, and run continuous improvement with change management.
2. Prerequisites. 7.1–7.3; D4.3.
3. Beginner. Treat prompts and configs like code: versioned, reviewed, tested, released.
4. Advanced. Versioning: prompts/templates, tool schemas, retrieval config, model IDs, eval datasets, and skills live in source control with semantic versions and a changelog; each release links to the eval run that approved it; deployments reference immutable versions (pin model IDs). Change management: PR review (including a domain expert for prompt changes), eval gate, canary, rollback; announce changes to support/ops. Documentation standards: README per component, ADRs, runbooks, data-flow diagrams, "how to add a tool/prompt/eval" guides, glossary; docs reviewed like code. Onboarding plan (first 30 days): week 1 overview + architecture walkthrough + sandbox access; week 2 shadow on-call, run evals locally; week 3 ship a small prompt/eval change through the gate; week 4 lead a runbook drill. Training and adoption: role-based sessions (developers, support, analysts), champions network, office hours, feedback channel, usage and satisfaction metrics. Continuous improvement: monthly scorecard review; failure-driven eval growth; deprecation calendar; quarterly architecture review; retire unused tools/prompts.
5. Steps. (1) Move prompts/configs into repo. (2) Add CI eval gates. (3) Write onboarding path and checklists. (4) Establish release cadence and change log. (5) Schedule reviews.
6. Diagram.
flowchart LR E[Edit prompt/config] --> PR[Pull request + reviewer] PR --> CI[CI: evals + safety + cost gates] CI --> RL[Release vX.Y + changelog] RL --> CN[Canary] --> PD[Production] PD --> FB[Feedback/incidents → new eval cases] FB --> E
7. Scenario. A well-meaning analyst edits a production prompt in a console; quality drops; nobody knows what changed. Fix: prompts only change via PR+gate; console edits disabled.
8. Recommended/alternatives. Everything-as-code with eval gates. Alternative: prompts in a shared doc or console (no audit, no rollback).
9. Governance. Change history is audit evidence; separation of duties for high-risk changes.
10. Cost/time. Gates add minutes but prevent expensive regressions.
11. Mistakes. Unversioned prompts; no pinned model IDs; docs rot; training once then never.
12. Scenario question. Two teams use different copies of "the" prompt and results diverge. → Single versioned source of truth with a release process.
13. Revision. Version everything; gate changes; onboarding path; owners; continuous improvement loop.
14. Docs. Prep Lesson 5; Exam Guide §6 D7.