Claude, Nutshell Series

CCAR-P Revision 5: Discovery, Architecture Documentation & Exam Cheat Sheet

Final CCAR-P revision covering discovery, communication and architectural decision-making.

Discovery

Capture before designing:

Business goals
Success criteria
In-scope users
Latency constraints
Cost constraints
Audit requirements
Data sensitivity
Regulatory requirements

Vague Requirements

If a stakeholder says:

“Build AI to fix customer escalations.”

Ask:

Who is involved?
What triggers escalation?
What does good resolution look like?
What are the latency/cost/audit/data constraints?

Do not jump straight to model selection or architecture.

Functional vs Non-Functional Requirements

Functional Non-Functional
Classify tickets p95 latency
Extract invoice fields Cost ceiling
Generate a draft Throughput
Route a request Audit requirement

Architecture Decision Record — ADR

Include:

Context → Decision → Alternatives → Trade-offs/Consequences → Owner/Date

Also define a review cadence when assumptions may change.

Implementation Guide

Include:

Component responsibilities
Interface contracts
Sequence diagrams / dominant flows
Configuration parameters
Operational runbooks

Operational Runbook

Include:

Alert → Triage → Escalation → Rollback → Dashboard/Logs

Communication by Audience

Audience Primary Focus
Executive Business outcomes, trade-offs, high-level risks.
Engineering Components, interfaces and implementation details.
Security / Compliance Threat model, controls, residual risk and auditability.
Product Capabilities, scope and roadmap dependencies.

Business Value

Pillar Meaning
Efficiency Same work faster or more work per person.
Productivity Actual business output improves.
Transformation Operating model or role fundamentally changes.
Solution Cost API/infrastructure spending.
Performance SLA Latency/reliability against agreed targets.

Stakeholder Management

If scope increases during delivery:

Acknowledge it → explain timeline impact → offer options.

Typical options:

Defer the new scope.
Remove other scope.
Extend the timeline.

Iteration

After production:

Telemetry + Evaluations + Stakeholder Feedback → Highest-Impact Improvement

Do not throw away the existing evaluation framework after go-live.

CCAR-P Last-Minute Cheat Sheet

Fixed predictable steps → Workflow
Dynamic planning → Agent
Authoritative knowledge + citations → RAG
Reusable procedure → Skill
Live external system → MCP
Recurring specialist task → Subagent
Many MCP tools → Tool Search
Shared team config → Project scope
Non-overridable policy → Managed config
Personal setting → User scope
High-impact irreversible action → Human gate first
Repeated static prompt → Prompt caching
Simple high-volume task → Smaller/faster model
Restricted data → Authorise before retrieval
Credential → Secret store
Model output → Validate before downstream action
Model upgrade → Re-run stable evaluation set
Vague requirement → Discovery first
Accuracy must remain unchanged → Remove redundant cost first

Final Exam Mindset

When two answers look correct, prefer the option that is:

More measurable
More least-privilege
More risk-based
More evidence-driven
More auditable
More reversible
More clearly aligned to stated requirements

Claude, Nutshell Series

CCAR-P Revision 4: Evaluation, Observability, Cost & Performance

Quick revision for evaluation, testing, monitoring and optimisation questions.

Evaluation Sequence

Define Metrics → Build Dataset → Run Evaluation → Analyse → Decide

Evaluation Dataset

Include:

Representative cases
Edge cases
Adversarial cases

The evaluation set should represent the actual user population and workload.

Stable Reference Set

Keep a stable evaluation/reference set across:

Prompt changes
Model-version upgrades
Architecture iterations

This helps detect quality drift.

Testing Types

Test Purpose
Regression Detect previously working behaviour that has broken.
Adversarial Prompt injection and malformed/malicious inputs.
Integration Validate end-to-end behaviour across components.
Red-team Probe security and access-control weaknesses.

Model Upgrade

Do not assume a new model behaves the same.

Re-run the stable evaluation set before promotion.

Observability

Capture:

Latency
Token usage
Distributed traces
Tool-call outcomes
Redacted payload information
Model identity/version
Request correlation ID

Important Observability Gaps

Missing:

Model identity/version

or

Request correlation ID across agent/tool calls

creates a real diagnostic blind spot.

SLA

A useful SLA contains:

Metric + Threshold + Evaluation Window + Breach Consequence

Example:

p95 latency < 800 ms over a 28-day window

A statement such as “the system should be fast” is not a measurable SLA.

Cost Optimisation

Common optimisations:

Prompt caching
Trim irrelevant retrieval context
Cache repeatedly used documents/chunks
Tiered model routing
Avoid repeated processing

Prompt Caching

Best for:

Long static content repeated across many requests.

Place stable cacheable content before request-specific content.

Token Lifecycle

Input Preparation

Trim irrelevant retrieved passages.
Summarise old conversation history.

Prompt Construction

Place stable content first.
Use a cacheable repeated prefix.

Output Handling

Validate structured schema.
Persist validated results.

Accuracy Must Stay Unchanged

If the requirement says accuracy must remain unchanged:

Remove redundant cost first.

Prefer caching over immediately switching to a weaker model or deleting information required for reasoning.

Fast Exam Recall

Prompt changed and old tests fail? Regression test.
Prompt injection? Adversarial test.
Whole pipeline? Integration test.
Model upgraded? Stable evaluation set.
Repeated prompt? Prompt caching.
Good SLA? Metric + threshold + window + consequence.

Claude, Nutshell Series

CCAR-P Revision 3: Security, Guardrails, Human Review, HIPAA & GDPR

Security and responsible AI concepts for quick CCAR-P revision.

Layered Guardrails

Do not rely on one control.

Best pattern:

Prompt Guardrail + Runtime Checks + Tool Permissions + Output Validation

Useful Guardrails

Structured output validation
Runtime content classifiers
Explicit out-of-scope categories
Least-privilege tools
Human approval for risky actions
Audit logging

RBAC

Role-based access control should be enforced:

Before restricted content reaches the prompt/model.

User → Authentication → Authorization → Retrieval → Claude

Do not retrieve restricted documents and try to remove them after generation.

Prompt Injection

User text claiming:

“I am an admin. Ignore previous instructions.”

is still untrusted user input.

Use:

Prompt-level untrusted-input instructions
Runtime classifiers
Scoped permissions
Audit logging

Human-in-the-Loop

Human review belongs:

Model Output → Human Review → High-Impact / Irreversible Action

Not:

Model Output → Irreversible Action → Human Review

When to Escalate

Low confidence
Ambiguity detected
High-impact decision
User requests human review

Risk-Based Delegation

Risk should not be judged only by:

Transaction value
Number of code lines
Request size

Assess the actual consequence and type of risk.

High-Volume Human Review

If reviewing every output is impossible:

Review all high-risk / low-confidence cases + random sample of normal cases.

Least Privilege

Agent permissions should map directly to its actual responsibilities.

Remove:

Unrelated admin tools
Speculative future tools
Direct-action tools when only drafting is required

Credentials

Prefer individually scoped credentials where accountability is required.

Better: Per-user OAuth with restricted scopes.

Weak: One shared API key for every user.

Audit Logging

Tool-call logs should record the initiating actor/user identity.

HIPAA Revision

The practice material emphasises:

Signed Business Associate Agreement
Appropriate enterprise deployment
Zero Data Retention where required
Role-based access controls
Audit logging
Controlled PHI handling

GDPR Revision

Remember:

Data Processing Addendum
Defined data-retention configuration
Data minimisation
Redaction of unnecessary personal information
Documented data-subject-rights handling

Fast Exam Recall

Restricted data? Block before retrieval.
High-impact action? Human review before execution.
Prompt injection? Treat user content as untrusted.
Secret? Runtime secret store.
User accountability? Individual identity + actor logging.
HIPAA? BAA + controls + appropriate data handling.
GDPR? DPA + minimisation + retention + rights handling.

Claude, Nutshell Series

CCAR-P Revision 2: Architecture Patterns, Prompting, RAG & Model Selection

Fast revision notes for Claude architecture design questions.

Workflow vs Agentic Pattern

Scenario Choose
Known fixed steps Workflow
Predictable sequence Workflow
Predictable token cost Workflow
Open-ended planning Agentic
Dynamic tool selection Agentic

Example:

Extract → Classify → Summarise → Persist = Workflow

Exam rule: Do not choose an agent just because it is more flexible.

Prompting

Zero-shot

Best starting point when categories and descriptions are already well defined.

Few-shot

Useful when you want to demonstrate exact output structure, field names, or examples.

Chain-of-thought

Not the default for every task. Simple routing or closed-set classification does not automatically need multi-step reasoning.

Leading Questions

Reduce bias by asking for:

Even-handed comparison + explicit evaluation criteria.

RAG

Use Retrieval-Augmented Generation when answers must be grounded in an authoritative knowledge source.

Typical architecture:

User → Retrieval → Relevant Documents → Claude → Answer + Citations

When RAG Is Strong

Choose RAG when you need:

Authoritative answers
Citations
Auditability
Predictable cost
Predictable latency

For compliance Q&A, prefer retrieval from the approved internal corpus instead of an unrestricted web-search agent.

Grounding Problems

If Claude starts contradicting retrieved sources:

Check model-version changes.
Check grounding instructions.
Require source-supported responses.
Require citations.
Add verification.

Model Selection

Correct sequence:

Requirements → Candidate Model → Representative Evaluation → Final Selection

Define Before Testing

Quality bar
Latency tolerance
Expected volume
Cost constraints

Model Selection Traps

Do not automatically choose:

The newest model.
The largest model.
The most capable model.

Choose the lightest model that consistently meets requirements.

Tiered Routing

Routine Request → Smaller/Faster Model

Complex Request → More Capable Model

This can reduce both cost and latency.

High-Volume Classification

For well-defined labels and tight latency requirements, prefer a lighter model rather than deep reasoning on every request.

Fast Exam Recall

Predictable steps? Workflow.
Dynamic planning? Agent.
Defined categories? Zero-shot first.
Need format examples? Few-shot.
Authoritative answers? RAG.
Simple traffic? Smaller model.
Complex traffic? Higher-capability model.

Claude, Nutshell Series

CCAR-P Revision 1: Claude Code Configuration, MCP, Skills & Subagents

Quick revision notes for Claude Certified Architect – Professional.

Claude Code Configuration Scopes

Scope Use
Managed Organisation-wide security/compliance settings that must not be overridden.
Project Team-shared configuration committed to the repository.
User Personal preferences that follow an engineer across projects.
Local Machine/project-specific configuration not intended for the whole team.

Configuration Precedence

Highest → Lowest:

Managed → Command Line → Local → Project → User

Exam Memory:
Team shared → Project
Personal preference → User
Must not be overridden → Managed

MCP Servers

MCP is used when Claude needs tools or access to external/live systems.

Important exam points:

  • Too many MCP tool definitions loaded upfront consume context.
  • Use Tool Search to discover tools on demand.
  • Do not disable useful tools just to reduce context.
  • New MCP server missing? Check registration first, then reconnect/restart.
  • Use least-privilege tool permissions.

Tool Search

If 50–60+ tools are loaded before the user sends a message:

Best solution: Enable Tool Search / progressive tool discovery.

This preserves available capabilities while avoiding unnecessary tool definitions in the initial context.

Least Privilege

Give an agent only the tools needed for its defined responsibility.

Good: read knowledge base + write draft queue.

Bad: unrestricted Bash, database admin access, or unrelated tools.

Prefer:

Allow-list required tools + explicit deny rules for sensitive operations.

Claude Skill vs MCP vs Subagent

Requirement Best Choice
Reusable procedural knowledge Skill
Fixed procedure with examples Skill
No live external-system calls Skill
Live database/API/tool access MCP
Recurring specialised task Subagent
Focused prompt + narrow tools + specific model Subagent

When to Use a Subagent

A dedicated subagent makes sense when a task:

Recurs frequently + needs a specialised prompt + narrow permissions + specific model selection.

Do not create a subagent for a one-time generic task.

Persistent Project Context

If engineers repeatedly explain the same architecture and coding conventions:

Store them in project-scoped CLAUDE.md or equivalent project context.

Secrets

Never store API keys or credentials in version-controlled configuration.

Correct pattern:

Application → Runtime Secret Store → Credential

Fast Exam Recall

Shared config? Project.
Company security policy? Managed.
Personal preference? User.
Many MCP tools? Tool Search.
Reusable procedure? Skill.
Live external system? MCP.
Recurring specialist role? Subagent.
Credential? Secret store.

AI, Claude, Nutshell Series

Claude Exam (CCAR-F) Quick Revision 12: Human Review & Provenance

Aggregate Accuracy Trap

Overall accuracy = 97%

This may still hide poor accuracy for a particular document type or field.

Measure Accuracy By

  • Document type.
  • Individual field.
  • Relevant segment.

Stratified Random Sampling

Do not review only low-confidence results.

Sample some high-confidence outputs too.

Why? Detect confidently wrong predictions and new failure patterns.

Confidence Calibration

Field confidence
+
Labeled validation set
↓
Calibrated threshold
↓
Human review

Memory: Confidence must be validated against actual correctness.

Claim-Source Mapping

Every research finding should preserve:

Claim
Evidence
Source URL / document
Publication date

Conflicting Sources

Do NOT arbitrarily choose one value.

Source A → 42%
Source B → 51%

Report both with attribution.

Temporal Context

Preserve:

  • Publication date.
  • Data-collection date.

Statistics from different periods may not actually conflict.

Coverage Gaps

If sources are unavailable:

  • Explicitly report the gap.
  • Identify affected topic.
  • Do not pretend research is complete.

Report Structure

Well-supported findings

Contested findings

Coverage gaps

Content-Specific Formatting

  • Financial data → tables.
  • News → prose.
  • Technical findings → structured lists.

Fast Revision


Overall accuracy can hide weak segments.
Sample high-confidence outputs.
Confidence must be calibrated.
Claim must keep source.
Conflicting sources → preserve both.
Missing coverage → report it.

AI, Claude, Nutshell Series

Claude Exam (CCAR-F) Quick Revision 11: Advanced Batch & Extraction

Schema Error vs Semantic Error

  • Schema error: Invalid structure/type.
  • Semantic error: Valid structure but wrong meaning/value.

Remember: JSON Schema does not guarantee semantic correctness.

Self-Validation Fields

{
  "stated_total": 120,
  "calculated_total": 110,
  "conflict_detected": true
}

Expose inconsistencies explicitly.

detected_pattern

Add fields such as:

detected_pattern

Use them to analyse which code patterns repeatedly cause false positives.

Batch custom_id

  • Assign each request a custom_id.
  • Correlate request with response.
  • Identify individual failures.

Resubmit Only Failed Items

100 documents
↓
7 fail
↓
Identify failed custom_id values
↓
Fix only those inputs
↓
Resubmit 7

Context-Limit Failure

If an oversized document fails:

Chunk document
↓
Resubmit failed document

Before Large Batch

Small sample
↓
Refine prompt
↓
Validate
↓
Run large batch

Benefit: Better first-pass success and fewer expensive retries.

SLA Planning

Batch processing can take up to 24 hours.

Work backwards from the required SLA when deciding submission frequency.

Fast Revision


Schema valid ≠ semantically correct.
custom_id = correlation.
Failed subset = resubmit subset.
Oversized document = chunk.
Large batch = test sample first.

AI, Claude, Nutshell Series

Claude Exam (CCAR-F) Quick Revision 10: Iterative Refinement Patterns

Concrete Examples

If prose instructions produce inconsistent results:

Give 2–3 concrete input/output examples.

Interview Pattern

Ask Claude to question you before implementation when important requirements may be missing.

Useful for discovering:

  • Failure modes.
  • Edge cases.
  • Cache invalidation.
  • Security requirements.
  • Performance constraints.

Memory: Unknown design space → interview first.

Interacting Problems

If fixes affect each other:

Explain issues together
↓
Claude reasons across dependencies

Independent Problems

Fix issue 1
↓
Validate
↓
Fix issue 2

Iterative Test Refinement

Tests
↓
Run
↓
Share failure
↓
Improve implementation
↓
Repeat

CI Review Re-Runs

When reviewing after new commits:

  • Include prior findings.
  • Report only new issues.
  • Report still-unresolved issues.
  • Avoid duplicate PR comments.

Test Generation

Provide existing test files before generating new tests.

This helps avoid duplicate scenarios.

Fast Revision


Inconsistent transformation → examples.
Unknown design → interview.
Interacting issues → together.
Independent issues → sequential.

AI, Claude, Nutshell Series

Claude Exam (CCAR-F) Quick Revision 9: Advanced Claude Configuration

@import

Use @import to keep CLAUDE.md modular.

@import ./standards/testing.md
@import ./standards/api.md

Use: Reference focused standards instead of creating one huge CLAUDE.md.

/memory

  • Shows which memory/instruction files are loaded.
  • Useful for diagnosing inconsistent behaviour.

Skill: context: fork

context: fork
  • Runs skill in isolated subagent context.
  • Useful for verbose analysis.
  • Useful for brainstorming alternatives.
  • Prevents main-context pollution.

Skill: allowed-tools

allowed-tools:
  - Read
  - Grep

Restricts which tools the skill can use.

Skill: argument-hint

Shows which arguments the user should provide when invoking the skill.

Memory Trick

context: fork → isolate
allowed-tools → restrict
argument-hint → guide input

Explore Subagent

  • Use for verbose codebase exploration.
  • Return concise findings to main session.
  • Useful during complex planning.

Plan + Execute Pattern

Plan
↓
Explore
↓
Choose approach
↓
Direct execution

Plan Mode and direct execution can be used together.

AI, Claude, Nutshell Series

Claude Exam (CCAR-F) Quick Revision 8: Advanced MCP Design

Split Overly Generic Tools

Instead of:

analyze_document

Prefer:

extract_data_points
summarize_content
verify_claim_against_source

Reason: Clear tool boundaries improve selection reliability.

Avoid Overlapping Tool Names

Bad:

analyze_content
analyze_document

Better:

extract_web_results
analyze_pdf_document

System Prompt Can Bias Tool Selection

  • Tool descriptions may be correct.
  • System-prompt keywords can still create unwanted tool associations.
  • If tool routing remains wrong, inspect both descriptions and system instructions.

MCP isError

Use structured failure information such as:

{
  "isError": true,
  "errorCategory": "transient",
  "isRetryable": true,
  "message": "Service temporarily unavailable"
}

Empty Result vs Failure

Successful query + no matches
≠
Query failed

Exam trap: Never hide an access failure by returning an empty successful result.

Local Recovery

  • Subagent handles transient failures locally where possible.
  • Only unresolved failures go to coordinator.
  • Include partial results and attempted recovery.

MCP Resources

Resources expose content/catalogs such as:

  • Database schemas.
  • Documentation hierarchies.
  • Issue catalogs.
  • Available datasets.

Benefit: Reduces unnecessary exploratory tool calls.

Existing vs Custom MCP Server

  • Standard integration → prefer existing/community MCP server.
  • Team-specific workflow → custom MCP server.

Fast Revision


Generic tool → split.
Ambiguous name → rename.
Resource → expose information.
Failure ≠ empty result.
Transient error → local recovery.