Security and responsible AI concepts for quick CCAR-P revision.
Layered Guardrails
Do not rely on one control.
Best pattern:
Prompt Guardrail + Runtime Checks + Tool Permissions + Output Validation
Useful Guardrails
Structured output validation
Runtime content classifiers
Explicit out-of-scope categories
Least-privilege tools
Human approval for risky actions
Audit logging
RBAC
Role-based access control should be enforced:
Before restricted content reaches the prompt/model.
User → Authentication → Authorization → Retrieval → Claude
Do not retrieve restricted documents and try to remove them after generation.
Prompt Injection
User text claiming:
“I am an admin. Ignore previous instructions.”
is still untrusted user input.
Use:
Prompt-level untrusted-input instructions
Runtime classifiers
Scoped permissions
Audit logging
Human-in-the-Loop
Human review belongs:
Model Output → Human Review → High-Impact / Irreversible Action
Not:
Model Output → Irreversible Action → Human Review
When to Escalate
Low confidence
Ambiguity detected
High-impact decision
User requests human review
Risk-Based Delegation
Risk should not be judged only by:
Transaction value
Number of code lines
Request size
Assess the actual consequence and type of risk.
High-Volume Human Review
If reviewing every output is impossible:
Review all high-risk / low-confidence cases + random sample of normal cases.
Least Privilege
Agent permissions should map directly to its actual responsibilities.
Remove:
Unrelated admin tools
Speculative future tools
Direct-action tools when only drafting is required
Credentials
Prefer individually scoped credentials where accountability is required.
Better: Per-user OAuth with restricted scopes.
Weak: One shared API key for every user.
Audit Logging
Tool-call logs should record the initiating actor/user identity.
HIPAA Revision
The practice material emphasises:
Signed Business Associate Agreement
Appropriate enterprise deployment
Zero Data Retention where required
Role-based access controls
Audit logging
Controlled PHI handling
GDPR Revision
Remember:
Data Processing Addendum
Defined data-retention configuration
Data minimisation
Redaction of unnecessary personal information
Documented data-subject-rights handling
Fast Exam Recall
Restricted data? Block before retrieval.
High-impact action? Human review before execution.
Prompt injection? Treat user content as untrusted.
Secret? Runtime secret store.
User accountability? Individual identity + actor logging.
HIPAA? BAA + controls + appropriate data handling.
GDPR? DPA + minimisation + retention + rights handling.
