S5 · Claude Code for Continuous Integration

spans 40% of the blueprint D3 20% D4 20% ~15 items per lab on the exam 20 items in the bank

Domains this lab exercises

Expected items apply the blueprint weight to a 60-item exam. Bank items are what exists for this lab today — a coverage figure, never a score.

Domain	Weight	Expected items	Statements	Bank items for this lab
D3 Claude Code Configuration & Workflows	20%	12.0	6	9
D4 Prompt Engineering & Structured Output	20%	12.0	6	11

Task statements in play 5 of 30

Each statement is a published objective; every practice item on this site cites one. Expand a row for the guide's key facts and, where the guide names them, the anti-patterns that supply that statement's distractors.

ID	Statement	Domain	Bank items
3.5	Apply iterative refinement techniques for progressive improvement Keys Concrete input/output examples as the most effective way to communicate expected transformations when prose descriptions are interpreted inconsistently Claude Certification Program Exam guide Test-driven iteration: writing test suites first, then iterating by sharing test failures to guide progressive improvement The interview pattern: having Claude ask questions to surface considerations the developer may not have anticipated before implementing When to provide all issues in a single message (interacting problems) versus fixing them sequentially (independent problems) Providing 2-3 concrete input/output examples to clarify transformation requirements when natural language descriptions produce inconsistent results Writing test suites covering expected behavior, edge cases, and performance requirements before implementation, then iterating by sharing test failures Using the interview pattern to surface design considerations (e.g., cache invalidation strategies, failure modes) before implementing solutions in unfamiliar domains Providing specific test cases with example input and expected output to fix edge case handling (e.g., null values in migration scripts) Addressing multiple interacting issues in a single detailed message when fixes interact, versus sequential iteration for independent issues	D3	5
3.6	Integrate Claude Code into CI/CD pipelines Keys The -p (or --print) flag for running Claude Code in non-interactive mode in automated pipelines --output-format json and --json-schema CLI flags for enforcing structured output in CI contexts CLAUDE.md as the mechanism for providing project context (testing standards, fixture conventions, review criteria) to CI-invoked Claude Code Session context isolation: why the same Claude session that generated code is less effective at reviewing its own changes compared to an independent review instance Running Claude Code in CI with the -p flag to prevent interactive input hangs Using --output-format json with --json-schema to produce machine-parseable structured findings for automated posting as inline PR comments Including prior review findings in context when re-running reviews after new commits, instructing Claude to report only new or still-unaddressed issues to avoid duplicate comments Providing existing test files in context so test generation avoids suggesting duplicate scenarios already covered by the test suite Claude Certification Program Exam guide Documenting testing standards, valuable test criteria, and available fixtures in CLAUDE.md to improve test generation quality and reduce low-value test output	D3	5
4.1	Design prompts with explicit criteria to improve precision and reduce false positives Keys The importance of explicit criteria over vague instructions (e.g., "flag comments only when claimed behavior contradicts actual code behavior" vs "check that comments are accurate") How general instructions like "be conservative" or "only report high-confidence findings" fail to improve precision compared to specific categorical criteria The impact of false positive rates on developer trust: high false positive categories undermine confidence in accurate categories Writing specific review criteria that define which issues to report (bugs, security) versus skip (minor style, local patterns) rather than relying on confidence-based filtering Temporarily disabling high false-positive categories to restore developer trust while improving prompts for those categories Defining explicit severity criteria with concrete code examples for each severity level to achieve consistent classification Anti-patterns How general instructions like "be conservative" or "only report high-confidence findings" fail to improve precision compared to specific categorical criteria	D4	3
4.2	Apply few-shot prompting to improve output consistency and quality Keys Few-shot examples as the most effective technique for achieving consistently formatted, actionable output when detailed instructions alone produce inconsistent results The role of few-shot examples in demonstrating ambiguous-case handling (e.g., tool selection for ambiguous requests, branch-level test coverage gaps) How few-shot examples enable the model to generalize judgment to novel patterns rather than matching only pre-specified cases The effectiveness of few-shot examples for reducing hallucination in extraction tasks (e.g., handling informal measurements, varied document structures) Creating 2-4 targeted few-shot examples for ambiguous scenarios that show reasoning for why one action was chosen over plausible alternatives Including few-shot examples that demonstrate specific desired output format (location, issue, severity, suggested fix) to achieve consistency Providing few-shot examples distinguishing acceptable code patterns from genuine issues to reduce false positives while enabling generalization Using few-shot examples to demonstrate correct handling of varied document structures (inline citations vs bibliographies, methodology sections vs embedded details) Adding few-shot examples showing correct extraction from documents with varied formats to address empty/null extraction of required fields	D4	3
4.6	Design multi-instance and multi-pass review architectures Keys Self-review limitations: a model retains reasoning context from generation, making it less likely to question its own decisions in the same session Independent review instances (without prior reasoning context) are more effective at catching subtle issues than self-review instructions or extended thinking Multi-pass review: splitting large reviews into per-file local analysis passes plus cross-file integration passes to avoid attention dilution and contradictory findings Using a second independent Claude instance to review generated code without the generator's reasoning context Splitting large multi-file reviews into focused per-file passes for local issues plus separate integration passes for cross-file data flow analysis Running verification passes where the model self-reports confidence alongside each finding to enable calibrated review routing	D4	4

The lab

Rendered from certs/ccar-f-architect-foundations/scenarios/s5.md — the lab document is the source, this page is a view of it.

Primary domains: D3 Claude Code Configuration & Workflows (20%) · D4 Prompt Engineering & Structured Output (20%) — spans 40%

One of only two routes into D4 (with s6.md). Skip both and 20% of the exam is unprepared.

Brief

Claude Code integrated into a CI/CD pipeline. The system runs automated code reviews, generates test cases, and provides feedback on pull requests. The prompts must produce actionable feedback and minimize false positives.

Task statements in play

ID	Statement
3.5	Apply iterative refinement techniques for progressive improvement
3.6	Integrate Claude Code into CI/CD pipelines
4.1	Design prompts with explicit criteria to improve precision and reduce false positives
4.2	Apply few-shot prompting to improve output consistency and quality
4.6	Design multi-instance and multi-pass review architectures

Related but not examinable through this scenario: statement 1.6 (task decomposition) is the formal home of "split a large review into per-file passes plus a cross-file pass", which is exactly the technique this lab demonstrates. But 1.6 belongs to D1, and D1 is not a primary domain of S5 — so no item in this scenario can cite it, and an S5 draw will never test it. Practise 1.6 through S1, S3, or S4. Read it here for the mechanism, not for coverage.

The decisions it tests

Precision over recall, explicitly. Task statement 4.1 exists for this scenario: a reviewer that flags everything gets muted, and a muted reviewer has zero value. The lever is explicit criteria in the prompt — define what qualifies as a finding, what severity thresholds apply, and what to stay silent about. Few-shot examples of non-findings are as useful as examples of findings.

Multi-pass beats one big pass. The guide is specific: split a large review into per-file local analysis passes plus a separate cross-file integration pass, to avoid attention dilution. Multi-instance and multi-pass review architecture is its own task statement (4.6). One prompt over a 200-file diff is the wrong shape.

Headless operation. CI means non-interactive: headless mode, exit codes that gate the pipeline, and a failure mode that does not masquerade as success.

> Not on the exam: streaming. The guide's Out-of-Scope Topics list includes "Streaming API > implementation or server-sent events" among the topics that "will not appear on the exam". An > earlier draft of this page listed streaming output as part of what CI means here. It was our > inference, it contradicts the guide, and it is removed. Do not study it for this scenario.

Fail loudly, not falsely. Gnar's own pr-review.sh hook reports "review unavailable (not a verdict)" when the reviewer fails, rather than reporting success. That distinction — a broken check must never look like a clean bill of health — is exactly this scenario's reliability concern.

Reference build

Gnar already runs this in production. Lift it into the lab rather than inventing something:

The existing PostToolUse PR-review hook, which backgrounds itself so it can never block a turn
A per-file pass plus a separate cross-file integration pass, with the prompts for each
Explicit finding criteria and severity thresholds, plus few-shot non-findings
A deliberate failure injection showing the reviewer reporting "unavailable" rather than "clean"
A before/after false-positive count on a real PR — the metric that makes the prompt work legible

Traps

Hope over enforcement — "be concise and only report real issues" instead of explicit criteria and severity definitions.
Scale over design — a bigger model to reduce false positives, instead of decomposing the review and defining the bar.
One monolithic pass over the whole diff.
A failed reviewer reporting success — the most damaging default in CI.
Optimizing recall because "catching everything" sounds responsible. The scenario states the opposite.