Coding agents can produce changes faster than a team can review them line by line. That makes verification—not generation—the limiting factor in AI-assisted delivery. Existing tests help, but they may encode the same misunderstanding as the generated implementation or confirm only the easiest path through the code.
This course develops a layered verification strategy for AI-generated software. Participants will learn to derive tests independently from requirements, challenge test quality with mutation and property-based techniques, automate deterministic checks and focus human attention on the decisions that carry the greatest risk.
The course treats AI output as untrusted until supported by evidence. The result is a practical quality harness that helps agents correct routine mistakes before a change reaches human review.
Learning Outcomes
Upon completion of this course, participants will be able to:
- Identify characteristic failure modes in AI-generated implementations and tests
- Design verification independently from the code-generation process
- Convert requirements and specifications into risk-based test strategies
- Use mutation and property-based testing to assess the strength of generated tests
- Combine static analysis, type checking, security scanning and runtime tests
- Evaluate large agent-generated diffs efficiently and systematically
- Build automated repair loops without weakening quality gates
- Define evidence requirements for approving AI-generated changes
Course Outline
The Verification Challenge
- Why increased code volume changes the economics of review
- Plausible code, fabricated assumptions and incomplete edge cases
- Tests that mirror the implementation rather than the requirement
- Cognitive debt and declining developer understanding
- Risk-based verification for different systems and change types
- Establishing that AI output is a proposal rather than evidence
Independent Test Design
- Deriving tests from specifications and business behaviour
- Separating implementation generation from test-oracle design
- Identifying boundaries, invariants and failure conditions
- Creating negative cases and counterexamples
- Selecting unit, integration, contract and end-to-end tests
- Using domain experts to validate consequential behaviours
Test-Driven Agentic Workflows
- Writing or approving tests before implementation
- Giving agents small feedback loops and clear stopping conditions
- Preventing generated code from weakening existing tests
- Reviewing changes to fixtures, snapshots and test configuration
- Requiring reproduction tests for defect fixes
- Using test results as machine-readable evidence
Property-Based and Metamorphic Testing
- Expressing properties that should hold across many inputs
- Generating boundary and adversarial test data
- Testing transformations when exact outputs are difficult to specify
- Applying invariants to financial and data-processing software
- Diagnosing failures discovered through generated cases
- Combining property tests with example-based suites
Mutation Testing and Test Strength
- Using controlled code mutations to challenge a test suite
- Interpreting mutation scores and surviving mutants
- Detecting assertions that execute code without checking behaviour
- Prioritising mutations around risk and business logic
- Using mutation feedback to guide agent self-correction
- Managing runtime cost in continuous integration
Static and Security Verification
- Type checking, linting and semantic code analysis
- Dependency and software-supply-chain checks
- Secret detection and insecure configuration scanning
- Secure coding rules for generated database and API changes
- Policy-as-code for architecture and compliance constraints
- Treating deterministic analysis as a non-negotiable release gate
Reviewing Agent-Generated Pull Requests
- Reconstructing intent from the specification and change summary
- Reviewing architecture and behaviour before implementation detail
- Detecting scope expansion, duplication and unnecessary abstraction
- Verifying claims with commands, tests and source documentation
- Sampling strategically without creating false confidence
- Requiring smaller changes when meaningful review is not possible
Evaluation and Repair Loops
- Distinguishing software tests from agent evaluations
- Measuring task completion, regression and review rework
- Feeding test and analysis results back to an agent safely
- Limiting retry loops and detecting repeated failure patterns
- Escalating ambiguous or high-impact failures to people
- Preserving logs and evidence for later investigation
Practical Capstone
- Review a deliberately flawed agent-generated change
- Build an independent risk-based test strategy
- Add property-based or mutation testing to expose weaknesses
- Configure deterministic quality and security gates
- Use bounded agent repair to address verified failures
- Produce an evidence-based release recommendation
Intended audience
This course is designed for software developers, test and quality engineers, technical leads, application security specialists and engineering managers responsible for reviewing or releasing AI-generated changes. It is suitable for teams using coding agents in professional delivery environments.
Prerequisites
Those attending this course should meet the following:
- Experience writing and reviewing production software
- Working knowledge of unit and integration testing
- Familiarity with continuous integration and pull requests
- Basic experience with AI-assisted coding tools