TDD in the AI Era

Barry S. Stahl

Solution Architect & Developer

Microsoft .NET MVP

Physics & Math Nerd

@bsstahl@cognitiveinheritance.com

https://CognitiveInheritance.com

Favorite Physicists

1. Harold "Hal" Stahl
2. Carl Sagan
3. Richard Feynman
4. Marie Curie
5. Nikola Tesla
6. Albert Einstein
7. Neil Degrasse Tyson
8. Niels Bohr
9. Galileo Galilei
10. Michael Faraday

Other notables: Stephen Hawking, Edwin Hubble, Leonard Susskind, Christiaan Huygens

Favorite Mathematicians

1. Ada Lovelace
2. Alan Turing
3. Johannes Kepler
4. Rene Descartes
5. Isaac Newton
6. Emmy Noether
7. George Boole
8. Blaise Pascal
9. Johann Gauss
10. Grace Hopper

Other notables: Daphne Koller, Grady Booch, Leonardo Fibonacci, Evelyn Berezin, Benoit Mandelbrot

• See my presentation from CodeMash 2025: How the Fediverse Could Save Democracy & Why It Probably Won't

• Hear my thoughts on why a thriving Fediverse is so critical on David Giard's Technology and Friends show.

• Learn more about the Fediverse at fediverse.party.

• Get your own Mastodon account at Join Mastodon.

  • Don't stress on what server to use, just pick one and go. You can always move later.

• Follow me on Mastodon at @bsstahl@CognitiveInheritance.com.

• Machine code
  • Exact instructions the hardware executes.
    • Maximum precision
• High-level languages
  • Express logic in readable source code.
    • Explicit control flow & data structures
• Frameworks and libraries
  • Reusable abstractions and patterns.
    • Accelerate implementation and consistency.
• AI-assisted intent
  • Describe outcomes, constraints, & behavior.
    • Tools synthesize structure.

• We can delegate implementation now
  • A strong spec can be thrown over the wall and AI may produce plausible code.
• Capability is not permission
  • The decision is not "can we"; it is "should we in this context".
• Use over-the-wall only with strict boundaries
  • Disposable code or permanently black-boxed services with stable contracts.
• Otherwise keep humans in the loop
  • Shared, long-lived, or high-risk code needs human-readable intent and design.

• Typing is no longer the main bottleneck
  • We produce code faster than we can understand it.
• Understanding the problem dominates
  • This is now the constraint.
  • Spec quality and architectural choices limit progress.
  • Architect for ability to reason about the system.
• Decomposition still matters
  • Large problems need careful slicing.
• Verification is the safety valve
  • Confidence costs (time & money).

• Start with the smallest meaningful step
  • Write the minimum failing test that proves you are solving the right problem.
• Then evolve safely
  • Make it pass, then refactor after behavior is protected by tests.
• Principle, not recipe
  • This is the same mindset we apply to evolving AI-assisted workflow.

• Use the loop as the decomposition boundary
  • Each step is a bounded implementation unit.
• Red defines control
  • Failing tests represent our definition of done.
• Green uses minimal bounded implementation
  • Implement only what is needed and in-scope.
• Refactor restores clarity before expansion
  • Review, validate, and clean up before moving on.
• Result: speed without loss of accountability
  • Controlled implementation in a logical process.

• TDD breaks work into manageable chunks
• Each loop covers a single behavior
• Small steps reduce cognitive load and risk
• Focused tests clarify intent and boundaries
• Confident refactoring possible with narrow scope

• Define what the system must do
  • Any agent (human or machine) can read the test to understand intent
• Guarantee correctness
  • Visible to both machines and humans
• Make implementation safely changeable
  • Any agent (human or machine) can refactor ruthlessly
• Tests outlive implementations
  • Code is temporary; behavior is durable
  • Tests preserve the behavior

Key takeaways - If the tests are right, the code can be anything - If the tests are wrong or missing, the code can't be trusted

• Coupling: dependency load between units
  • Tighter coupling means 1 change forces many edits
  • Increases complexity and raises risk
• Cohesion: focus within a unit
  • High cohesion means modules have 1 reason to change
  • Changes are isolated to a single module
• The TDD loop exposes design friction
  • Red reveals setup and dependency pain
  • Green shows overreach pressure
  • Refactor makes unclear responsibilities obvious

• Facade creates a stable test seam
  • Callers get a small, stable surface area
• Repository is facade for data access
  • Tests can isolate persistence with fakes/mocks
• Strategy is facade for business logic
  • Behavior can be tested independent of algorithm

• Do we need 100% test coverage?
• What is "too trivial" to test?
• Should we test infrastructure code like logging?

• London School — Interaction-Driven

  • Steve Freeman & Nat Pryce
    • Growing Object-Oriented Software, Guided by Tests
  • Unit = a single class (collaborators mocked)
  • Tests assert interactions
  • Pros: strict boundaries, explicit roles
  • Cons: brittle tests tied to internals

• Detroit School — Behavior-Driven

  • Kent Beck & Martin Fowler
    • Test-Driven Development: By Example
  • Unit = a behavior across objects (few mocks)
  • Tests assert outcomes
  • Pros: resilient tests, refactoring freedom
  • Cons: harder to isolate business rules

• Detroit Philosophy at the Core

  • Test behavior, not interactions
  • Use real collaborators within a layer
  • Keep tests stable under refactoring

• Guiding Principle

  • Define layer boundaries using SRP
  • Mock/Fake boundaries, not objects
  • Test behaviors, not collaborations
  • Examples: data stores, external services, etc.

• Why This Works

  • Invariants tested in full isolation
  • Internal object graph can evolve freely
  • Clear seams for integration, observability, and resilience

• Red: write one failing test.
  • Describe intent and any boundaries.
  • Tools might help stub the tests.
• Green: Write the minimum code for tests to pass.
  • Most manual typing happened here.
  • Don't create structure, just get to green.
• Refactor.
  • Clean up names and logic.
  • Improve structure and design.
  • Run full test suite to prevent regressions.

Change Password service

• Invariants
  • newPassword must meet criteria
  • oldPassword must match current password
• If invariants are satisfied, password is updated
  • Change is successful → return true
  • Return false otherwise
• Start with Test 1: minimum length check
  • Stub the function
  • Create the test
  • See the test fail appropriately
• Questions:
  • Method signature (input/output)
  • API Surface location
  • Criteria for password strength

Test Cases:

[*] <n chars → false

[ ] old <> current → false

[ ] All rules satisfied → true

• Duplication is growing
  • Similar logic appears in multiple places.
• Responsibilities are blurred
  • Single Responsibility Principle (SRP) is violated.
• Naming no longer matches intent
  • Tests pass, but names and structure obscure the behavior.
• Change impact is widening
  • Behavior updates require touching many files or call sites.
• Tests are brittle or hard to write
  • Setup is heavy, seams are unclear, or assertions over-specify implementation details.
• Invariants are scattered
  • Critical rules are enforced in several locations instead of one clear boundary.

• Next behavior to add
  • Reject the password change when the supplied current password does not match the stored password.
• New pressure introduced
  • The test needs current-credential context that likely comes from storage.
• Constraint to preserve
  • Keep scope small and feedback fast while protecting design boundaries.

• Option 1: Compare in method
  • Repository returns credential data; matching rule stays local.
• Option 2: Compare via in-process service
  • A domain service inside this app owns behavior.
• Option 3: Delegate to external auth
  • Use an external authority and consume the result.

• What stays the same

  • Red-Green-Refactor remains the control loop.
  • Tests still define intent and verify correctness.
  • Human judgment still owns architecture, boundaries, and acceptance criteria.

• What changes with AI pairing

  • Idea/prototype throughput increases.
  • Prompting and decomposition quality become bottlenecks.
  • Verification rigor must increase because plausible output is cheap.

• Mutual learning relationship
  • You learn from their approaches and methods.
  • They learn and remember your patterns & preferences.
• Dialogue and refinement
  • Have conversations with AIs about approaches.
  • You guide direction; they generate options.
• Key difference from junior developers
  • No ego or defensiveness; they adapt to your feedback.
  • Consistent availability and tireless iteration.
  • Stop anytime and pick up where you left off.

• Red: compact context and discuss scope with AI.
  • Architecture constraints, done criteria, and test strategy.
  • Externalize assumptions before the first failing test.
• Green: generate code and supporting docs.
  • AI proposes implementation options.
  • Human forces limitation to "Simplest thing...".
  • Keep documentation aligned with the implementation.
• Refactor: clean up structure and improve.
  • Better naming, shape, and reuse.
  • Push toward a DSL whenever reasonable.
  • Use mutation testing to validate the test suite.

• Setup
  • Compact context if needed.
  • Discuss scope and strategy.
• Define inputs and outputs.
  • Data shape, required/optional fields.
  • Validations and state changes.
  • Emitted events and visible behavior.
• Define constraints and done criteria.
  • Feature Invariants.
  • Allowed external calls and dependencies.
  • Idempotency, and ordering.
  • Errors and retries that are in-scope.
  • What is explicitly out-of-scope.
• Add failing test(s) that enforces scope.
  • More than 1-2 tests to start is a scope smell.
  • Prompt pattern: "Strict TDD mode; no behavior yet."

• Implement the minimum behavior to make tests pass.
  • Lock scope to Red-step constraints.
• Propose implementation options.
  • Only what is required to make the tests pass.
• Determine tools/frameworks only when required.
  • Refactor toward preferred patterns later.
  • Do not build object graphs yet.
• Keep documentation aligned with implementation.
  • Update or create docs to reflect behavior and constraints.
• Run tests and confirm the suite is green.
  • Prompt pattern: "Strict TDD mode; satisfy tests only."

• Refactor for clarity without changing behavior.
  • Improve names, decomposition, and reuse.
  • Create object graphs and add frameworks as needed.
  • Refactor both tests and production code if appropriate.
    • Be sure tests are stable before touching production code.
• Strengthen confidence in the suite.
  • Run full tests after each meaningful refactor batch.
  • Run coverage and inspect uncovered paths.
  • Use mutation testing to expose weak assertions.
• Squash here if appropriate to keep history clean.
• Prompt pattern: "Strict TDD mode; refactor only; preserve behavior."

• Use builders setup data more maintainably
  • For both tests and production code
• Extract methods to improve readability
  • Make liberal use of Extension methods or equivalents
• Create tests that are as declarative as possible
  • The test's intent should be directly readable in the test
  • Readable tests matter more than clean prod code
  • Clean Code -- But for Tests!

• Did we test what we built?
  • Verify expected and edge behaviors have explicit assertions.
• Missing coverage is a critical signal
  • Uncovered paths may indicate code is not acting as expected.

• Did we build what we tested?
  • Confirm implementation still matches test intent after refactor.
• Missing coverage is a critical signal
  • Uncovered paths may mean we coded ahead of the test.

• Question every gap
  • Missing test?
  • Poorly behaving code?
  • intentional scope boundary?
• Decision before proceeding
  • If alignment is unclear, pause and strengthen tests before moving on.
• Proceed rule
  • Move forward when confidence is behavior-based, not percentage-based.

• Mutate code intentionally
  • Mimic realistic defects
• Run existing tests against each mutant
• Killed mutants - Tests fail as expected
  • Validates tests detect incorrect behavior
• Surviving mutants - Tests still pass
  • Expose weak assertions or missing cases
• AI agents automate the loop
  • Mutations can be run early and often
  • Used to be too dificult for regular use

• Run mutation tests early and often
  • Validate that tests catch real defects, not just happy paths.
• Treat surviving mutants as feedback
  • Strengthen assertions, add missing cases, or tighten boundaries.
• AI changes the economics
  • What used to be costly and slow is now practical inside normal loops.

• Boundary conditions

  • Original: if (count < max)
  • Mutated: if (count <= max)

• Arithmetic operators

  • Original: total + fee
  • Mutated: total - fee

• Null / optional paths

  • Original: user?.email handling
  • Mutated: force null branch behavior

• Return / exception behavior

  • Original: return Success
  • Mutated: throw ValidationException

• Select candidates + run tests

  • "Knowing this feature and tests, create code mutations likely to expose gaps in the tests, and run the tests with them in place."

• Improve tests

  • "For surviving mutants, implement stronger tests and assertions. Do not modify production code."

• Iterate

  • Repeat until survivors are equivalent or out of scope. When complete, remove the mutations.

• Refactoring prompts and mutation testing
  • Increase confidence, speed, and signal quality.
• Capability amplifiers
  • These are optional power-ups, not entry requirements.
• Use when risk is high
  • Apply them when complexity or cost of change increases.
• Skip when loop is already crisp
  • If feedback is clear and fast, keep the cycle lean.
• Baseline stays constant
  • Red-Green-Refactor remains the control system.
• Next step
  • Define the default workflow teams can run consistently first.

• Start narrow, then iterate
  • Strong adopters begin with constrained use and expand as confidence grows.
• Speed gains do not remove human responsibility
  • AI helps with implementation throughput.
  • Humans still own decomposition, design judgment, and verification.
• Calibration beats novelty
  • Teams improve by iterating on outputs.
• TDD-aligned takeaway
  • Begin with the simplest safe loop, then refactor

Research: Barke et al. (2023); Nair et al. (2023); Chen et al. (2021).

• By host
  • In the browser
  • In the terminal
  • In the IDE
• By purpose
  • Code generation
  • Specification generators
  • Workflow orchestrators

• What they do
  • Generate code, tests, and refactor suggestions.
• How they fit TDD
  • Used inside each red-green-refactor step.
  • Verify with test suite before accepting output.
• Why they help
  • Reduce time spent on syntax and boilerplate.
  • Humans focus on correctness and risk decisions.
• Practical guardrails
  • Small prompts, short diffs, and frequent test runs
  • Review generated code as if from a new teammate.

• What they Produce
  • Structured requirements
  • Constraints
  • Acceptance criteria
• How they fit TDD
  • First, create a specification
  • Then derive tests from it
  • Finally, implement only what is needed.
• Why they help
  • Reduces churn
  • Catches requirement gaps
  • Keeps test intent aligned with behavior

• What they do
  • Coordinate multi-step tasks.
  • Bridges tools, repositories, and prompts.
  • Includes state, retries, and handoffs.
• How they fit TDD
  • Automate repetitive workflow segments.
  • Preserve human checkpoints.
• Where to add guardrails
  • Require plan approval
  • Enforce test pass gates
  • Cap diff size
  • Require explicit review before merge

• What AI Engineer Coach is
  • VS Code extension to analyze coding session logs.
  • Turns usage details into coaching insights.
  • Helps improve how your team uses those tools.
• What it provides
  • Anti-pattern detection
  • Skill discovery
  • Lots more now and in the future
• Reference
  • AI Engineer Coach

• Define the problem and constraints clearly
  • Give the model bounded, testable targets.
• Recognize good architecture and tool choices
  • Match patterns and tools to the scenario.
• Verify output with tests and quality gates
  • Treat model output as a draft, not a guarantee.
• Evolve workflows and tools as confidence changes
  • Start simple, then increase capability deliberately.
• Delegate effectively to people and agents
  • Orchestrate grunt-work.
  • Keep accountability human.
• In short: think like a solution architect
  • Own outcomes, tradeoffs, and verification.

• If you already have Copilot in VS Code
  • Start there and keep your loop tight.
• If you already have Claude Code or OpenCode
  • Start there and apply the same TDD gates.
• If you are new to these tools
  • Start with Copilot or Gemini in the browser.
  • Learn the prompts and review habits before adding complexity.

• Pick a project you already understand
  • Real constraints make the exercise more valuable.
  • Familiar domains keeps focus on workflow.
• Greenfield or Brownfield
  • Create that new project you've been thinking about.
  • Or add features to an existing codebase.
• Start with a small, testable slice
  • Keep scope tight so loop stays observable.
  • Work in a new branch for safety.
• If you prefer a safer sandbox
  • Use one of the packaged options on the next slide.

• Greenfield: Scheduled post publisher
  • Build a new, small system end-to-end with tests from the start.
  • Define the domain and constraints yourself.
• Brownfield: Meeting scheduling system
  • Refactor a tightly-coupled file-based system.
  • Leverage the architecture diagram and starter code.
• Choose by comfort level
  • Favor the option where you can focus on workflow discipline, not domain discovery.

• This Slide Deck
• Design Patterns for Loose Coupling
• TDD FAQ on CognitiveInheritance
• My TDD Articles
• Repository Pattern on Microsoft Architecture guidance
• Tools for AI Assisted Development