Use engineering requirements as the test oracle and make each test support a specific, reviewable claim.

What you will learn

  • Define expected behavior independently of the generated implementation.
  • Cover nominal values, exact boundaries, values just outside the limit, invalid upstream status, and non-finite inputs.
  • Use Red-Green-Refactor: fail for the intended requirement gap, make the smallest correction, then rerun regression checks.
  • Apply the same reasoning in unit tests, host or target harnesses, RTL testbenches, assertions, or an existing UVM environment.

Practice focus

Add the requirement that a non-finite measurement cannot be evaluated against a numerical acceptance limit. Establish the expected INVALID result, verify the intended failure, and check the minimal correction without changing existing thresholds.

Engineering deliverable

A requirement-to-test mapping, predefined expected results, and recorded targeted and regression evidence.

Lesson transcript

Scene 01

A test suite can be completely green and still miss an engineering requirement. The risk is especially high when the same AI generates both the implementation and its tests. In this unit, expected behavior comes from the approved requirement first. GitHub Copilot can help generate cases and code, but it does not define the engineering oracle.

Scene 02

Start by making the requirement observable. Our new training requirement says that a non-finite measurement cannot be evaluated against the numerical limit. For a not-a-number input, the expected status is INVALID and there is no numeric output. Define that expectation before looking at how the current implementation happens to behave.

Scene 03

Build a small test matrix that reflects the failure modes that matter: nominal behavior, exact boundaries, values just outside the limit, invalid upstream status, and the new non-finite case. Ask Copilot for candidate cases, but require every expected result to trace back to an approved rule. More generated tests are not automatically better coverage.

Scene 04

In the Red phase, encode the requirement before changing production code. Add the non-finite case and run it against the baseline. The important check is not simply that the test fails. It must fail for the intended requirement gap. A syntax error, fixture problem, or unrelated exception is not useful Red evidence. If the failure reason is wrong, stop and investigate.

Scene 05

In the Green phase, make the smallest change that satisfies the new requirement. Reject non-finite values before numerical classification and preserve the existing limit behavior. Do not redesign the reporting interface or change unrelated thresholds. The objective is not the most elegant rewrite; it is the smallest reviewable implementation that satisfies the approved behavior.

Scene 06

After the minimal change, run the new case and then the existing regression set. Where approved tool access exists, Copilot may execute the specified commands; otherwise, run them yourself. Record the revision, environment, commands, failures, and results. A targeted test checks the new requirement. Regression tests check whether the change disturbed previously accepted behavior. Neither result should be invented or inferred.

Scene 07

The same structure transfers across engineering domains. In embedded software, the harness may be a host or target test. In RTL, it may be a testbench, assertion, or UVM environment. The implementation technology changes, but the logic does not: derive stimulus and expected behavior from the requirement, then define an observable check. The engineer still owns the oracle.

Scene 08

Before accepting the change, ask what requirement each important test actually proves and where its expected result came from. Confirm that the new case failed for the intended reason before implementation, then review the regression evidence. Passing tests support specific engineering claims; they do not prove that every relevant condition or the complete hardware system has been validated.