A failure tour

The Northstar.ProtoTest sample suite ships four tests that fail on purpose. Each drill is paired with a test that runs the same journey and passes. The difference is one practice.

Level 0, lesson 2About 10 minutes

By the end

• Read a drill trace and name the question it failed to answer.
• Say what the paired test changed, and why that is enough.
• Record the four failures in your own run.

Before you start

• The four questions (lesson 1).
• Nothing installed. The archives are on this site; running the suite needs the .NET SDK and the repository.

The scenario

A failure you cannot reproduce is hard to trust. The sample makes the failures part of the suite. Four tests fail when the drills are enabled. Four paired tests pass.

Set ProtoTest__Sample__Drills=true, and both halves run. The drills report their failure and leave their trace; the fixes run the same journey and stay green.

Four cards, four questions

Open a card to see what the drill recorded and what the test beside it does instead. The values are from a recording of the sample with the drills enabled, so they are what a reader would find, not an illustration.

What a failure looks like

Four questions a failing integration test usually asks. Open one to compare the drill that fails with the test that holds, as their traces recorded them.

Each card is a real recorded run. The drill failed on purpose, and the paired test runs the same journey the right way. Each card links its own archive.

The four pairs match the four questions from lesson 1. Time moves the clock instead of waiting. State creates the data it reads. Environment takes the address from the composition. Visibility asserts the body the application sent.

Record the failures in your own run

The drills run only when the suite is asked for them. In an ordinary run their bodies skip themselves, so the suite stays green.

$env:ProtoTest__Sample__Drills = "true"
dotnet test samples/Northstar.ProtoTest

The run now reports four failures, one per question, and writes their traces. The warning journey beside them passes with a warning, so its trace records a partial outcome and a finding. Each card above links its own archive from docs/static/lessons/, written by the same generator that writes the archive your run produces. Download one and drop it on the viewer to walk it yourself.

One of the fixes, line by line

The time drill waits a real second. Its fix moves the test clock instead and then pays the invoice. This is the body of TheTestClockClosesTheDueWindow.

FailureDrills.cs5 notes

1var invoice = await Proto.Context.Data().IssueInvoiceAsync();1
2
3Proto.Context.Clock.Advance(TimeSpan.FromDays(8));2
4using var organization = await Proto.Context.Rest().GetAsync("/api/v1/organization");3
5organization
6.Should.HaveHttpStatus(HttpStatusCode.OK)
7.Should.MatchShape(new { status = SubscriptionStatuses.PastDue });4
8
9using var paid = await Proto.Context.Rest()
10.Body(new PayInvoiceRequest(PaymentMethods.Visa))
11.PostAsync("/api/v1/invoices/{invoiceId}/pay", new { invoiceId = invoice.Id });5
12paid
13.Should.HaveHttpStatus(HttpStatusCode.OK)
14.Should.MatchShape(new { status = InvoiceStatuses.Paid });

1. 1
   Build and provision the invoice

   Data() builds the request, the provisioner creates the invoice, and teardown removes it.

2. 2
   Move the test clock

   Advance records a clock.advance event, and the in-process application reads the same clock.

3. 3
   Call through the composed client

   Rest() takes the address from the run, so the same test works in-process or against a published environment.

4. 4
   Assert the property the behavior depends on

   MatchShape reports the JSON path and both values when it fails.

5. 5
   Pay, then check the result

   The second call reuses the same client, the same context and the same trace.

From samples/Northstar.ProtoTest/FailureDrills.cs. The drill next to it waits on real time and the due window never closes.

Every fix in the tour reads like this one: the journey stays, and one answer changes.

Checkpoint

The environment drill ran for about two seconds and its trace holds no HTTP request. What does the trace tell you?

Verify

Download l0-environment-drill.prototrace, open it in the viewer, and compare its execution layer with the paired fix, l0-environment-fix.prototrace.

What you learned

• Each drill fails because one of the four questions has no answer.
• The paired fix changes one habit: the clock, the data, the address or the assertion.
• The trace is the evidence; the drill and the fix both leave one.

Keep exploring