Take the evidence to CI

A red job that prints one line is not evidence. The suite already wrote the trace and the reports. The job must keep them. One step turns them into a comment the reviewer can open.

Level 4, lesson 5About 9 minutes

By the end

• Point every output at one directory and upload it as one CI artifact.
• Keep the evidence when the test step fails, not only when it passes.
• Post the digest with the action, and name the three jobs a pipeline uses.

Before you start

• The archive and the reports (lesson 3).
• A repository with CI, if you want to try the workflow. Reading it also works.

The scenario

The same suite runs in CI and on your machine. In CI nobody can open the trace from the test output folder, and the job is the last moment the files exist: once it ends, the runner is gone.

The setup below gives CI one directory to upload, keeps it on failure, and posts a digest with the archive link. The reference pages carry the working workflow; this lesson is the shape of it.

One directory for the evidence

Relative output paths resolve below the test project's build output, which forces CI to search bin/**. Give CI one absolute directory instead:

Setup.cs2 notes

1var results = Environment.GetEnvironmentVariable("PROTOTEST_RESULTS")1
2?? Path.Combine("TestResults", "ProtoTest");
3
4builder
5.ConfigureTracing(trace =>2
6trace.OutputPath = Path.Combine(results, "run.prototrace"))
7.AddSink<JsonReportSink>(sink =>
8sink.OutputPath = Path.Combine(results, "report.json"))
9.AddSink<HtmlReportSink>(sink =>
10sink.OutputPath = Path.Combine(results, "report.html"));

1. 1
   Let CI name the directory

   The environment variable is the job's artifact directory; a local run keeps the default below TestResults.

2. 2
   Trace and reports in one place

   One upload step then carries the story and the report, which is what makes the artifact link useful.

The sink registration the sample uses, with CI's artifact directory in front. Locally the fallback keeps the same files below TestResults/ProtoTest/.

Keep it when the test step fails

- name: Keep the evidence
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: prototest-results
    path: TestResults/ProtoTest
    if-no-files-found: error

The if: always() is the line that matters. Without it the upload is skipped exactly when the trace is worth reading, because a failing test fails the step that ran it.

Post the digest

The feedback action installs the CLI, uploads the trace, posts the digest and checks it against a baseline when both reports are given:

- name: Post the evidence
  if: always()
  uses: MSeys/ProtoTest/.github/actions/feedback@main
  with:
    trace: ${{ env.PROTOTEST_RESULTS }}/run.prototrace

The comment carries the failing tests, the cause and the artifact link. One check annotation lands on each failing test's source location, and a missing target skips with its reason instead of failing the job. Give the action a baseline-report and a current-report as well, and the step also fails the pull request when the run is worse than the baseline. A digest comment reads like this:

ProtoTest evidence: 1 failed, 0 passed, 0 skipped
FAILED Northstar.ProtoTest.FailureDrills.ARealWaitDoesNotCloseTheDueWindow
  $.status: expected past_due, actual active
Trace: run.prototrace (open it in the viewer)
Report: report.json, report.html

Three shapes, one suite

A pipeline around this suite usually splits into three jobs:

Job	What it runs	What the reviewer opens
Pull request	the suite in-process, on every change	the digest comment with the failing tests and the trace artifact link
Nightly	the same suite against the container topology	the same digest, with the store and the broker as real processes the run owned
Smoke (optional)	the suite pointed at a deployed environment	the same digest; capability skips drop the journeys that need the test host

These are shapes a suite of this kind fits, not a fixed pipeline. The CI page carries the same three, and its workflows are the ones to start from. The suite is the same in all three. What changes is the composition, and the composition is what decides which capabilities exist and which journeys skip.

Naming the build in the trace

A trace downloaded from CI should say which build it came from. Name the variables once and every report carries them:

builder.ConfigureTracing(trace =>
{
    trace.RunMetadataEnvironmentVariables.Add("GITHUB_RUN_ID");
    trace.RunMetadataEnvironmentVariables.Add("GITHUB_SHA");
});

Each entry appears in the report's run metadata section and on the run in the archive, so a trace three months old still names the commit it tested.

Checkpoint

The test step failed. Which steps still run, and what does the reviewer open from the comment?

Verify

Read Put every artifact in one place and The feedback action on the CI page, then the two steps in this lesson that carry if: always(). The three jobs further down are shapes a pipeline can take, not the contract.

What you learned

• One output directory and one artifact step keep the trace and the reports together.
• The artifact step runs with if: always(), because the failed run is the one worth reading.
• The action posts the digest and the artifact link; the verdict comes from comparing two reports.
• The job split is a shape a pipeline can take. The action and the artifact wiring are the contract.

Keep exploring