Golden harness

A nightly check that an agent connected to the workspace answers from it, graded deterministically.

You get a set of fixed questions asked through the connector every night. Each is graded on which tools the agent called and what its answer contained, and a pull request reports a pass or fail for every question.

The prompt

Read the skill at https://github.com/fred-abrahamson/midland-examples/blob/main/golden-harness/SKILL.md and follow it. Create a Claude Code routine on this repository that runs the goldens in goldens/*.json against our Foundation workspace nightly, with only the Foundation connector, and opens a pull request titled "Golden harness <date>: N passed, M failed". Start me with three goldens: one term, one metric, one skill.

The skill

golden-harness/SKILL.md covers the golden file shape (question, expected tool calls with required inputs, required and forbidden substrings), the runner and the grader. It also shows how to read a failure.

Check

The first run’s pull request shows three lines. Change one entity’s description and run again; the golden that depends on it tells you whether the description still steers the agent.