promptdojo_

GitHub Actions for model/content checks

When you push a change to an AI-content project (like this one), a CI workflow runs a set of checks before the change can merge. These aren't generic "does it compile" tests — they're the specific gates that catch the ways AI-written content goes wrong. PromptDojo's own pipeline runs exactly these:

  • validate_solutions — every exercise's solution actually satisfies its grader (no broken graders).
  • contracts — the public counts, copy, and schemas stay in sync with the generated content (no stale "48 lessons" when there are 50).
  • scaffold_detector — no template/filler lesson is live (the check born from the fake-content incident).
  • smoke — the key routes still return real content.

In prose this is a GitHub Actions workflow (on: pull_request, a job per check). Run the editor: a bad change trips scaffold_detector, and the gate reports exactly which check blocked it.

Each check catches a different bug

The skill is mapping a bad change to the check that stops it: a fake lesson → scaffold_detector; a broken answer key → validate_solutions; a count that drifted out of sync → contracts; a dead route → smoke. The gate merges only when all pass.

Why a builder cares

This is the machinery that kept PromptDojo honest after the fake-content incident: the scaffold detector and contracts now run on every PR, so the same failure can't merge silently again. Knowing which check guards which risk is how you read a red CI run and fix the right thing. (This is a browser-safe simulation — no real Actions runner needed.)