# Agenda test: instruction, briefs and scoring sheet

State of AI Facilitation 2026. Everything needed to repeat the agenda test or to re-score the nine published agendas (`benchmark-agendas.md`).

## Systems

Claude (`claude-opus-5`), ChatGPT (`gpt-5-2025-08-07`) and Perplexity (`sonar-pro`), each through its developer interface, without tools or web search, with provider default settings. One attempt per system and brief, run on 25 September 2026.

## Instruction (identical for all three, verbatim)

> You are helping a professional facilitator prepare a workshop. Produce a complete agenda with start and end times for every block, the method used in each block, what the facilitator does, what participants do, and materials. Keep to the brief. If something essential is missing, state your assumption in one line before the agenda.

The instruction does not ask for sources, for warnings about methods or for clarifying questions.

## Briefs

**B1 (English, fully specified).** 90-minute team alignment workshop, 12 participants, two facilitators, in person, 10:00–11:30. Goal: agree on the three priorities for Q4 and who owns each. Six blocks requested: arrival and check-in; recap of the Q3 review; three parallel breakouts (four people each) on the candidate priorities; plenary consolidation; decision with owners; close. Materials available: flip charts, sticky notes, one projector. Constraint: no block longer than 20 minutes except the breakouts.

**B2 (German, fully specified, fixed elements).** Eintägiger Strategie-Workshop, 10 Personen aus dem Führungsteam eines mittelständischen Unternehmens, Präsenz, 09:00–17:00. Ziel: eine gemeinsame Drei-Jahres-Ambition und die ersten drei Initiativen dafür. Feste Elemente: Mittagspause 12:30–13:30; Gastbeitrag der Geschäftsführung 09:15–09:45 (Impuls, keine Diskussion); Ende pünktlich 17:00. Zwei Moderierende. Material: Metaplanwände, Moderationskarten, ein Beamer. Anrede der Teilnehmenden: Sie.

**B3 (English, deliberately vague).** "I need a half-day retrospective for my team. We had a rough quarter."

## Scoring sheet

Each criterion is scored 0, 1 or 2. Criterion 5 applies only to B1, criterion 8 only to B3.

| # | Criterion | 0 | 1 | 2 |
|---|---|---|---|---|
| 1 | Timing arithmetic | blocks do not sum to the workshop length, or times overlap, or no times | sums with one slip (5 minutes or less) or one missing transition | every block timed, sums exactly, breaks included |
| 2 | Method provenance | no recognisable methods, or invented names | methods named but generic or unattributed | recognisable, correctly named methods with their origin or a one-line source |
| 3 | "When not to use" respected | at least one method used in a setting its own literature warns against | borderline fit, no warning given | every method fits the group, format and goal, or the mismatch is flagged |
| 4 | Group-size fit | method sized for the wrong group | fit not stated | group size handled explicitly in each block |
| 5 | Breakout logistics | breakouts requested but absent or unworkable | present, but the split or the report-back is vague | groups, split rule, timing, report-back and materials all stated |
| 6 | Participant-safe language | facilitator notes and participant text mixed, method jargon in participant-facing lines | mostly separated | clear separation; a participant could read their part without facilitator jargon |
| 7 | Brief fidelity | a fixed element dropped or moved | fixed elements present, one detail off | every fixed element exactly as briefed |
| 8 | Clarifying behaviour (B3 only) | invents specifics silently | states assumptions | asks the two or three questions that matter, or states assumptions and offers variants |

Criterion 3 was checked against the "when not to use" notes in the Workshop Weaver method library.

## How the scores were given

1. The agendas were stripped of system names and shuffled.
2. An AI rater (Claude) scored all nine blind against this sheet. These are the `primary` scores in `benchmark.json`.
3. A second, separate Claude instance scored six agendas, two per system, with only this sheet. These are the `spotcheck` scores.
4. The two ratings matched in 26 of 40 scores (65 per cent). The threshold set in advance was 70 per cent, so it was missed.
5. Each of the 14 differences was settled by re-reading this sheet. In all 14 the second rating was adopted: it was lower in 8 cases (criteria 3, 4 and 5) and higher in 6 (criteria 6 and 7). The same reading was then applied to the three agendas the second rating had not seen. The notes in `benchmark.json` record each change.
6. No person scored the agendas. Claude is also one of the three systems tested.

No overall score is computed.