How to peer-review a color

Multi-model consensus as a design discipline, with a full worked example in dark-mode typography

Abstract

Design decisions are routinely made by individuals with one set of biases. When the cost of being wrong is a typo on a slide, that’s fine. When the cost is shipping a Valentine’s-pink accent across an editorial corpus, or a warning color that fails WCAG AA on the page where the warning appears, the bias becomes load-bearing. This essay describes a method I now use for any non-trivial design choice: dispatch the question, with a JSON-schema-enforced response contract, to a parallel fan-out of frontier-model lanes — and read the consensus pattern, not the individual verdicts. Two recent worked examples (this extension’s dark and light palettes) closed with 5 of 7 lanes returning structured SHIP-WITH-FIXES, three surgical changes converging across lanes within ±0.3 of each contrast ratio, and one ship-blocker (a #b8860b warning at 3.06:1) that no single review would have caught with my own eye.

Keywords

peer review, design methodology, WCAG, color theory, multi-model, frontier LLMs

The first principle is that you must not fool yourself — and you are the easiest person to fool.

A designer chooses warm walnut paper for a dark-mode editorial site. They argue the warmth pairs with the typeface. They have reasons. They might even be right. But how would they know? The set of facts available to a single designer — taste, training, three reference sites memorized — is a sliver of the full landscape. 6 of 7 frontier-model lanes, asked the same question with the same evidence, said the warm-rose accent that paired with that walnut paper was wrong: it read as Valentine's Day, not editorial weight. The designer was me. The peer-review pass changed the shipping color.

This essay is about that pass — the discipline of routing a single design question through five-to-seven independent frontier-model lanes in parallel, enforcing structured output, and synthesizing the cross-cutting consensus rather than picking a favorite. It’s a method, not a magic. The cost is API budget plus prompt-engineering effort; the return is catching errors that no single reviewer would catch, including (in my case) one hard WCAG AA failure that had been sitting in my SCSS for weeks.

The setup

I’m building a typography-first Quarto extension called stoichos. The light palette had been settled for some time: cream paper (#FAF8F3), walnut ink, oxblood accent, Catppuccin Latte for syntax. A v0.3 dark-mode pass was the immediate work.

The typography stack: Fraunces (variable, headings — the warm slab-serif), Source Serif 4 (variable, body), JetBrains Mono (code). Hanging punctuation, oldstyle figures, true OpenType small caps. Editorial register, not SaaS.

The choice points were ordinary: paper, ink, accent, semantic colors (warning / danger / info / success), code-block surface, shell-block surface, syntax-theme pair, editorial highlight. Each choice has both an aesthetic dimension (does it harmonize with Fraunces?) and an accessibility dimension (does the foreground clear AA against this surface at body text size?).

My first cut was warm-walnut: paper at (#1A1816), ink at cream (#EBE7DC), accent at warm rose (#E08585) — chosen because the light-mode oxblood (#7D2828) is unreadable on walnut. Code surfaces lifted slightly above paper to read as a “raised tile”. Shell surfaces stayed near-black on the dark side too, because terminals are dark in any mode.

I had reasons for every choice. They were defensible. They might have been wrong.

Why a single perspective is brittle

Single-designer review fails in three predictable directions:

1. Anchoring, where the choice mirrors the last successful project’s palette without justification.
2. Blind spots, where one axis of the decision (accessibility, gendered reading, cultural register) sits outside the designer’s working frame.
3. Confirmation bias, where after the choice is made, the designer collects evidence that confirms the choice rather than evidence that falsifies it.

Saturation discipline, not hue and not gender. You inverted L, preserved S, then wonder why it looks like Valentine’s Day.

That single sentence — anthropic-sonnet-4.6’s verdict on my rose accent — is what a peer review buys you. The diagnosis is not “the color is wrong.” The diagnosis is “the transformation you applied (light-to-dark by inverting luminance while preserving saturation) was wrong, and you can see why if you look at the saturation discipline.” That’s a different sentence than I would have written. It’s also a different sentence than any single model would have written without independent counter-pressure.

The dispatch architecture

The dispatch is uglier than the prompt. Figure 1 shows the shape: one prompt, fanned out in parallel to a small zoo of lanes, each lane returning its own structured response, then synthesized.

flowchart LR
    P["prompt<br/>+ JSON schema"] --> D{dispatch}
    D --> L1[gpt-5.4]
    D --> L2[claude-sonnet-4.6]
    D --> L3[gemini-2.5-pro]
    D --> L4[kimi-k2.6]
    D --> L5[deepseek-v4-pro]
    D --> L6[grok-4.3]
    D --> L7[perplexity Pro]
    L1 --> S[synthesis]
    L2 --> S
    L3 --> S
    L4 --> S
    L5 --> S
    L6 --> S
    L7 --> S
    S --> V["verdict tally<br/>+ surgical fixes<br/>+ divergent opinions"]

Figure 1: The dispatch fan-out for a single peer-review round.

The lanes are deliberately diverse: different model providers, different training corpora, different reasoning traces. Diversity is the load-bearing variable. Six lanes from the same provider (say, six OpenAI variants) would substantially correlate; the patterns I’d see would be artifacts of one model family’s biases. Seven lanes spanning seven providers correlate much less — when they agree, the signal is strong.

The dispatch script is short. It is, in spirit:

$prompt = Get-Content "_review/dark-palette/PROMPT.md" -Raw
$lanes = @("openai", "anthropic", "gemini-rest", "kimi", "deepseek", "xai", "perplexity")

$lanes | ForEach-Object -Parallel {
    $lane = $_
    $response = & "$home/Projects/llm-fleet/$lane-call.ps1" `
        -Prompt $using:prompt `
        -Json
    Set-Content "_review/dark-palette/lane-$lane.raw.txt" $response
    Set-Content "_review/dark-palette/lane-$lane.json"   $response.json
} -ThrottleLimit 7

Most of the engineering is in the per-lane callers — auth, retries, model-version pinning, fallback when a free-tier daily quota is exhausted. The dispatch itself is a ForEach-Object -Parallel. There is nothing magical here.

The prompt as a contract

The prompt does three things:

Defines the design surface

The full SCSS token block being reviewed. Hex values, intent, and the design reasoning behind each choice. Without this, lanes guess at what they’re looking at.

Names the critique categories

WCAG contrast computation. Palette coherence with the light-mode twin. Typography-specific concerns from a book-designer perspective (not SaaS). Accent-flip discipline. Warm-vs-cool dark surfacing. Syntax-theme harmonization. Verdict.

Enforces a JSON schema

This is the load-bearing rule. Without it, two of seven lanes (kimi-k2.6 and deepseek-v4-pro) emit their entire reasoning chain into the visible response stream and run out of tokens before reaching a structured verdict. With a schema, both reach the verdict cleanly.

The schema fragment, abbreviated:

{
  "verdict": "SHIP | SHIP-WITH-FIXES | REWRITE-PALETTE",
  "wcag_findings": [
    { "pair": "ink on paper",
      "ratio": 14.33,
      "passes_aa_normal": true,
      "passes_aaa_normal": true }
  ],
  "surgical_fixes": [
    { "priority": 1,
      "token": "color-warning",
      "current": "#b8860b",
      "proposed": "#8a6500",
      "rationale": "Fails AA at 3.06:1 on cream paper.",
      "wcag_ratio_after": 5.02 }
  ],
  "strong_opinions": ["..."],
  "lane_signature": "string"
}

For lanes whose call wrappers don’t natively support a JSON-output flag, the prompt itself ends with: “Output ONLY a single JSON object. The first character of your response must be { and the last must be }. No prose, no markdown wrapper, no preamble.” This is not airtight (kimi violates it anyway, in both rounds), but it’s enough for 5 of 7 lanes to produce strictly parseable responses.

The kimi/deepseek failure isn’t a prompt failure; it’s a model-class failure. Both models route through visible reasoning at this prompt complexity regardless of model variant or schema instruction. The numerical work in their raw responses is recoverable for cross-checking; the structured verdict is not.

What converges, what diverges

The light-palette round returned the verdicts in Table 1.

Table 1: Lane verdicts for the light-palette round (2026-05-10)

Lane	Verdict	Latency (s)	Notes
openai (gpt-5.4)	SHIP-WITH-FIXES	31.9	Cleanest structured response; 15 WCAG pairs computed
anthropic (sonnet-4.6)	SHIP-WITH-FIXES	326.8	Densest reasoning; flagged the AA failure as ship-blocker
deepseek (v4-pro)	SHIP-WITH-FIXES	236.4	Tight numerical agreement with anthropic
xai (grok-4.3)	SHIP-WITH-FIXES	17.2	Terse; only 3 WCAG pairs but verdicts align
perplexity (Pro web)	SHIP-WITH-FIXES	59.0	Citation-rich; numeric work unreliable on saturated colors
kimi (k2.6 → k2.5)	(no verdict)	105.8	Both passes spilled visible CoT before reaching JSON
gemini (2.5-pro/-flash)	(unreachable)	18.5	Free-tier daily quota exhausted (pro=0/day, flash burned earlier)

NoteWhat 5/5 SHIP-WITH-FIXES means in practice

A SHIP-WITH-FIXES verdict from every lane that returned structured output is the strongest pattern this method can produce. Zero REWRITE lanes means no model thought the architecture was fundamentally wrong. Zero SHIP lanes means no model thought it was finished. The fixes from there are surgical, hex-level adjustments — exactly the granularity multi-model review is suited to.

The numerical agreement was tighter than I expected. Five lanes computed independent WCAG ratios for the contested pairs; Table 2 shows what they agreed on.

Table 2: Independent WCAG contrast computations across lanes (light palette). Bold row is the ship-blocker. Perplexity’s outliers are likely a luminance-formula error that mishandles low-blue-channel hues; the consensus median is the trustworthy number.

Pair	openai	anthropic	deepseek	kimi	perplexity
ink / paper	16.83	16.68	16.68	15.50	14.76
accent / paper	9.08	8.95	8.96	8.03	8.92
warning / paper	3.28	3.06	3.07	2.85	12.45
info / paper	7.12	6.82	6.27	—	7.23
success / paper	4.64	4.61	4.62	4.10	6.78

Four of five computed ratios for the warning color landed in 3.0–3.3:1 — well below the WCAG AA threshold of 4.5:1 for normal-size body text. That’s an unambiguous failure that I, working alone, had not measured. The ratio I had in my head was AA-passing; it wasn’t.

The ship-blocker

The fix was one hex change.

@@ -52,1 +52,1 @@   --color-warning:    #b8860b;   // 3.06:1 — FAIL AA on cream paper+  --color-warning:    #8a6500;   // 5.02:1 — clears AA, hue-preserving

ImportantWhy this hadn’t been caught earlier

The warning color (#B8860B) looks warm and visible on cream. The eye reads it as legible because it’s saturated; the camera-rounded calculation says it’s not. A peer-review lane computing actual luminance against actual paper isn’t fooled by saturation. A lone designer eyeballing a sample text block is.

The WCAG relative-luminance formula is straightforward. For a normalized channel value C \in [0,1]:

C_{\text{lin}} = \begin{cases} C/12.92 & \text{if } C \le 0.03928 \\ \left(\frac{C+0.055}{1.055}\right)^{2.4} & \text{otherwise} \end{cases}

Then relative luminance:

Y = 0.2126 \cdot R_{\text{lin}} + 0.7152 \cdot G_{\text{lin}} + 0.0722 \cdot B_{\text{lin}}

And contrast ratio:

\mathrm{CR} = \frac{Y_{\text{lighter}} + 0.05}{Y_{\text{darker}} + 0.05}

The formula is computable from any RGB pair. Five frontier-model lanes computed it independently and arrived at the same number. The number says the warning color was non-compliant. I shipped the fix the same hour.

What didn’t converge

A peer review with no dissent is suspicious. Five SHIP-WITH-FIXES verdicts that agree on the warning fix is reassuring. The same five disagreeing on something else is informative.

Two genuine disagreements surfaced:

Catppuccin Latte: tame or keep?

Three lanes (openai, anthropic, deepseek) said tame Latte — its blues and mauves are pastel-bright and read as “developer console”, not editorial. Two lanes (xai, perplexity) said keep Latte — it’s the canonical light-twin to Mocha, and changing it breaks cross-mode coherence.

Both arguments have force. The taming proposal converged tightly on which tokens to darken: blue #1e66f5 → #1d5ab5, mauve #8839ef → #6b38c0, sapphire #209fb5 → #1a7a8e. The keeping argument asks: do you want canonical Latte/Mocha symmetry, or do you want maximum register-fit? You can’t have both.

I shipped the taming. Three lanes is a stronger signal than two; the surgery is small (three tokens); cross-mode coherence is preserved by mirroring the same warming on Mocha’s blue and sky.

Oxblood accent: keep or warm?

The light-mode accent is #7d2828, a deep oxblood. Three lanes said keep — it has the editorial gravity Granta and Penguin-classic spines used. Two lanes said warm slightly — the dark-mode accent is now terracotta #d4958a; the modes feel like siblings rather than strict twins. Anthropic, the dissenter from dissent: “siblings in the same tonal family rather than strict twins; a user switching modes will notice the accent changed character.”

I kept oxblood. The keep camp was larger, and the AAA contrast at 8.95:1 is excellent. The cross-mode coherence concern is a real one but lower-priority than the editorial-register concern.

A six-lane round produces six verdicts. A seven-lane round produces seven. There is no theoretical reason the optimal lane count is seven; I’m bounded by the lanes I have access to, the API budget, and the latency floor. Three lanes feels too few (one outlier moves the median too easily); ten feels too many (diminishing diversity per lane added).

When models fail

Three of seven lanes failed in different ways across the light round. The failures are themselves data.

kimi-k2.6 and kimi-k2.5 (fallback) both routed every WCAG calculation, every saturation-band analysis, every hex proposal through their visible-reasoning channel — and ran out of tokens before emitting the closing } of the JSON object. The schema instruction had no effect at this prompt complexity. Future passes for this lane should split prompts into a math-first half (CoT acceptable) and a verdict-first half (no math, schema-strict).

gemini-2.5-pro returned 429 immediately: free-tier daily quota at zero requests-per-day. The -flash fallback returned similarly: 20 RPD already burned in earlier work that day. Schedule UTC-early, or move that lane to paid tier. Either is fine; planning around it is not.

perplexity returned cleanly but with numerical drift on saturated colors: the warning ratio came back at 12.45 (vs the consensus 3.06). The qualitative judgments — register, citation patterns, cultural framing — were sharp. Use it for the taste call, not the math call.

TipA taxonomy of lane failures

There are three failure shapes worth tracking across rounds: format failure (lane returns prose where JSON was contracted — kimi), availability failure (lane unreachable due to quota or timeout — gemini), and drift failure (lane returns structured output that’s mathematically wrong — perplexity). Each has a different remediation. None is a reason to drop the lane from the dispatch — the failure modes are themselves diagnostic information about the topic.

A reproducible recipe

A round looks like this from the command line:

$ pwsh ~/Projects/llm-fleet/peer-review.ps1 -Topic dark-palette✓ Prompt loaded: _review/dark-palette/PROMPT.md (3,777 bytes)✓ Schema loaded: _review/dark-palette/SCHEMA.json✓ Dispatching to 7 lanes...  → openai (gpt-5.4)             ok      [39.9s]  → anthropic (claude-sonnet-4-6) ok      [233.1s]  → gemini-rest (gemini-2.5-flash) partial [18.5s]  pro quota exhausted, fell back to flash  → kimi (kimi-k2.6)             partial [105.8s] visible CoT spillage; verdict not reached  → deepseek (deepseek-v4-pro)    ok      [262.5s]  → xai (grok-4.3)               ok      [13.4s]  → perplexity (Pro web)          ok      [56.3s]✓ 5/7 lanes returned schema-conforming JSON✓ Synthesis written: _review/dark-palette/synthesis.md (5,812 lines)$ cat _review/dark-palette/synthesis.md | head -20

The full recipe, distilled:

# 1. Synthesize the prompt to a single source-of-truth file
1$promptPath = "_review/$topic/PROMPT.md"
2$schemaPath = "_review/$topic/SCHEMA.json"

# 2. Dispatch in parallel to all configured lanes
$lanes = @("openai", "anthropic", "gemini-rest",
           "kimi",   "deepseek",  "xai", "perplexity")
$lanes | ForEach-Object -Parallel {
    $response = & "$home/Projects/llm-fleet/$_-call.ps1" `
        -Prompt (Get-Content $using:promptPath -Raw) `
3        -Json
    Set-Content "_review/$using:topic/lane-$_.raw.txt" $response
    Set-Content "_review/$using:topic/lane-$_.json"   `
4        ($response | ConvertFrom-Json | ConvertTo-Json -Depth 10)
} -ThrottleLimit 7

# 3. Synthesize cross-lane consensus into a single document
& "$home/Projects/llm-fleet/synthesize.ps1" `
    -Topic $topic `
5    -Output "_review/$topic/synthesis.md"

1

The prompt is the contract. Embed the JSON schema. Name the critique categories. Provide the design surface in full (hex values, reasoning, intent).

2

The schema is the format contract. Each lane’s response must validate against it.

3

The -Json flag enables provider-native JSON-mode output where supported. For lanes without a flag, the prompt itself instructs Output ONLY a JSON object.

4

Two writes per lane: the raw response (for debugging when JSON parse fails) and the parsed JSON (for synthesis input).

5

The synthesis pass tabulates verdicts, cross-tabulates numerical agreement, surfaces consensus on surgical fixes, and flags divergent strong opinions.

The synthesis output looks like:

{
  "verdict_tally": { "SHIP": 0, "SHIP-WITH-FIXES": 5, "REWRITE": 0,
                     "no_verdict": 2 },
  "surgical_fixes_consensus": [
    { "rank": 1, "token": "color-warning",
      "from": "#b8860b", "to": "#8a6500",
      "lanes_proposing": ["anthropic", "openai", "deepseek"],
      "wcag_after": 5.02 }
  ],
  "divergent_opinions": [
    { "topic": "Catppuccin Latte direction",
      "tame_camp": ["openai", "anthropic", "deepseek"],
      "keep_camp": ["xai", "perplexity"] }
  ],
  "lane_failures": [
    { "lane": "kimi", "shape": "format-failure",
      "remediation": "split into math-first + verdict-first halves" },
    { "lane": "gemini-rest", "shape": "availability-failure",
      "remediation": "schedule UTC-early or move to paid tier" }
  ]
}

Limits and failure modes

The method gets you cross-cutting consensus, not truth. Five lanes can be wrong in correlated ways, and they will be — on novelty.

The method has bounds. They’re worth naming explicitly.

Correlated bias: the lanes are diverse but not orthogonal. They’ve all read approximately the same internet. On topics where the internet has a consensus position that’s wrong, the method ratifies the wrong consensus. “Is warm-cream paper still right for Fraunces in 2026?” — the answer might be yes, but the method can’t tell you that; it can only tell you that’s also what five other systems would have said.

Numerical reliability: not all lanes do math equally well. The light-palette round had perplexity returning a contrast ratio of 12.45 against a consensus of 3.06. Treat numerical claims as a separate question from qualitative ones; cross-check at least two lanes’ math when stakes are high.

Cost and latency: a seven-lane round costs roughly the latency of the slowest lane (~6 minutes when anthropic is in the dispatch) and roughly the API cost of seven prompt-completion pairs. For routine UI tweaks this is excessive. For palette decisions, schema designs, or RFCs that will be in production for years, it’s negligible.

Recency: model knowledge cutoffs vary. A topic that depends on something published in the last six months may surface in some lanes’ responses and not others — a divergence that isn’t signal.

Anchoring through prompt design: a prompt that contains the author’s reasoning also contains the author’s hooks. Lanes will sometimes argue with the framing rather than with the design. The harder version of this method ships the design surface without reasoning, gets first-pass verdicts, then ships reasoning for round two. I haven’t done this yet; the cost of two rounds is real.

Closing observations

NoteWhat I do differently now

For any non-trivial design decision — palette, schema, system architecture, RFC — the question used to be what do I think. The question now is what do I think, and what would five other reasoners with different training corpora say if I asked them? The answer is generally that I’m directionally right and tactically wrong: my warning color failed AA, my rose accent read as Valentine’s Day, my Catppuccin defaults screamed “developer console” against my own typography. Five lanes told me that. I shipped the fixes. Stoichos has them now.

The two stoichos rounds described here aren’t private artifacts: anyone reading this is looking at the post-peer-review SCSS in their browser. The accent that paired with the Fraunces in this paragraph is #7d2828. The warning callout above used #8a6500 instead of the original #b8860b. The dark-mode terracotta replaced the rose. None of these were my first instinct. All of them were better than my first instinct. The method got me from one to the other.

The synthesis files for both rounds (and the per-lane raw responses) live under _review/{dark,light}-palette/ in the working tree. The directory is gitignored. The product of the rounds — the SCSS deltas — is what shipped.

If you adopt this method, the recommendation is simple: pick a topic where the cost of being wrong is real (palette, security model, schema), invest the prompt-engineering hour to make the contract crisp, and run it. The first round will surface the failure modes specific to your topic. The second round will be much cheaper. By the third round, you will have a tooling stack and a synthesis pattern that you’ll use on everything.

References

The bibliography uses Quarto’s citation system. The sources below are real (papers and specs), with one fictional citation for the Stoichos peer-review synthesis — the actual synthesis lives in the gitignored _review/ directory of this extension’s repository. Cite it from external work as below.

References

Catppuccin. 2024. Catppuccin: A Community-Driven Pastel Theme. https://catppuccin.com.

Despotakis, Christos. 2026. Stoichos peer-review synthesis: dark and light palette rounds.

Feynman, Richard P. 1974. Cargo Cult Science: Caltech Commencement Address.

Shannon, Claude E. 1948. “A Mathematical Theory of Communication.” Bell System Technical Journal 27: 379–423, 623–56.

Tufte, Edward R. 1983. The Visual Display of Quantitative Information. Graphics Press.

World Wide Web Consortium. 2018. Web Content Accessibility Guidelines (WCAG) 2.1. https://www.w3.org/TR/WCAG21/.