The Political Technology Awards

This heuristic inherits the approach in v5: ITN/A multi-agent deliberation heuristic with 6 independent AI juries that run an ITN/A deliberation on a shortlist of 183 projects. The jury with the highest confidence score picks the project winner.

v6 inherits the full ITN/A evaluation and deliberation pipeline from v5 — the same 4-agent structure (political, relational, experimental), the same multi-argument format with a facilitator, and the same scoring tiers (deliberated: 51–90, 2+ greens: 45, 1 green: 20, none: 5).

The changes are in who runs the jury, how the shortlist is built, and how the winner is selected.

Total pipeline spend: $11.30 USD

Phase 1: Evaluation

Grok 4.1 Fast, Claude Sonnet 4, and Kimi 2.5 independently evaluate all 321 candidates across three lenses: political, relational, and experimental. Each assessment produces a bucket (green / yellow / red / grey) and a rationale per dimension.

Phase 2: Shortlist

Shortlist rule: at least 2 of the 3 models rated the project green or yellow in any dimension.

This 2-of-3 rule produced 183 candidates.

Phase 3: Six Jury Deliberations

The three original models deliberate on their own evaluation data (each model reads only its own assessments). Each jury runs the full ITN/A multi-agent deliberation — agents argue, contest, revise scores, and produce a ranked shortlist with a winner, confidence score, and case for/against.

Original juries:

• Grok 4.1 Fast
• Claude Sonnet 4
• Kimi 2.5

Mixed juries (deliberate from merged assessments):

• GPT-4o — mainstream / average voter
• DeepSeek-R1 — adversarial / contrarian
• Specialist panel — Gemini 2.5 Pro (political), Llama 3.3 70B (relational), Mistral Large (experimental)

Phase 4: Promotion

The six jury verdicts are compared by confidence score. The highest-confidence jury is promoted and selects the final award winner.

v5 showed us that a single deliberating agent via Grok could produce a well-reasoned winner (one that made sense in an obvious political-tech-award sort of way), but left an open question:

Is the result robust to model choice?
Do different AI “worldviews” produce the same answer?

v6 tests this theory.

Three original juries read their own evaluation data. The three mixed juries read merged assessments and deliberately introduce different perspectives: GPT-4o as a mainstream / institutional voice, DeepSeek-R1 as a contrarian with different political training data, and a specialist panel routing each evaluation lens (political, relational, experimental) to a purpose-fit model.

Confidence scoring

Winner selection is based on the confidence score — the jury that is most certain of its verdict wins.

Grok 4.1 Fast again had the highest confidence (as in v5). It was also the cheapest and fastest model while still producing interesting analysis.

Human-readable Markdown summaries of all six jury deliberations: jury README.

All six juries disagreed on their winners:

Jury	Winner	Confidence
Grok	AlgorithmWatch	95
Specialist Mixed Jury	Alaveteli	90
Adversarial	SlopStop	85
Kimi	Worker Info Exchange	82
Mixed Jury	Bellingcat Toolkit	75
Claude	Awesome Gov Datasets	42

Different AI training backgrounds appear to encode different political values:

• Grok — systemic advocacy, evidence-to-policy pipelines
• DeepSeek — decentralised infrastructure
• Specialist panel — civic access tools with proven relational networks

AlgorithmWatch as the promoted winner (score 97, confidence 95) is defensible: it scored near-uniformly high across all three lenses (political 98, relational 97, experimental 97) and was contested, meaning it survived challenge.

Findings & reflections

Why is it always Grok?

Grok performed well on v5; testing against other model juries, it again came out on top by confidence. Grok may be trained to be opinionated (e.g. if its corpus includes Twitter/X debates). This task structure — high velocity, high conviction, argument format — may suit Grok’s design. The confidence score Grok reports (95/100) might reflect a greater willingness to say it’s confident (analogous to Claude being less willing to assign green). The ITN/A deliberation format (make a case, contest, revise) may be one Grok is natively good at.

Claude (claude-sonnet-4-6) is structurally conservative as an evaluator

Model	Greens	Green+Yellow
Grok	120	191
Kimi	33	230
Claude	3	92

Claude assigned green to only 3 / 321 projects, compared with Grok’s 120.

The shortlist rule was originally a union of green projects but had to be redesigned after evaluation. Research into Claude Sonnet 4 suggests this behaviour is related to RLHF calibration around political statements and alignment. The shortlist rule was changed to the 2-of-3 model rule, widening the pool to 183 projects.

Shortlist rule design

Goal: roughly 100 projects (~⅓ of the dataset).

Strategy	Count	Verdict
Union — any model green or yellow	242	Too loose; single-model noise
2-of-3 — at least 2 models green or yellow	183	Chosen
Intersection — all 3 models green or yellow	88	Too tight; Claude’s calibration removes too many

Rule chosen: A project enters the shortlist if at least 2 of the 3 models rated it green or yellow in any dimension.

Jury structure

Original juries (each reads its own evaluation data):

Jury	Model	Assessments
grok	x-ai/grok-4.1-fast	assessments-grok.json
claude	anthropic/claude-sonnet-4-6	assessments-all-claude.json
kimi	moonshotai/kimi-k2	assessments-all-kimi.json

Mixed juries (read merged assessments):

Jury	Model(s)	Rationale
mixed	openai/gpt-4o	mainstream / institutional perspective
adversarial	deepseek/deepseek-r1	contrarian political training data
specialist	gemini-2.5-pro + llama-3.3-70b + mistral-large	per-lens specialists

Winner selection compares winner.confidence across all six deliberation outputs.

Open questions

• Does Claude’s calibration asymmetry reflect a different political epistemology, or just noise? The adversarial (DeepSeek) jury is an interesting contrast.
• The specialist jury (per-lens model specialisation) is experimental — does routing political/relational/experimental to different models improve deliberation quality?
• Confidence scores across juries indicate how contested the winner is. High variance = interesting; low variance = robust consensus.
• Open expansions from the v5 iteration.
• Trying a different framework could further test reflections on Grok vs Claude on deliberations and confidence.