--- name: cross-examination description: "Cross-examine a decision with TWO different AI models - Claude and GPT-5.6 (via the Codex CLI). Each answers independently, then cross-examines the other's answer blind, then a chairman rules. The accessible 2-model version of Karpathy's LLM Council: no OpenRouter account, no extra API key, no config file. MANDATORY TRIGGERS: 'cross-examine this', 'run the cross-exam', 'חקירה נגדית', 'תחקור את זה', 'second opinion on this'. STRONG TRIGGERS (only with a real decision or tradeoff attached): 'should I X or Y', 'which option', 'is this the right move', 'check this before I ship it', 'what am I missing'. Do NOT trigger on factual lookups, casual chat, or creation tasks ('write me a tweet'). DO trigger when being wrong is expensive and the user wants a second architecture's eyes on it." --- # Cross-Examination — two models, one verdict Ask one AI a question and you get one answer. It might be excellent. You have no way to tell, because you only ever saw one perspective — and a model's blind spots are *systematic*, not random. Ask it three times in three phrasings and it misses the same thing three times. A second model trained on different data misses *different* things. That's the whole mechanism. This is Andrej Karpathy's [LLM Council](https://github.com/karpathy/llm-council) cut down to its load-bearing part: **two models, answering independently, then cross-examining each other, then synthesized.** No OpenRouter account, no second API key, no `config.py`. If you have Claude Code and the Codex CLI, you already have everything. ## Why two and not five The published evidence says the jump from one model to two is where nearly all the value is: - On a factual-reasoning task, precision went **73% (one model) → 94% (two) → 95.6% (three)**. The third model added 1.6 points. - Multi-agent research consistently finds the gain comes from **the second independent opinion plus a good synthesis** — not from models "debating" each other, which several 2025–26 papers find oversold. - The model that writes the *final* answer matters more than the number of proposers. So the chairman seat stays on the strongest model available and never gets downgraded. Five seats cost roughly 2.5× the tokens and wall-clock of two, for a fraction of the first jump. This skill takes the jump and stops. ## The two seats | Seat | Runs on | Job | |---|---|---| | **Seat A** | Claude (native, this session) | Answer the question fully and independently. | | **Seat B** | **GPT-5.6 Sol** via the Codex CLI | Same question, no sight of Seat A. Genuinely different training prior. | | **Chairman** | Claude, full reasoning | Reads both answers + both critiques, writes one verdict. | Both seats answer the *same* framed question with *no* assigned persona. That is deliberate — personas manufacture disagreement. Here the disagreement is real or it isn't, and the useful signal is where two different architectures actually diverge. ## When to run it Run it when being wrong is expensive: - "Should I price this at 2,000 or 5,000?" - "Is this architecture going to bite me in six months?" - "Here's my landing page copy — what's weak?" - "I'm about to send this to a client. What am I missing?" Don't run it on: factual lookups, creation tasks, or anything where you already know the answer and want applause. It costs two model calls and about a minute; a quick question doesn't earn that. --- ## How a run works ### Step 0 — check the bench (once per session) A binary on PATH is not proof it works. Codex can be installed, authenticated, and still refuse — expired login, or a plan quota that's been burned through. Only a real call tells the truth: ```bash codex exec --skip-git-repo-check -c model_reasoning_effort="low" "reply with exactly: OK" ``` Seat B is **available** only if this returns normal text. Treat it as unavailable if the output contains `usage limit`, `IneligibleTierError`, `invalid_token`, `error authenticating`, `quota`, `RESOURCE_EXHAUSTED`, or `command not found`. Probe once per session and remember the result — don't re-probe per call. **If Seat B is unavailable, say so in one line and stop for a decision:** ``` [cross-exam] Codex unavailable (usage limit) — no second architecture this run. Options: (1) wait / top up credits, (2) run both seats on Claude with different lenses (cheaper, but you lose the cross-model blind-spot coverage — which is the point). ``` Never silently fall back to two Claude seats. A single-architecture "council" is a different, weaker thing, and the user should know they're getting it. ### Step 1 — frame the question Restate the user's question as one neutral prompt both seats will receive. Strip leading framing ("why is X better" → "is X better") — models measurably bend toward the question's slant. Include, when they exist: - the actual constraint (budget, deadline, skill level, existing stack) - what's already been tried - what "good" looks like to the user Then show the framed question to the user in two or three lines before firing. A wrong frame wastes the whole run, and it's cheap to correct here. ### Step 2 — fire both seats in parallel Never sequentially — the point is independence. **Seat A (Claude):** a sub-agent via Task, given the framed question and this instruction: ``` Answer this question directly and specifically. Take a position. Don't hedge, don't present a balanced menu of options — say what you'd actually do and why. If you see a fatal flaw, lead with it. 200-350 words. No preamble. ``` **Seat B (Codex / GPT-5.6 Sol):** preferred route is the `codex:codex-rescue` agent if the Codex plugin is installed, with the prompt starting *"Read-only opinion task — review/diagnosis only, NO file edits, --effort medium."* Raw fallback (works without the plugin): ```bash codex exec --skip-git-repo-check -c model_reasoning_effort="medium" "" ``` Leave the model unset — `~/.codex/config.toml` already defaults to `gpt-5.6-sol`. Medium effort is correct here; a 300-word opinion doesn't need max reasoning, and high effort makes the run painfully slow. **Strip the CLI noise.** Codex prints hook lines, skill-budget warnings, and a preamble to stdout. Keep only the actual analysis — never paste log lines into the council as if they were content. ### Step 3 — blind cross-review Each seat now reads the *other* answer, unlabeled. Do not tell either model who wrote what, and do not tell them it's their own counterpart — models favor their own output when they can recognize it, and that self-preference is exactly what anonymity kills. Send each seat: ``` Here is another answer to the same question: --- [the other seat's answer] --- You are reviewing it, not defending anything. Answer three things, briefly: 1. What does it get right that matters? 2. What is wrong, unsupported, or missing? 3. If you had to act on it tomorrow, what would bite you? 150-250 words. Be specific. "Good points overall" is a failed review. ``` With only two seats there is no ranking stage and no majority vote — those need three or more members to mean anything. The cross-critique is what replaces them, and it is where most of the catches happen. ### Step 4 — chairman synthesis Claude, full reasoning, reads: the framed question, both answers, both critiques. Then writes the verdict. The chairman is not a summarizer. It resolves. When the two seats disagree, it decides *by reasoning* which is right and says why — never by splitting the difference, and never by "both make good points." Output format: ```markdown ## The verdict ## Where they agreed ## Where they clashed ## What the second model caught ## Do this first ``` **"Where they agreed" is the highest-value section** — two different architectures reaching the same conclusion independently is much stronger evidence than either one asserting it alone. Say so when it happens. If they converged completely, report that honestly. It's a real result: it means the question wasn't as uncertain as it felt. ### Step 5 — offer the transcript Offer to save the full run (framed question, both answers, both critiques, verdict) to `cross-exam--.md` in the working directory. Don't save unprompted. --- ## Honest limits — tell the user these when they matter - **It helps most on verifiable things.** Code, math, factual claims, catching an error. On creative and subjective work the effect is mostly variance reduction, not better output. - **It helps weaker models more.** Two cheap models gain a lot; two flagships gain less, because a flagship's first answer is already good. - **Two models can be wrong together.** Shared training data means correlated errors. A council reduces blind spots, it does not eliminate them, and it is not a fact-checker. - **It costs about 4 model calls and a minute.** Don't run it on every prompt. Run it when the answer matters. Never sell this as "X% smarter." The honest line: *ask two different AIs the same question, let each critique the other, and you reliably catch mistakes one AI is confident about — on things that have a real right answer.* ## Requirements - **Claude Code** — you're running it. - **Codex CLI** ≥ 0.144 — `npm install -g @openai/codex@latest`, then `codex login`. Uses your existing ChatGPT plan. Older CLI versions error with "requires a newer version of Codex" on `gpt-5.6-sol`. - No OpenRouter account, no API keys, no config file to edit. Codex usage is metered by your ChatGPT plan. On Plus you can hit a limit; the skill detects this at Step 0 and tells you instead of silently degrading.