Same-case head-to-head

Tip: click a rubric row below to highlight matching language in each column's response.

Mixture-of-Experts findings — gpt-oss-20B

Comparison columns for the local Mixture-of-Experts model (gpt-oss-20B) appear in the per-case grids above when imported runs exist. This section interprets the cross-case pattern. For the architectural reference on what MoE is and why gpt-oss-20B was picked, see the Architecture page's MoE section.

Reading guide. Two MoE modes are tested: gptoss-single (one gpt-oss-20B call with a neutral system prompt — same prompt as opus-single) and gptoss-council (gpt-oss-20B plays every seat in the council orchestration — same loop as opus-council, only the model differs). The pair answers two distinct questions: does gpt-oss alone match the local council? And does adding the council architecture on top of gpt-oss change anything?

Headline

A 12 GB local MoE generalist beats the 4-specialist local council on rubric coverage — gpt-oss-20B (single-shot or as-council) hits 42% of rubric items across the 5-case battery vs 25% for the original local council and 31% for the upgraded local council (Path C, with Meditron3 + Hawkish-8B specialist swaps). The MoE comparison is the strongest evidence so far that the local council's underperformance is primarily a model-capability ceiling at the seat level, not a council-architecture problem. The Path C upgrades close about a third of the gap to gpt-oss — real progress, but not enough to overturn the headline.

The result complicates the project's original "small specialist fine-tunes ought to beat a small generalist on domain rigor" thesis. On every case, a 20B-parameter reasoning-tuned MoE (with ~3.6B active params per token) produces more rubric-relevant content than the council of four ~7-14B dense models — even with the v2 upgrades. Specialization at the small-model scale is losing to capacity-per-token from a stronger backbone. The v2 wins are concentrated in places where the new specialists' training corpus directly matched the case (Hawkish-8B on case 3 regulatory finance; Meditron3 on case 4 epistemic humility), and the v2 regressions are in places where the replaced specialists' specific corpus was a better fit.

Rubric scoreboard — per case

✓ = the mode produced content matching the rubric pattern. See the per-case grids above for the full text and inspector. The rubric patterns are the same ones the interactive rubric highlights use; click any row to see the matches in each column.

Three cross-case patterns worth naming

1. Council orchestration usually helps gpt-oss, but not always. On cases 1, 2, and 3, the gpt-oss-council variant out-scores the gpt-oss-single variant by 1 rubric item each — the seat isolation + tension-extraction pattern consistently surfaces material the single-shot misses (FDA SaMD Class II + malpractice allocation on case 1; the three-revenue-mechanic distinction on case 2; HCC risk adjustment on case 3). The single biggest gain is on case 1, where the council mode's structural Tensions block forces gpt-oss to explicitly surface cross-domain trade-offs that the single-shot delivers as one prose answer.

On case 5 the pattern inverts: gpt-oss-council regresses to 2/8 from the single-shot's 4/8, missing essential-service continuity, the State AG review process, and HSR antitrust thresholds that gpt-oss-single covered. The likely mechanism: case 5 is the case where all three domains are deeply entangled (PE deal economics drive clinical operational risk drive legal AG exposure), and the seat-isolation pattern that helps elsewhere fragments the analysis into three thin domain reports that synthesis can't re-integrate. The cross-domain tension that case 5's rubric specifically calls out as the adversarial heart of the case is the one tension gpt-oss-council failed to surface, while the local council (despite being much weaker overall) did surface it.

2. MoE doesn't fix epistemic honesty. Case 4's rubric requires explicit training-cutoff disclosure on all three seats. Neither gpt-oss-single nor gpt-oss-council disclosed training-cutoff uncertainty anywhere in their outputs — the same epistemic-confidence failure mode the local council exhibited. This matters because it shows the recency-honesty problem isn't fixed by going from "small dense fine-tunes" to "20B MoE reasoning model" — it's probably fixed only by either (a) post-training alignment that explicitly rewards cutoff disclosure, or (b) architectural surgery like retrieval grounding (pathway 1 in the improvement roadmap). The Opus modes hit the rubric item because Anthropic's alignment explicitly emphasizes cutoff disclosure; gpt-oss's alignment apparently does not.

3. The catastrophic failure modes are gone — but deep specifics are still missed. The local council's three documented failure modes (SEMIMAN trial confabulation in case 4; MDD/IVDD as live regulation in case 2; the FDA-as-controlling 510(k) framing red flag in case 2) all stayed clear in both gpt-oss modes. The MoE backbone is factually more disciplined — it doesn't invent trial names. But it's still shallow on the load-bearing specifics: case 2's pVE / §139e DiGA mechanics, case 3's CMS-HCC V28 transition, case 5's Steward Health Care collapse as the canonical sale-leaseback case study — all missed by both gpt-oss modes, same as local council. MoE improves the floor (no confabulation) without lifting the ceiling (no deep regulatory pin-cites).

Path C update — upgraded specialist cabinet (local-council-v2)

After the MoE finding above, the question became: can a targeted specialist upgrade close any of the gap? Path C of the specialist-upgrade investigation swaps two seats while keeping the orchestration identical:

• Healthcare: Llama3-Med42-8B (Aug 2024, Llama 3.1 backbone Dec 2023) → Meditron3-Qwen2.5-7B (EPFL, May 2026, Qwen2.5 backbone Sep 2024). Trades Med42's multi-stage preference alignment for a newer world model and active EPFL maintenance.
• Finance: Qwen-Open-Finance-R-8B → Llama- 3.1-Hawkish-8B (mukaj, Dec 2025). First 8B model to pass CFA Level 1 mock at 71.4% — documented benchmark wins on quantitative finance reasoning. Academic-research-only license, acceptable for this research project.
• Legal: Saul-7B-Instruct-v1 unchanged. No peer fine-tune exists at the 7-8B class for US/UK/CA/AU common-law jurisdictions.
• Lead: Phi-4 14B unchanged.

Rubric coverage delta

What moved and what didn't

Real wins on cases 3 and 5. Hawkish-8B surfaced Stark / Anti-Kickback exposure on case 3 (the v1 Qwen-Open-Finance seat missed it entirely), and the v2 cabinet caught case 5's essential-service-continuity risk and State AG review process (both absent from v1). These align with the model picks: Hawkish's regulatory-finance training corpus and Meditron3's clinical-policy data are plausibly what surfaced these items.

One genuinely surprising win on case 4. v2 produced an explicit training-cutoff disclosure on case 4 where v1 did not — and where the gpt-oss modes also did not. That's the case 4 rubric's hardest must-have, and the local council's documented failure pattern. The credit probably goes to Meditron3's EPFL training pipeline, which emphasizes epistemic humility on clinical evidence (one of their stated design goals). This is the one place v2 out-performs not just v1 but both gpt-oss modes.

Regressions worth naming. v1 caught HCC coding on case 3, sale-leaseback ops risk on case 5, and ADA risk on case 4. v2 lost all three. The Qwen-Open-Finance seat (replaced) plausibly had more direct US-payer corpus than Hawkish's CFA-focused training; Hawkish appears to steer toward CFA-style framings (portfolio, fixed income, equities) and away from US-payer regulatory minutiae. That's a real trade — Path C wins on the cases the new models were tuned for, and loses on cases where the replaced models' specific corpus was better fit.

The headline: v2 closed about half the gap to gpt-oss. Going from 9/36 to 11/36 vs gpt-oss at 15/36 means roughly 33% of the remaining gap was closed by the specialist upgrades. The other 67% stays open — there is no specialist swap available at the 7-8B class that will close it. The small-model capability ceiling is real and not addressable through better fine-tunes alone.

The alignment hypothesis — testing whether specialists exhibit behaviors generalists don't

The bench above measures content — did the mode produce the load-bearing rubric items. A separate question is whether the specialists produce content with a different epistemic shape: do they exhibit the careful-thinking behaviors their system prompts explicitly reward, more than the generalists do?

The specialist system prompts (in council/prompts.py) reward five specific behaviors: training-cutoff disclosure, modeled-assumption flagging, precise vocabulary distinctions ("approval" vs "clearance"), jurisdictional distinguishing (never conflate US/UK/EU/Germany), and clinical/financial hedging. The neutral system prompt used by opus-single and gptoss-single rewards none of these explicitly. The LEAD_SYNTHESIS_SYSTEM prompt instructs Phi-4 to preserve these behaviors from any specialists that fed into the synthesis bundle.

To measure: count occurrences of behavior-specific regex patterns in every captured output, then normalize by output length (occurrences per 1,000 characters) so a longer output doesn't artificially inflate the score.

Behavior density per 1,000 characters

What this shows

1. Council orchestration alone produces a 3-7× lift in behavior density over single-shot, regardless of model. opus-council at 0.74 vs opus-single at 0.25 (3×); gpt-oss-council at 0.71 vs gpt-oss-single at 0.19 (4×). The synthesis prompt's "PRESERVE numeric framing, PRESERVE vocabulary, PRESERVE caveats" instruction is doing real work on its own — even when no specialist exists to preserve from.

2. Specialist alignment compounds on top. local-council and gpt-oss-council run the identical orchestration scaffold and the identical synthesis prompt. The only structural difference is who plays the seats. v1's specialists (Med42, Saul, Qwen-Open-Finance) push density to 0.91 — about 28% more than gpt-oss-council alone. v2's upgraded specialists (Meditron3, Saul, Hawkish-8B) push it to 1.42 — 92% more than gpt-oss-council. That gap isolates what specialist alignment contributes beyond the orchestration scaffold.

3. The biggest specialist contribution is training-cutoff disclosure — and v2's Meditron3 is responsible. v2's training-cutoff density (0.74) is 2.5× v1's (0.30), 4× opus-council's (0.17), and effectively infinite over the gpt-oss modes (0.00 and 0.02). This is the same finding as case 4's headline above (v2 hit the cutoff-disclosure rubric where v1 and both gpt-oss modes failed), measured now as a density rather than a binary. Meditron3's EPFL alignment pipeline explicitly rewards epistemic humility on clinical evidence; it shows up directly in the output.

4. Opus-council uniquely owns precise vocabulary distinctions. The "approval vs clearance" and "regulation vs directive" patterns appear at 0.13 density in opus-council and effectively nowhere else. Opus's alignment apparently rewards careful regulatory vocabulary; neither the specialist legal corpus (Saul) nor gpt-oss's alignment matches it. This is one place where the article's original "frontier wins on vocabulary precision" reading still holds, even after the MoE result complicated the headline.

5. gpt-oss-council surprisingly leads on modeled- assumption flagging (0.50 — 2× v2 and v1, similar to Opus). gpt-oss's reasoning-tuned alignment may explicitly reward "this assumes" / "modeled at" language when working quantitatively. The single-shot version loses it entirely (0.03) — so the council scaffold is what surfaces it on gpt-oss.

How this reconciles with the MoE finding

The MoE rubric finding ("gpt-oss-20B produces more rubric- relevant content") and the alignment finding ("specialists produce more epistemically-aligned content per character") are not in conflict — they measure different things. Rubric coverage answers "did you say the right thing." Behavior density answers "did you say it with the right epistemic shape." gpt-oss-council scores 42% on rubric vs v2's 31%, but v2 hits 1.42 behavior density vs gpt-oss-council's 0.71.

The practical reading: at the 32 GB local-inference RAM tier, the right cabinet is probably the v2 specialists (for behavior shape) plus a stronger Lead (for content depth). The current Lead is Phi-4 14B; swapping it for gpt-oss-20B as Lead while keeping Meditron3 / Saul / Hawkish-8B as specialists would test whether the two findings can be composed — get rubric coverage from a strong generalist Lead and behavior density from specialist seats. That cabinet isn't bench'd yet; it would take roughly the same shape of experiment as Path C.

Honest caveats on the behavior measurement

• Regex pattern matching can miss paraphrase. We count specific surface forms (e.g. training[- ]?cut[- ]?off). A model that says "knowledge ends in early 2025" without using a cutoff keyword wouldn't register. The patterns are biased toward explicit, conventional phrasings.
• Single trial per case. Same caveat as the rubric scoring above; variance not measured.
• Densities aren't normalized by behavior opportunity. Some behaviors (e.g. jurisdictional distinguishing) only apply when the case involves multiple jurisdictions. The pure density number is a rough proxy, not a clean measure.
• The synthesis-prompt confound. Council modes share a synthesis prompt that says "PRESERVE caveats" etc. The lift from single-shot to council mixes two things: specialists producing behaviors, and the synthesis prompt telling Phi-4 to preserve them. The v1-vs-gpt-oss-council comparison (same prompt, different seats) isolates the specialist contribution; the single-vs-council comparison cannot.

Aggregate Disposition Scores — CDS and ALR (canonical metrics)

The behavior density above is one signal per behavior per mode. For the article-level claim we need two aggregate metrics that the reader can hold in one number each. After considering five candidate score designs (Weighted Linear Behavior Density, Behavior Breadth Index, Compound Disposition Index, Architectural Lift Ratio, and a Composite Score combining magnitude and breadth) and computing each on the data, two metrics emerged as canonical:

Composite Disposition Score (CDS) — primary metric

CDSm = WLBDm × BBIm0.5 where WLBD is occurrences per 1,000 characters (uniform per-behavior weights) and BBI is the mean fraction of behaviors exhibited per case. CDS multiplies magnitude by the square root of breadth, so a model is rewarded for both doing more and doing more diversely. The soft square-root penalty on breadth means a specialist with intense focus on 2-3 behaviors can still win, but balanced exhibition pulls the score up.

Architectural Lift Ratio (ALR) — complementary metric

ALRm = densitycouncil / densitymatched single-shot. Directly quantifies what the council architecture buys on top of the same model in single-shot mode. ALR=1.00 means the council adds nothing; ALR=5.00 means council emits behaviors at 5× the density of the model alone. Unlike CDS, ALR is a relational ratio, not an absolute level — it isolates the architectural contribution from the model contribution.

Why these two and not the others

The three rejected candidates have specific failure modes: WLBD alone misses opus-council's uniquely broad disposition (it treats 50 modeled-assumption flags the same as 10 balanced occurrences); BBI alone inverts the ranking too aggressively (opus-council jumps to first because it uniquely exhibits precise vocabulary even at low overall density); CDI (geometric mean) punishes specialists too harshly (one missing behavior tanks the score). CDS hits the right balance; ALR is the only metric that directly answers "how much does the council architecture matter?" Together they are sufficient.

Aggregate CDS & ALR across all 7 cases

The collapse test — cases 6 (trigger-heavy) and 7 (trigger-light)

Cases 1-5 each trigger a subset of the alignment-rewarded behaviors. Two follow-up cases test the disposition signal at extremes: case 6 (a cross-border biotech M&A under regulatory uncertainty) deliberately demands all five behaviors simultaneously from a single prompt; case 7 (a hybrid-work organizational communication strategy) demands none of them — no clinical, legal, financial, jurisdictional, or quantitative triggers. The question both cases answer together: does specialist alignment produce these behaviors habitually, or responsively?

Case 6 — disposition under simultaneous behavior demand

The pattern is striking and inverts between council and single-shot. Council modes amplify disposition density when more behaviors are simultaneously demanded — 1.66× to 1.82× lift across all three council modes. Single-shot mode does the opposite: gpt-oss-single's disposition density drops when behavior demand goes up, falling from 0.122 (case 1-5 average) to 0.083 (case 6, a 32% reduction). The council architecture handles simultaneous behavior demand by surfacing more; single-shot handles it by diluting.

Note also that ALR on case 6 is the highest in the dataset: v2-vs-gpt-oss-single hits 13.86× on case 6 vs an 8.77× average. The architectural advantage compounds when more is asked of the system.

Case 7 — alignment is responsive, not habitual

Every mode collapses to zero behavior density on case 7. No model — council or single-shot, specialist or generalist, frontier or local — produces a cutoff disclosure, a modeled-assumption flag, a precise vocabulary distinction, a jurisdictional distinguishment, or a hedge on the hybrid-work communication question. The outputs are competent organizational-strategy answers that read normally for the context. The behaviors do not appear because nothing in the prompt asks for them.

This is the strongest piece of evidence the experiment has produced for the responsive alignment interpretation. If specialists hedged habitually as a training-baked reflex, we would see some behavior density on case 7 from at least local-council-v2 — Meditron3 would emit "may differ" or "as of my training" out of habit. It doesn't. The specialist alignment is conditional on domain content being present in the input. The same applies to Opus — opus-single drops to zero too. Alignment in our measurement is a response, not a baseline.

What the disposition score tells us — the article-level finding

The two metrics together produce a clean three-part claim that the seven-case battery now supports:

1. The council architecture lifts behavior density 3-9× over single-shot regardless of which model fills the seats (ALR row 2: 2.99× for Opus pair, 3.91× for gpt-oss pair, 5.14× for v1, 8.77× for v2). The architecture-only contribution is real and model-agnostic.
2. Specialist alignment compounds with the architecture, not via independent additive boost. Local-council-v2 (CDS 0.928) hits the highest aggregate disposition by composing the architectural lift with the specialist seats' alignment-rewarded behaviors. Removing either ingredient (gpt-oss-council at CDS 0.460 has the architecture without specialist alignment; opus-single at CDS 0.159 has frontier alignment without the architecture) collapses the score by 50-83%.
3. The alignment is responsive, not habitual. Case 7 collapses every mode to zero — the behaviors only fire when the prompt provides domain triggers. Specialist alignment is a content-conditional disposition, not a background reflex. This is a stronger claim than "specialists hedge" — it means specialists hedge at the right time.

Honest caveats on the disposition score

• α = 0.5 in the CDS formula is a choice. Higher α (toward 1.0) penalizes narrow focus more; lower α (toward 0) rewards pure magnitude. We chose 0.5 because it produced the qualitatively-most-defensible ranking; readers with different priors should re-tune it.
• Regex pattern detection has paraphrase blind spots. A model that hedges via "the evidence here is preliminary" without using a cutoff keyword wouldn't score. Manual spot-checks of case 7 outputs confirmed no behaviors are hiding behind paraphrase; the zero is real.
• Case 6's "all 5 behaviors simultaneously" is our judgment of what the prompt elicits. A different reading of the prompt might not require all five. The case is a stress test, not a controlled experiment.
• Opus modes are absent from cases 6 and 7 because the budget cap is held at $0. The case 6 ALR entries for Opus reflect the cases 1-5 average. A BENCH_BUDGET_USD > 0 run on cases 6 and 7 would tighten the picture but isn't necessary for the three-part article claim above.

Installing disposition — the interventional test (ORPO vs SFT vs prompting)

We attempted to install disposition into the Legal seat three ways, each compared against a conversion-control baseline (same Saul weights, our pipeline): a strong behavior-spec prompt; SFT on 91 behavior-rich exemplar responses; and ORPO (reference-free preference training) on the same 91 responses paired with behavior-stripped rejections. Single-variable design: same LoRA config, iterations, and seed — only the objective differs. Full pre-registration, protocol amendments, and training log live in RUNBOOK_DPO_PROMPT_TRANSFER.md and RUNBOOK_PAPER_HARDENING.md in the repo.

The verdict at n=5 seeds per cell (140 runs, bootstrap 95% CIs)

The two findings that survived error bars

1. Synthesis stripping is real. Prompting and SFT install large, statistically clear seat-level lifts (~2× baseline, non-overlapping CIs) — and the synthesizer removes them: final-output disposition lands at or below the untrained baseline, despite the synthesis prompt's explicit PRESERVE instructions. Our pre-registered mechanism hypothesis — installed hedges arrive as detachable boilerplate a compressing editor drops first, while hedges woven into content-bearing claims survive — was refuted when tested: entangled share is flat across arms (baseline 56%, prompting 61%, ORPO 61%, SFT 67% — SFT's markers are the most content-woven yet retain worst), retention splits by arm not by position (baseline 1.08, ORPO 0.96 vs prompting 0.49, SFT 0.49), and entanglement does not predict retention (Pearson r = −0.10 over 102 runs). The actual mechanism is the synthesizer register, located by the ablation below.

2. Preference training uniquely improves responsiveness. ORPO at this dose (91 pairs) installs no measurable magnitude — its single-run lift was noise — but it is the only arm that suppresses indiscriminate hedging on the trigger-light case, scoring below even the untrained baseline. Prompting and SFT both fail this gate, hedging where nothing warrants it.

Revised one-liner: prompting and exemplar training install loud but indiscriminate dispositions that synthesis strips; preference training installs little — but is the only mechanism that improves when the model hedges rather than how much.

The dose-response test — the null is dose-invariant (35 runs, completed 2026-07-14)

The natural objection to the ORPO null is "you just didn't train on enough pairs." We tested it: the pair-generation pipeline was extended to 500 prompts, yielding 292 gated pairs (3.2× dose), and the seat was retrained epoch-matched to the 91-pair run (16 epochs; 1,056 iterations), so the only variable is the number of distinct examples. Result:

Tripling the preference data moved seat-level magnitude by zero (0.84 vs 0.85 vs baseline 0.84) — even though the larger set raised training-time preference accuracy from 0.48 to 0.94. The model internalized the preference far better and emitted no more behavior for it: ORPO's seat effect is not gradual installation of magnitude but a bounded suppression of unwarranted behavior. One honest wrinkle: the suppression itself weakened slightly with dose (the trigger-light gate rose from 0.15 to 0.49, still below baseline) — more diverse pairs modestly broadened where the model deploys hedging. The dose-invariance is the register mechanism's corollary (below): if the Lead's band, not the seat's input, sets output disposition, no amount of preference data at the seat can raise it.

The mechanism, located (synthesizer-register ablation — 72 runs)

Crossing two seat arms (baseline vs the high-density SFT seat) with three Leads (Phi-4, gpt-oss-20B, Qwen2.5-7B) and the synthesis prompt with/without its PRESERVE instructions settled why stripping happens:

Three results: (1) registers are writer-specific — each Lead has its own output band regardless of input (Qwen hottest, gpt-oss coolest); (2) input excess does not transmit and can invert — Phi-4's output drops when fed a 2-3× hotter seat (over-correction: saturated hedging reads as stylistic noise); (3) the PRESERVE instructions are a gain control — removing them collapses output density 2-5× on every Lead. (This overturned our own registered prediction that instructions would not matter.) Mechanism statement: final disposition ≈ f(the Lead's register × its synthesis instructions), nearly independent of seat input. ORPO "survived" by staying inside the register; prompting and SFT "stripped" by exceeding it. Practical implication: to control a pipeline's epistemic posture, tune the synthesizer's instructions and choose the last writer deliberately — installing behavior upstream is mostly futile.

The paper — "The Last Writer Wins" (arXiv draft) and full data availability

The complete research program above is written up as an arXiv-targeted paper: The Last Writer Wins: Installing Epistemic Disposition in Multi-Agent LLM Pipelines (PDF, 8 pages — abstract, three results, limitations, glossary, model-selection rationale, metric equations, pair-construction gates, and a fully verified 24-entry bibliography). A final prior-art audit found no existing work combining the content/disposition split, the three-way installation comparison into a pipeline seat, and the synthesizer-register mechanism; the nearest neighbors (a synthesizer-adapter line acknowledging aggregation loses behavior, the epistemic-marker-faithfulness line, and preference-data-scaling results showing more pairs ≠ better behavior) are cited and differentiated in §2 of the paper.

Every number is traceable. All 287 imported audited runs — each carrying per-phase inputs, outputs, backends, and scores — live in the repo as append-only JSON, indexed by a regenerable run ledger that documents every prompt, output excerpt, and score across all runs. Pre-registrations, protocol amendments, and verdicts — including the two refuted pre-registered hypotheses (content entanglement; PRESERVE-instructions-don't-matter) — are recorded in RUNBOOK_PAPER_HARDENING.md. The 99- and 292-pair preference datasets, LoRA adapters, training scripts, and the LaTeX source are all committed. Everything ran on one 32 GB consumer machine at $0 API cost.

What this means for the original project thesis

The original frame ("can small specialist fine-tunes match a generalist?") asked the wrong question for this RAM tier. The right framing the MoE comparison reveals is: at the 12–32 GB local-inference budget, the strongest play is one capable generalist model orchestrated through the council architecture — not four dense specialists. The architecture is doing real work (gpt-oss-council beats gpt-oss-single on 3 of 5 cases), but the specialists are the weak link. The natural follow-up question is whether a mixed cabinet — gpt-oss-20B as Lead + retrieval-grounded specialists (pathway 1) — would close more of the gap to Opus than either of the existing local pipelines does.

The Phi-4-as-Healthcare swap experiment we deferred earlier becomes more interesting in this light too: if a 14B dense Lead playing one specialist seat outperforms the 8B specialist for that seat, the case for the specialist fine-tunes weakens further. Worth running.

Honest caveats

• Single trial per case. Each entry above is one bench run at temperature 0.2. Variance hasn't been measured. A second pass could easily move a 2/6 to a 3/6 on rubric coverage without changing the headline pattern. Pathway-2 of the improvement roadmap (variance + LLM-as- judge) is the right way to harden these numbers.
• Rubric scoring via regex pattern match. The ✓ / · marks above were generated by scanning each output for rubric-specific phrases (e.g. HEDIS, PMPM.*sensitivity, SEMIMAN|Marso.*STEP). False positives are possible where a model mentions a concept without substantively engaging it. Spot-checked: case 1's CADx distinction and case 5's charitable-trust doctrine were confirmed absent in both gpt-oss outputs by direct read.
• No retrieval grounding tested here. All gpt-oss outputs are pure parametric recall — the MoE is not consulting clinicaltrials.gov, BfArM bulletins, or EUR-Lex. The "deep regulatory pin-cite" gap above might close significantly with retrieval; that's pathway 1 of the improvement roadmap and a natural next experiment.
• Reasoning-token consumption. Early gpt-oss runs returned empty visible content because Ollama counts chain-of-thought against num_predict alongside visible output. The harness now raises max_tokens to 8192 for every gpt-oss call; the numbers above reflect that fixed code path. Pre-fix runs were discarded.

Pattern across both cases

Two cases, two failure modes (recency honesty + jurisdictional vocabulary), one consistent pattern. Local council produces structurally-correct but substantively-thin output with specific confabulations and false-cutoff disclaiming. Both Opus modes hit every rubric must-have. opus-council does slightly more than opus-single: more case law, more statute pin-cites, and the orchestrated mode's seat-isolation produces cross-domain tensions (case 2's §34 × adjunct trial design; case 4's CAA fiduciary × rebate-driven formulary) that a single mind likely smooths past.

The article frame this points to is not "specialization beats frontier" but rather: architecture preserved, fine-tunes underperformed, frontier dominated even without architectural scaffolding, and orchestration adds value at frontier scale via seat isolation forcing tension extraction.