Cycle 2

18 findings. Ordered by priority: invariants, calibration, degeneracy, plausibility, saturation, sensitivity, framework.

F-cycle-002-01: The backcast ends trade and investment 1.9 points too low against the 2026 ratings.

Layer: parameter. Class: calibration_error. Source: rules

Evidence: checks.parquet backcast_bias:trade_and_investment: trade_and_investment: simulated -1.92 points of change on average against -0.04 observed (bias -1.88, error 3.35 against 1.76 for persistence, direction right 42 per cent) (calibration nations; reconstructed 1996 starting positions) [backcast_comparison].

Hypothesis: Capability in this domain decays too fast for what nations actually spent on it, or costs too much to build: the model cannot reproduce the gains history shows.

Proposed change: Multiply the trade_and_investment class's decay rates by 0.75 in data/sim/seed/params_v0_1.yaml

Scope: parameter

Expected effect: |bias| in trade_and_investment under 1 point on the calibration nations, without worsening the hold-out.

Verification: Rerun the backcast; accept if backcast_bias:trade_and_investment passes and backcast_skill_holdout does not fall.

Risk: Changes every nation's trajectory in this domain, not only the backcast's; the reconstruction itself may be what is wrong.

F-cycle-002-02: The backcast ends financial strength 1.8 points too low against the 2026 ratings.

Layer: parameter. Class: calibration_error. Source: rules

Evidence: checks.parquet backcast_bias:financial_strength: financial_strength: simulated -1.87 points of change on average against -0.04 observed (bias -1.83, error 3.32 against 1.79 for persistence, direction right 35 per cent) (calibration nations; reconstructed 1996 starting positions) [backcast_comparison].

Hypothesis: Capability in this domain decays too fast for what nations actually spent on it, or costs too much to build: the model cannot reproduce the gains history shows.

Proposed change: Multiply the financial_strength class's decay rates by 0.75 in data/sim/seed/params_v0_1.yaml

Scope: parameter

Expected effect: |bias| in financial_strength under 1 point on the calibration nations, without worsening the hold-out.

Verification: Rerun the backcast; accept if backcast_bias:financial_strength passes and backcast_skill_holdout does not fall.

Risk: Changes every nation's trajectory in this domain, not only the backcast's; the reconstruction itself may be what is wrong.

F-cycle-002-03: The backcast ends information and influence 1.7 points too low against the 2026 ratings.

Layer: parameter. Class: calibration_error. Source: rules

Evidence: checks.parquet backcast_bias:information_and_influence: information_and_influence: simulated -1.70 points of change on average against -0.03 observed (bias -1.67, error 2.73 against 1.93 for persistence, direction right 50 per cent) (calibration nations; reconstructed 1996 starting positions) [backcast_comparison].

Hypothesis: Capability in this domain decays too fast for what nations actually spent on it, or costs too much to build: the model cannot reproduce the gains history shows.

Proposed change: Multiply the information_and_influence class's decay rates by 0.75 in data/sim/seed/params_v0_1.yaml

Scope: parameter

Expected effect: |bias| in information_and_influence under 1 point on the calibration nations, without worsening the hold-out.

Verification: Rerun the backcast; accept if backcast_bias:information_and_influence passes and backcast_skill_holdout does not fall.

Risk: Changes every nation's trajectory in this domain, not only the backcast's; the reconstruction itself may be what is wrong.

F-cycle-002-04: The backcast ends production and innovation 1.6 points too low against the 2026 ratings.

Layer: parameter. Class: calibration_error. Source: rules

Evidence: checks.parquet backcast_bias:production_and_innovation: production_and_innovation: simulated -1.63 points of change on average against -0.04 observed (bias -1.59, error 2.92 against 1.50 for persistence, direction right 33 per cent) (calibration nations; reconstructed 1996 starting positions) [backcast_comparison].

Hypothesis: Capability in this domain decays too fast for what nations actually spent on it, or costs too much to build: the model cannot reproduce the gains history shows.

Proposed change: Multiply the production_and_innovation class's decay rates by 0.75 in data/sim/seed/params_v0_1.yaml

Scope: parameter

Expected effect: |bias| in production_and_innovation under 1 point on the calibration nations, without worsening the hold-out.

Verification: Rerun the backcast; accept if backcast_bias:production_and_innovation passes and backcast_skill_holdout does not fall.

Risk: Changes every nation's trajectory in this domain, not only the backcast's; the reconstruction itself may be what is wrong.

F-cycle-002-05: The backcast ends national security 1.0 points too low against the 2026 ratings.

Layer: parameter. Class: calibration_error. Source: rules

Evidence: checks.parquet backcast_bias:national_security: national_security: simulated -0.98 points of change on average against +0.04 observed (bias -1.02, error 2.29 against 1.78 for persistence, direction right 44 per cent) (calibration nations; reconstructed 1996 starting positions) [backcast_comparison].

Hypothesis: Capability in this domain decays too fast for what nations actually spent on it, or costs too much to build: the model cannot reproduce the gains history shows.

Proposed change: Multiply the national_security class's decay rates by 0.75 in data/sim/seed/params_v0_1.yaml

Scope: parameter

Expected effect: |bias| in national_security under 1 point on the calibration nations, without worsening the hold-out.

Verification: Rerun the backcast; accept if backcast_bias:national_security passes and backcast_skill_holdout does not fall.

Risk: Changes every nation's trajectory in this domain, not only the backcast's; the reconstruction itself may be what is wrong.

F-cycle-002-06: The backcast does worse than assuming nothing changed since 1996.

Layer: mechanic. Class: calibration_error. Source: rules

Evidence: checks.parquet backcast_skill_calibration: 48 calibration nations: simulated 2026 groups miss the ratings by 2.66 points on average, against 1.69 for assuming nothing changed (skill -0.57); bias -1.00; the change in each domain has the right sign 39 per cent of the time; index error 1.97 against 1.25 [backcast_comparison].

Hypothesis: The model moves capability by the wrong amounts or in the wrong places: compare simulated with observed change by domain.

Proposed change: Treat the domain biases first; then test whether the credit calibration (income proportional to the cost of the starting position) lets low-rated nations catch up as history shows.

Scope: structural

Expected effect: backcast_skill_calibration above 0.

Verification: Rerun the backcast; accept if skill is positive on the calibration nations and not worse on the hold-out.

Risk: Over-fitting to reconstructed starting positions.

F-cycle-002-07: The same move is repeated in most rounds in 5 experiments.

Layer: harness. Class: degenerate_strategy. Source: rules

Evidence: checks.parquet same_action: e4-rus 63% of rounds; e4-nga 63% of rounds; e4-phl 77% of rounds; e4-per 77% of rounds; e4-kaz 73% of rounds [decision_sequence].

Hypothesis: The mirror-peer policy measures its gaps against current scores and ignores levels still in delivery, so it keeps funding a gap it has already paid to close; near its horizon greedy can only value fast (bought) moves, so it repeats them.

Proposed change: Count levels still in delivery when mirror_peer measures a gap and when greedy measures the room left below 20 (pipeline/sim/policies).

Scope: harness

Expected effect: same_action passes for the mirror-peer experiments.

Verification: Rerun e3-*-mirror-peer; accept if same_action passes for all five nations.

Risk: Changes the baselines' behaviour, so cycle-on-cycle comparison of those experiments breaks once.

F-cycle-002-08: Groups move more than 3 points in a year without any shock.

Layer: parameter. Class: implausible_dynamics. Source: rules

Evidence: 3 experiments: e4-jpn move_without_shock 3.67; e4-rus move_without_shock 3.65; e4-isr move_without_shock 3.48 (checks.parquet) [decision_sequence]. Of the 130 group-years that moved more than 3 points without a shock, investment arriving averaged 3.43 points and propagation +0.004: the moves are investment, not the graph.

Hypothesis: A move is a share of income, and income scales with the cost of holding the whole position, so the largest chunk buys several levels of a weak group in one year.

Proposed change: Lower chunk_large from 0.05 to 0.03 in data/sim/seed/params_v0_1.yaml (model).

Scope: parameter

Expected effect: move_without_shock at or below 3 points in the E3 experiments that fail it.

Verification: Rerun e4-isr, e4-jpn, e4-rus; accept if the share of failing experiments falls by half.

Risk: Slower growth for every nation and policy; less of the headroom is spent, so carried credits rise.

F-cycle-002-09: Domains gain more than two tiers within five years, including under the hold policy, which makes no moves.

Layer: methodology. Class: methodology_question. Source: rules

Evidence: 33 experiments: e4-rus 11 tiers; e4-grc 8 tiers; e4-arg 7 tiers; e4-irl 7 tiers; 0 of them hold experiments (checks.parquet) [tier_transitions].

Hypothesis: Below Tier 1 the Pareto layers of a domain are close together (a tolerance of a quarter of a point across 197 nations), so a small move crosses several layers. The heuristic's two tiers may suit coarse tiers, not Pareto layers.

Proposed change: Decide how the model should read tier movement: against the Index's published tier bands, with a wider tolerance, or not at all.

Scope: methodology

Expected effect: A tier measure that moves only when capability moves materially.

Verification: Rerun the cycle batch with the chosen measure; accept if tier_jump passes for the hold experiments.

Risk: Changing the tiering here would make results incomparable with the Index's own tiers.

F-cycle-002-10: Groups are driven to 20 and held there in many runs, across nations and policies.

Layer: parameter. Class: saturation. Source: rules

Evidence: 12 experiments above 25 per cent of runs: e4-sgp 100%; e4-irl 100%; e4-twn 100%; e4-isr 100%; e4-fin 100%. Groups most often driven to 20: National Resilience & Crisis Management (133), Semiconductors & Advanced Computing (120), Health System Capability (120), Energy Technologies (90), Communications & Digital Infrastructure (90) [trajectory_by_domain].

Hypothesis: Costs above the frontier (17) do not steepen enough to stop a well-funded nation buying the last levels.

Proposed change: Raise frontier_premium from 0.6 to 0.9 (model).

Scope: parameter

Expected effect: The share of runs ending with a group newly at 20 falls below 25 per cent in the greedy E3 experiments.

Verification: Rerun the E3 greedy experiments; accept if saturation_high passes in at least half of those that failed.

Risk: Frontier nations gain less; the frontier parameter is already the most sensitive.

F-cycle-002-11: Groups of low-rated nations are driven to 0 by shocks and stay there.

Layer: mechanic. Class: saturation. Source: rules

Evidence: e4-est: 97% of runs end with a group at 0 that did not start there (median 1 groups); e4-chl: 100% of runs end with a group at 0 that did not start there (median 1 groups); e4-fin: 100% of runs end with a group at 0 that did not start there (median 1 groups); e4-tur: 100% of runs end with a group at 0 that did not start there (median 1 groups) (checks.parquet) [shock_damage_by_group].

Hypothesis: Shock damage is in points regardless of the group's score, so a group rated near 1 loses everything to one shock; nothing rebuilds it unless the policy chooses it.

Proposed change: Scale a shock's points on each group by min(1, score / 10) in model.event_scale, so damage is proportional for weak groups.

Scope: structural

Expected effect: saturation_low passes for KEN.

Verification: Rerun e3-ken-*; accept if saturation_low passes and the shock tests of the scenario library are unchanged for nations above 10.

Risk: Departs from the scenario load test, which is in points; needs approval.

F-cycle-002-AI01: All nine domain decay rates produce simulated 1996-2026 changes below observed, with biases from -0.43 to -1.88.

Layer: parameter. Class: calibration_error. Source: ai:deepseek-v4-pro

Evidence: [backcast_by_domain] biases: trade_and_investment -1.88, financial_strength -1.83, information_and_influence -1.67, production_and_innovation -1.59, national_security -1.02; all others negative.

Hypothesis: The decay rates are uniformly overestimated as a family, not five independent domain errors.

Proposed change: Scale all built and bought decay rates by 0.75 and rerun the backcast before any domain-specific changes.

Scope: parameter

Expected effect: Domain biases move closer to zero and MAE falls below 1.5 for most domains.

Verification: Check backcast_by_domain bias and MAE; require no domain bias below -0.5.

Risk: May understate decay for domains with genuinely faster erosion; monitor whether high-end saturation worsens.

F-cycle-002-AI02: Low-rated nations lose capability because income is calibrated to the cost of the starting position, so they never accumulate credits for catch-up.

Layer: mechanic. Class: calibration_error. Source: ai:deepseek-v4-pro

Evidence: [research_notes] fastest risers observed +3.5 vs simulated -1.4; [backcast_by_nation] CHN bias -10.18, IND -6.60, KEN -2.76, GEO -4.76.

Hypothesis: The credit function's base calibration is the primary block to catch-up growth.

Proposed change: Replace base_credits with a country income term not tied to the 2026 score, or add a convergence bonus for groups below their historical potential.

Scope: structural

Expected effect: Emerging economies with low starting scores can sustain positive moves and reproduce observed rises.

Verification: Rerun backcast for CHN, IND, KEN, GEO; require biases above -1.5 and rising directions positive.

Risk: Could over-boost convergence and produce unrealistic growth in weak states; bound any bonus.

F-cycle-002-AI03: One-year moves exceed 3 points with no shock in ISR and RUS in 100% of runs.

Layer: parameter. Class: implausible_dynamics. Source: ai:deepseek-v4-pro

Evidence: [checks] move_without_shock: e4-isr largest one-year move 3.48, e4-rus 3.65, 100% above 3; [model_params] chunk_large 0.05.

Hypothesis: chunk_large=0.05 is too large relative to the 0-20 scale and convexity.

Proposed change: Lower chunk_large to 0.03.

Scope: parameter

Expected effect: Maximum no-shock one-year moves fall below 3 points.

Verification: Rerun move_without_shock check; require no run above 3.

Risk: May slow legitimate large investments; test medium-term responsiveness.

F-cycle-002-AI04: Five nations end 100% of runs with newly saturated groups at 20, medians 1-5 groups.

Layer: parameter. Class: saturation. Source: ai:deepseek-v4-pro

Evidence: [checks] saturation_high: e4-sgp median 4, e4-isr median 5, e4-fin median 3, e4-twn median 1, e4-irl median 1.

Hypothesis: frontier_premium=0.60 and convexity are too low to make top levels costly enough.

Proposed change: Raise frontier_premium to 1.0 and increase domain convexity by 0.5 for domains that saturate.

Scope: parameter

Expected effect: Fewer runs end with groups at 20; saturated_high_share falls below 0.5.

Verification: Rerun saturation_high check; compare distributions of final scores.

Risk: May create too much resistance and reduce top-end differentiation.

F-cycle-002-AI05: Weak groups are driven to 0 by shocks and remain there, producing low-saturation warnings.

Layer: mechanic. Class: saturation. Source: ai:deepseek-v4-pro

Evidence: [checks] saturation_low: e4-est 97%, e4-chl 100%, e4-tur 100%, e4-fin 100%; [batch_summary] ETH 5.96 to 1.74, ZWE 7.22 to 1.18.

Hypothesis: Shock points are applied proportionally to exposure but not to the group's current score, so low-score groups receive full damage.

Proposed change: Scale shock points by min(1, score/10) in event_scale before applying to each group.

Scope: structural

Expected effect: Low-score groups decline less sharply and are less likely to be pinned at 0.

Verification: Rerun saturation_low check; require no 100% low-saturation warnings and median new zeros below 1.

Risk: May understate vulnerability of already weak systems; test shock realism separately.

F-cycle-002-AI06: Domain tier jumps of 2-7 within five years are widespread, including under scripted policy, making the lexicographic tier rank a noisy objective.

Layer: methodology. Class: methodology_question. Source: ai:deepseek-v4-pro

Evidence: [checks] tier_jump warnings in at least 15 experiments: e4-irl 7 tiers, e4-kor 6, e4-est 6, e4-bra 6, e4-chl 5, e4-pol 5, e4-vnm 5, e4-mys 5, etc.; [batch_summary] tier_rank_p50 changes sharply for many.

Hypothesis: Pareto tiering with 0.25 tolerance is too fine for the model's noise and propagation lags.

Proposed change: Make the capability index and resilient score the primary objectives; report tier metrics as secondary with wider tolerance or only against published Index bands.

Scope: methodology

Expected effect: Objective comparisons become more stable and less driven by arbitrary tier thresholds.

Verification: Compare rank correlations between index and tier_rank across runs; check if tier_rank volatility exceeds index volatility.

Risk: Loses some policy nuance from Pareto dominance; may require stakeholder discussion.

F-cycle-002-AI07: No sensitivity analysis has been run for the model scalars.

Layer: harness. Class: data_gap. Source: ai:deepseek-v4-pro

Evidence: [sensitivity] not run.

Hypothesis: The loop cannot prioritise scalar fixes without local sensitivity information, so recommendations remain piecemeal.

Proposed change: Run +-20% one-at-a-time sensitivity on credit_eta, chunk_large, frontier_premium, decay rates, shock magnitudes and upside scale for one advanced (KOR or ISR) and one emerging (VNM or KEN) nation.

Scope: harness

Expected effect: Identifies which parameters dominate the observed failures and saturation.

Verification: Produce tornado plots of index, saturation and move_without_shock.

Risk: Computational cost; may require simplifying scenario runs.

What the batch taught (research notes, not defects)

Backcast: across 48 calibration nations, the correlation between observed and simulated change in the index from 1996 to 2026 is -0.10; the observed changes range from -5.6 to +6.5 points, the simulated from -6.0 to +0.9 [backcast_comparison].

Backcast: closest to history IRL (error 0.02), SAU (error 0.04), BRA (error 0.09), QAT (error 0.11); furthest CHN (observed +6.5, simulated -3.6), IND (observed +3.7, simulated -3.0), GEO (observed +1.1, simulated -3.9), TUR (observed +1.3, simulated -3.6) [backcast_comparison].

Backcast: the five fastest risers in history (CHN, IND, KOR, ARE, VNM) gained +3.5 index points on average; the model gives them -1.4. It has no mechanism for catch-up growth: income is calibrated to the cost of the starting position, so a nation that starts low stays poor in credits [backcast_comparison].

Across the batch, the graph's busiest edges are Economic Output & Productivity to Fiscal Capacity & Public Finance (-0.35 points a run); Fiscal Capacity & Public Finance to Social Infrastructure (-0.26 points a run); Fiscal Capacity & Public Finance to Force Generation & Mobilisation (-0.26 points a run) [top_edges].

This batch of 47 scripted 30-year runs shows a model that is stable at the top but unable to reproduce historical catch-up and prone to corner-saturating group scores. The median capability index at the horizon is close to or above the 2026 start for most advanced economies (e.g. JPN 17.08 from 16.49, DEU 17.58 from 16.98, NOR 16.67 from 16.23) but below start for many emerging and frontier nations (CHN 5.49 from 9.13, IND 5.92 from 8.92, KEN 4.18 from 7.31, ETH 1.74 from 5.96) [batch_summary]. The backcast against 1996-2026 history confirms the failure: the correlation between observed and simulated index change is -0.10, and the five fastest historical risers gained +3.5 points on average while the model gives them -1.4 [backcast_comparison, research_notes]. Domain-level biases are negative for all nine domains, from -0.43 to -1.88 [backcast_by_domain]. Saturation warnings appear in both directions: 100% of runs end with a newly saturated high group in SGP, IRL, TWN, ISR and FIN, and 97-100% end with a newly saturated low group in EST, CHL, TUR and FIN [checks]. One-year moves above 3 points occur without any shock in ISR and RUS in 100% of runs [checks]. Tier jumps of 2-7 within five years are widespread [checks]. These distributions point to excessive decay and/or a self-reinforcing credit function, insufficient cost convexity near the frontier, and event damage that is disproportionate for weak groups. The experiment design is informative; the main question is whether the credit calibration and event scaling are the right frame for catch-up growth, not whether individual nations are tuned.