Improvement loop
Every cycle runs the model, records everything that happened, writes up what it shows, diagnoses what should change and turns that into instructions for the next cycle. The loop is recursive, not autonomous: parameter changes are made one at a time and verified, structural changes wait for a person, and questions about the framework are flagged, never tuned.
1 cycle · 2,401 runs · parameters now 0.1.0 · graph 0.2.0
One cycle
Six steps. The harness owns the first two, the loop's analyst the next three, and a person the gate.
1Run
The harness runs the cycle batch: the null run and the baselines for five nations under four policies, with fixed seeds.
2Record
Every round is written to Parquet: trajectory, decisions with reasons, events, the propagation ledger, credits, tiers, objectives and counterfactual probes.
3Narrate
A report for every experiment, written from named queries over the record, distributions not single runs.
4Diagnose
The checks become findings, each with evidence, a hypothesis, a proposed change, a scope, an expected effect, a verification and a risk.
5Recommend
Each finding becomes a recommendation at its layer: a parameter, a mechanic, the methodology, the harness. An AI reviewer judges every layer and adds its own.
6Master and gate
One master recommendation chooses and orders a package to run next. A person reviews and approves it; the next cycle applies it and runs again.
Master recommendation
M-cycle-001, from cycle-001, written by deepseek-v4-pro from every recommendation of the cycle. It changes nothing until a person approves it: add "M-cycle-001: approved" under masters in loop/approvals.yaml, then run the next cycle.
Fix measurement and policy degeneracy before parameter reductions: backcast, pipeline-aware moves, objective demotion, shock scaling, and two parameter steps
This package first repairs measurement and policy degeneracy before parameter reductions. It adds historical backcast states and shock schedules, makes gap measurement delivery-aware, adds a two-round lookahead comparison policy, demotes tier rank to a reported outcome, applies score-sensitive shock scaling, then reduces chunk_large and raises frontier_premium. The next run repeats E3 with these changes, adds E4 backcast, and runs a multi-context sensitivity sweep across three nations, two policies, a stress scenario, and frontier/frontier_premium ranges. Hold-out nations are validation only; 2026 starting scores remain fixed.
- 1Harness and databackcast data
Reconstruct historical nation_states for KOR 1990, EST 1995, ARE 2005, SGP 1990, IRL 1995 plus hold-out CHL and POL, with provenance and confidence grades; add scripted historical shock schedules and an E4 backcast comparison batch.
No backcast comparison exists, so the model cannot be tested against observed historical capability changes. This fixes measurement before any tuning. From F-cycle-001-01, F-cycle-001-AI05.
- 2Harness and datapipeline/sim/policies
Add pipeline-aware gap and remaining-room measurement by counting levels already in delivery; add a greedy two-round lookahead comparison policy; add a diagnostic decomposing each non-shock one-year move into new move, prior-year delivery, partner delivery, and propagation.
Same actions are repeated and greedy marginal dominates because policies ignore delivery lags. Diagnosing move composition is also a prerequisite for the chunk_large change. From F-cycle-001-02, F-cycle-001-03, F-cycle-001-AI01, F-cycle-001-AI04.
- 3Methodologyobjectives
Rank and optimise policies with the capability index and score_sum; retain tier rank and Tier 1 count as reported outcomes only; add a tier transition check against published bands with an explicit tolerance.
Tier rank is coarse, ties policies, and shows implausible jumps even under hold. A continuous capability objective must be the primary ranking. From F-cycle-001-05, F-cycle-001-06, F-cycle-001-AI06.
- 4Mechanicmodel.event_scale
Apply score-sensitive shock scaling to each group's shock points: multiplier = min(1, score / 10), so damage is proportional for weak groups and does not drive them to zero.
Weak groups in low-rated nations are driven to zero and remain there. Scaling damage avoids a global reduction in shock magnitudes while protecting weak groups. From F-cycle-001-08, F-cycle-001-AI03.
- 5Parameterchunk_large
Reduce chunk_large from 0.05 to 0.03 after the decomposed move diagnostic confirms new chunk delivery causes more than 3 non-shock one-year moves; if the diagnostic attributes the moves to stacked delivery, retain 0.05. Sets
chunk_largeto 0.03.Parameter judgement says too_high. One-year moves above 3 points in the absence of shocks are implausible for most capability groups, but the diagnostic must confirm the cause before changing the parameter. From F-cycle-001-04.
- 6Parameterfrontier_premium
Raise frontier_premium from 0.60 to 0.90 as a first step to dampen saturation; run companion runs at 0.60 and 1.20 while holding frontier at 17. Sets
frontier_premiumto 0.9.Saturation warnings show high-rated active runs reliably reach 20. Parameter judgement suggests testing 0.60 to 1.20, beginning with 0.90. From F-cycle-001-07, F-cycle-001-AI02.
The run
Run the E3 batch with the updated model, pipeline-aware policies, added greedy two-round lookahead policy, shock scaling, and parameter changes for KOR, GBR, EST, BRA, KEN at 20-year and selected 10/30-year durations. Run E4 backcast on reconstructed historical states. Run a multi-context sensitivity sweep across KOR, EST, KEN under hold and greedy, plus a stress scenario, including frontier_premium [0.60, 0.90, 1.20] and frontier [14, 20]. Do not tune parameters to hold-out CHL or POL; use them only for validation. Do not change 2026 starting scores.
Accept if
- backcast_comparison returns at least one row for each reconstructed start year and nation (KOR 1990, EST 1995, ARE 2005, SGP 1990, IRL 1995, CHL, POL).
- greedy_dominates: the two-round lookahead policy beats greedy marginal on at least one primary objective for at least 3 of 5 nations.
- same_action: no policy repeats the same move in more than 50% of rounds for KOR and KEN mirror-peer and greedy-10y.
- move_without_shock: after diagnostic and chunk_large change, no group-year moves more than 3 points due to new chunks in the absence of shocks.
- saturation_high: e3-gbr-greedy-marginal and e3-kor-greedy-marginal share of runs ending with groups at 20 falls below 50%.
- saturation_low: e3-ken-hold and e3-ken-mirror-peer share of runs ending with a group at 0 falls below 5%.
- tier_transitions: hold policy tiers change by no more than 2 published-band widths within five years.
- no dead parameter is removed until the multi-context sensitivity sweep shows a max relative change above 0.5% across all contexts.
Deferred
- F-cycle-001-10 Frontier elicitation is deferred to the expert network. The package tests frontier 14 and 20 in sensitivity runs but does not set a single point until an elicited range is available.
Risks
- Frontier_premium increase may over-dampen high scores and cause stagnation if frontier is too low; the companion frontier sweep will reveal this.
- Score-sensitive shock scaling may reduce realism for shocks that should be able to wipe out weak groups; historical backcast validation will test this.
- Chunk_large reduction may be unnecessary if stacked delivery is the cause, not new large chunks; the decomposed move diagnostic must confirm before finalising the parameter.
- Backcast reconstructions carry confidence uncertainty and must not be used to tune parameters, especially for hold-out nations.
- The two-round lookahead policy may be computationally heavier than current policies; seed comparability must be maintained.
Every recommendation, by layer
All the recommendations of the latest cycle (cycle-001), from the rules and from the AI review, each tied to the finding behind it. The master recommendation above chooses among them.
No backcast can be scored: there are no reconstructed historical starting states.
Change. Reconstruct nation_states rows for KOR 1990, EST 1995, ARE 2005, SGP 1990 and IRL 1995 (calibration) and CHL and POL (hold-out) from the GINC Research, Patents, Trade and Infrastructure indices, with provenance = reconstructed and a confidence grade; add scripted historical shock schedules and an E4 batch.
Verify. Run E4; accept if the backcast check returns an error for all seven nations.
The finding behind it (F-cycle-001-01)
- Evidence
- checks.parquet backcast: nation_states holds year 2026 only; backcast_comparison returns no rows.
- Hypothesis
- The calibration loop needs starting states for the five calibration nations and the two hold-outs before 2026.
- Expected effect
- The backcast check reports an error by nation and domain.
- Verification
- Run E4; accept if the backcast check returns an error for all seven nations.
- Risk
- Reconstructed data is itself uncertain: every backcast report must carry the caveat. Never tune to CHL or POL.
Greedy marginal is the best policy on every objective for 3 of the five E3 nations.
Change. Add a sequencing-aware comparison policy to the E3 batch (greedy with a two-round lookahead on the same value model) so that the bar a solver must beat is tested against a policy that can exploit lags.
Verify. Run E3 with the lookahead policy; accept if it differs from greedy on at least one objective for at least two nations.
The finding behind it (F-cycle-001-02)
- Evidence
- checks.parquet greedy_dominates: BRA: greedy is best on 4 of 4 objectives; EST: greedy is best on 4 of 4 objectives; KEN: greedy is best on 4 of 4 objectives [baseline_comparison].
- Hypothesis
- The other baselines are deliberately naive, so greedy's lead says little about whether the model rewards sequencing. Its value model is the only one that sees the graph's long-run spillovers.
- Expected effect
- A comparison that can tell whether order matters: lookahead beats greedy where lags matter.
- Verification
- Run E3 with the lookahead policy; accept if it differs from greedy on at least one objective for at least two nations.
- Risk
- Adds run time; a lookahead policy multiplies policy cost by the branching factor.
The same move is repeated in most rounds in 3 experiments.
Change. Count levels still in delivery when mirror_peer measures a gap and when greedy measures the room left below 20 (pipeline/sim/policies).
Verify. Rerun e3-*-mirror-peer; accept if same_action passes for all five nations.
The finding behind it (F-cycle-001-03)
- Evidence
- checks.parquet same_action: e3-kor-mirror-peer 95% of rounds; e3-ken-mirror-peer 65% of rounds; e3-ken-greedy-marginal-10y 80% of rounds [decision_sequence].
- Hypothesis
- The mirror-peer policy measures its gaps against current scores and ignores levels still in delivery, so it keeps funding a gap it has already paid to close; near its horizon greedy can only value fast (bought) moves, so it repeats them.
- Expected effect
- same_action passes for the mirror-peer experiments.
- Verification
- Rerun e3-*-mirror-peer; accept if same_action passes for all five nations.
- Risk
- Changes the baselines' behaviour, so cycle-on-cycle comparison of those experiments breaks once.
Groups move more than 3 points in a year without any shock.
Change. Lower chunk_large from 0.05 to 0.03 in data/sim/seed/params_v0_1.yaml (model).
Verify. Rerun e3-gbr-greedy-marginal, e3-gbr-mirror-peer, e3-gbr-random; accept if the share of failing experiments falls by half.
The finding behind it (F-cycle-001-04)
- Evidence
- 7 experiments: e3-kor-random move_without_shock 3.84; e3-gbr-mirror-peer move_without_shock 3.33; e3-kor-greedy-marginal move_without_shock 3.20; e3-gbr-greedy-marginal move_without_shock 3.20; e3-gbr-random move_without_shock 3.18 (checks.parquet) [decision_sequence]. Of the 135 group-years that moved more than 3 points without a shock, investment arriving averaged 3.11 points and propagation -0.000: the moves are investment, not the graph.
- Hypothesis
- A move is a share of income, and income scales with the cost of holding the whole position, so the largest chunk buys several levels of a weak group in one year.
- Expected effect
- move_without_shock at or below 3 points in the E3 experiments that fail it.
- Verification
- Rerun e3-gbr-greedy-marginal, e3-gbr-mirror-peer, e3-gbr-random; accept if the share of failing experiments falls by half.
- Risk
- Slower growth for every nation and policy; less of the headroom is spent, so carried credits rise.
Domains gain more than two tiers within five years, including under the hold policy, which makes no moves.
Change. Decide how the model should read tier movement: against the Index's published tier bands, with a wider tolerance, or not at all.
Verify. Rerun the cycle batch with the chosen measure; accept if tier_jump passes for the hold experiments.
The finding behind it (F-cycle-001-05)
- Evidence
- 22 experiments: e3-bra-greedy-marginal 6 tiers; e3-kor-greedy-marginal-30y 6 tiers; e3-kor-greedy-marginal-10y 6 tiers; e3-kor-greedy-marginal 6 tiers; 3 of them hold experiments (checks.parquet) [tier_transitions].
- Hypothesis
- Below Tier 1 the Pareto layers of a domain are close together (a tolerance of a quarter of a point across 197 nations), so a small move crosses several layers. The heuristic's two tiers may suit coarse tiers, not Pareto layers.
- Expected effect
- A tier measure that moves only when capability moves materially.
- Verification
- Rerun the cycle batch with the chosen measure; accept if tier_jump passes for the hold experiments.
- Risk
- Changing the tiering here would make results incomparable with the Index's own tiers.
The tier rank picks the same best policy as the capability index for 5 of 5 nations, and ties policies for 3.
Change. Keep tiers as a reported outcome; use the capability index (the mean of domains of groups) and score_sum as the objectives policies optimise and are ranked by; review whether the tier rank should be dropped from the objective set.
Verify. Rerun E3; compare policy rankings by index and by tier rank.
The finding behind it (F-cycle-001-06)
- Evidence
- batch_summary: median tier rank and median index by nation and policy, 20-year E3 experiments [objective_distributions]. Tier rank has whole-number values over 197 nations and frozen rivals, so it is coarse at the top and saturates once a nation leads.
- Hypothesis
- As an objective, the lexicographic tier rank mostly repeats the index where they agree and adds noise where they tie; its value is in describing position (which domains lead), not in choosing between policies.
- Expected effect
- Policy rankings stop depending on ties in the tier rank.
- Verification
- Rerun E3; compare policy rankings by index and by tier rank.
- Risk
- Dropping tier rank from the objectives loses the brief's lexicographic reward, which the search layer was designed around.
Groups are driven to 20 and held there in many runs, across nations and policies.
Change. Raise frontier_premium from 0.6 to 0.9 (model).
Verify. Rerun the E3 greedy experiments; accept if saturation_high passes in at least half of those that failed.
The finding behind it (F-cycle-001-07)
- Evidence
- 8 experiments above 25 per cent of runs: e3-gbr-greedy-marginal 96%; e3-kor-greedy-marginal 95%; e3-kor-greedy-marginal-10y 92%; e3-kor-mirror-peer 76%; e3-gbr-random 65%. Groups most often driven to 20: Technical & Vocational Skills (321), Advanced Education (223), Foundational Education (215), Digital & AI Competence (209), Lifelong Learning & Reskilling (174) [trajectory_by_domain].
- Hypothesis
- Costs above the frontier (17) do not steepen enough to stop a well-funded nation buying the last levels.
- Expected effect
- The share of runs ending with a group newly at 20 falls below 25 per cent in the greedy E3 experiments.
- Verification
- Rerun the E3 greedy experiments; accept if saturation_high passes in at least half of those that failed.
- Risk
- Frontier nations gain less; the frontier parameter is already the most sensitive.
Groups of low-rated nations are driven to 0 by shocks and stay there.
Change. Scale a shock's points on each group by min(1, score / 10) in model.event_scale, so damage is proportional for weak groups.
Verify. Rerun e3-ken-*; accept if saturation_low passes and the shock tests of the scenario library are unchanged for nations above 10.
The finding behind it (F-cycle-001-08)
- Evidence
- e3-ken-hold: 31% of runs end with a group at 0 that did not start there (median 0 groups); e3-ken-mirror-peer: 26% of runs end with a group at 0 that did not start there (median 0 groups) (checks.parquet) [shock_damage_by_group].
- Hypothesis
- Shock damage is in points regardless of the group's score, so a group rated near 1 loses everything to one shock; nothing rebuilds it unless the policy chooses it.
- Expected effect
- saturation_low passes for KEN.
- Verification
- Rerun e3-ken-*; accept if saturation_low passes and the shock tests of the scenario library are unchanged for nations above 10.
- Risk
- Departs from the scenario load test, which is in points; needs approval.
23 of the model's scalar parameters move no objective by more than 0.5 per cent at +-20 per cent.
Change. Run the sensitivity sweep across three nations (KOR, EST, KEN), two policies (hold, greedy) and a stress scenario before removing any parameter; remove only those still inert.
Verify. The next scheduled sensitivity batch.
The finding behind it (F-cycle-001-09)
- Evidence
- checks.parquet dead_parameter: maintenance_share, upstream_lambda, depth_gain, depth_erosion, depth0_intercept, depth0_slope, partner_vulnerability, propagation_floor, carry_decay, headroom_base, headroom_slope, chunk_small, chunk_medium, chunk_large, shock_magnitude_low, shock_magnitude_high, upside_magnitude_low, upside_magnitude_high, upside_probability_scale, absorption_max, credit_effect_high, credit_effect_medium, credit_effect_low (sensitivity.parquet; one nation, greedy, 12 seeds).
- Hypothesis
- The sweep runs one nation under one policy, which exercises few mechanics: shock magnitudes matter little to a rich nation, chunk sizes change how lumpy spending is but not how much, and depth acts only through vulnerability. Most are untested rather than dead.
- Expected effect
- A list of parameters inert across the wider sweep, each then removed or justified.
- Verification
- The next scheduled sensitivity batch.
- Risk
- Removing a parameter is a model change and bumps the minor version.
frontier dominates the model's sensitivity.
Change. Put frontier to the expert network for an elicited range, and report results at both ends.
Verify. Rerun E3 at both ends of the elicited range.
The finding behind it (F-cycle-001-10)
- Evidence
- sensitivity.parquet: +-20 per cent moves the score sum by -213.4 / +42.3 (max 100.0 per cent of an objective).
- Hypothesis
- A structural judgement with no evidence behind it carries the results.
- Expected effect
- A range with provenance 'expert' instead of a structural point value.
- Verification
- Rerun E3 at both ends of the elicited range.
- Risk
- None to the model; delays the next change to this parameter.
One-year moves above 3 points occur in groups with no shock landing, but the check does not distinguish new moves from delivery of earlier multi-year build/buy/partner decisions.
Change. Add a diagnostic that decomposes each non-shock one-year move into new move, prior-year delivery, partner delivery and propagation before changing chunk sizes.
Verify. Re-run the warning after decomposition; if new moves dominate, then reduce chunk_large.
The finding behind it (F-cycle-001-AI01)
- Evidence
- move_without_shock warnings for KOR random 3.84, GBR mirror 3.33, KOR greedy 3.20 [checks].
- Hypothesis
- Stacked delivery from prior years, not only current chunk_large, contributes to the largest one-year changes.
- Expected effect
- Would identify the true source of large moves and avoid over-tuning chunk_large.
- Verification
- Re-run the warning after decomposition; if new moves dominate, then reduce chunk_large.
- Risk
- Delays a parameter fix, but prevents an ineffective change.
Many high-income active runs end with groups at 20 that did not start there.
Change. Raise frontier_premium in steps, beginning with 0.9, while holding frontier fixed and monitoring distributional effects.
Verify. Re-run the batch; expect saturation_high share to fall substantially, but not eliminate all gains.
The finding behind it (F-cycle-001-AI02)
- Evidence
- e3-gbr-greedy-marginal 96 per cent of runs, median 8 groups; e3-kor-greedy-marginal 95 per cent, median 8 groups [checks].
- Hypothesis
- The current frontier_premium of 0.60 does not make the top of the scale expensive enough under high income and greedy spending.
- Expected effect
- Would reduce the share of runs with saturated groups and increase differentiation among high performers.
- Verification
- Re-run the batch; expect saturation_high share to fall substantially, but not eliminate all gains.
- Risk
- May over-penalise high scorers and reduce realistic high-end progress.
Weak groups in Kenya can be driven to zero and remain there, especially under hold and mirror-peer.
Change. Apply a score-sensitive shock scaling such as min(1, score/10) or a floor that leaves weak groups at a small positive level, rather than reducing global shock magnitudes.
Verify. Re-run Kenya experiments; saturation_low should fall without eliminating shock losses altogether.
The finding behind it (F-cycle-001-AI03)
- Evidence
- e3-ken-hold saturation_low 31 per cent of runs; e3-ken-mirror-peer 26 per cent [checks].
- Hypothesis
- Shock damage is proportional to exposure but not to the group's starting level, so low-score groups can be eliminated by a single large shock.
- Expected effect
- Would reduce pathological zero saturation while preserving relative vulnerability across groups.
- Verification
- Re-run Kenya experiments; saturation_low should fall without eliminating shock losses altogether.
- Risk
- May understate the collapse risk of very weak groups.
Greedy marginal is the best policy on every objective for three of five nations, and the same move is repeated in most rounds in several experiments.
Change. Add a sequencing-aware policy such as greedy with a two-round lookahead on the same value model, and count levels already in delivery when measuring gaps or remaining room.
Verify. Compare policies across all five nations; if the lookahead beats greedy on index or resilient score, replace the default.
The finding behind it (F-cycle-001-AI04)
- Evidence
- Research notes: greedy best for 3 of 5; same_action warnings for e3-kor-mirror-peer median 95 per cent, e3-ken-mirror-peer 65 per cent, e3-ken-greedy-marginal-10y 80 per cent [checks].
- Hypothesis
- The policy set lacks a comparator that accounts for multi-year delivery lags, so greedy marginal wins too easily.
- Expected effect
- Would provide a stronger benchmark and reduce repeated identical moves.
- Verification
- Compare policies across all five nations; if the lookahead beats greedy on index or resilient score, replace the default.
- Risk
- More complex policy may be harder to interpret and verify.
No backcast comparison exists, so the model cannot be tested against observed historical capability changes.
Change. Construct reconstructed nation_states rows for KOR 1990, EST 1995, ARE 2005, SGP 1990 and IRL 1995, plus CHL and POL as hold-outs, with provenance and confidence grades, and add scripted historical shock schedules.
Verify. Run an E4 batch and compare simulated backcasts with observed historical scores.
The finding behind it (F-cycle-001-AI05)
- Evidence
- Both experiment reports end with 'No backcast comparison' [TWO_EXPERIMENT_REPORTS].
- Hypothesis
- Reconstructed historical states and event schedules would allow a meaningful out-of-sample test of the model's dynamics.
- Expected effect
- Would enable backcast scoring and reveal whether the model's decay, shock and propagation dynamics reproduce history.
- Verification
- Run an E4 batch and compare simulated backcasts with observed historical scores.
- Risk
- Reconstructed data quality may be low and could mislead calibration if treated as observed.
The tier rank chooses the same best policy as the capability index for all five nations and ties policies for three, while tier_jump warnings show tier movement is noisy.
Change. Use capability index and score_sum as the objectives policies optimise and are ranked by; retain tier rank and Tier 1 count as reported outcomes, not primary objectives.
Verify. Confirm policy ranking remains stable and that dropping tier rank from objectives does not change the recommended best policy.
The finding behind it (F-cycle-001-AI06)
- Evidence
- F-cycle-001-06: tier rank same best policy for 5 of 5 and ties 3; tier_jump warnings all experiments, including hold [checks].
- Hypothesis
- The lexicographic Pareto tier rank adds little decision value over the capability index and score_sum, but may still be a useful reported diagnostic.
- Expected effect
- Would reduce noise in policy selection while preserving the non-linear top-end information for descriptive analysis.
- Verification
- Confirm policy ranking remains stable and that dropping tier rank from objectives does not change the recommended best policy.
- Risk
- Could lose a signal that matters for top-ranked nations if the capability index is too coarse.
Many scalar parameters show zero sensitivity in the single Korea greedy sweep, but this may not generalise across nations, policies or scenarios.
Change. Run the sensitivity sweep across KOR, EST and KEN, under hold and greedy, plus a stress scenario, before removing any parameter.
Verify. Compare sensitivity ranks across conditions; remove only parameters inert in all.
The finding behind it (F-cycle-001-AI07)
- Evidence
- Sensitivity table: 23 parameters with max_rel_change 0.0000 or below 0.001 [sensitivity]; F-cycle-001-09 already flags this.
- Hypothesis
- Parameters inert under one baseline may become active under low-income or stress conditions.
- Expected effect
- Would identify genuinely dead parameters and avoid deleting context-dependent ones.
- Verification
- Compare sensitivity ranks across conditions; remove only parameters inert in all.
- Risk
- More computational cost, but affordable for 23 candidate parameters.
AI review
What deepseek-v4-pro made of cycle-001, given the batch summary, the checks, the rule findings and two reports. Its proposals appear above marked as AI proposals and always wait for approval.
Batch cycle-001 shows a model in which the hold policy decays the capability index for all five e3 nations over 20 years, with median changes from -0.31 for Great Britain (16.454 to 16.142) to -1.10 for Estonia (11.487 to 10.589) [batch_summary]. Active policies reverse some of this: greedy_marginal is best in median index for all five, but the gain over holding is uneven, from +0.98 points for Kenya to +2.38 for Brazil [research_notes]. The e0-null run for Korea falls to 6.214 after 30 years, consistent with no income by design. Shocks are a large drag: removing the shock library raises median index by 0.205 to 1.509 points across e3 experiments, with Kenya hold benefiting by 1.473 and Korea hold by 1.008 [batch_summary, probe_attribution]. Saturation is common in high-income active runs: 95-96 per cent of Korea and Great Britain greedy runs end with at least one group at 20 that did not start there, median 8 groups [checks]. Low-rated Kenya can be driven to zero groups: 31 per cent of hold runs and 26 per cent of mirror-peer runs end with a group at 0 that did not start there [checks]. The graph's busiest edges are Semiconductors & Advanced Computing to Artificial Intelligence and to Economic Leverage & Dependence, implying technology investments spread widely [research_notes]. The sensitivity sweep shows the frontier parameter dominates, with a relative change of 1.000 at +-20 per cent, while many scalars show zero movement in these probes [sensitivity]. Policy ranking is stable for the best policy but horizon changes choices: 10-year greedy stops investing in slow builds near its horizon [research_notes]. Tier measures are noisy: tier_jump warns in all experiments, including hold, with up to six tiers in five years [checks].
Greedy's advantage is largest for nations that already have more capability
The median index gain from greedy over holding is +0.98 for Kenya, +2.21 for Great Britain, +2.27 for Korea, +2.36 for Estonia and +2.38 for Brazil, suggesting the policy's returns are constrained by low income in weak nations.
Research notes: what greedy is worth over holding [baseline_comparison].
The scenario library's shocks are priced very differently across nations
Removing the shock library raises median index by 0.205 to 1.509 points across e3 experiments; the largest benefits appear in Kenya hold (1.473) and Estonia hold (1.402), showing weak nations bear more shock cost.
Research notes: probe_attribution [probe_attribution]; batch summary no_shocks_d_index column [batch_summary].
Technology edges propagate strongly through the graph
Semiconductors & Advanced Computing to Artificial Intelligence is the busiest edge per run in the batch, carrying +0.51 points overall and +1.38 points in Kenya greedy, while Semiconductors to Economic Leverage & Dependence also carries significant change.
Research notes top_edges [top_edges]; Kenya report top edges [TWO_EXPERIMENT_REPORTS].
The largest moves by credits are not necessarily the most valuable
In Korea greedy, removing any single one of the five largest moves changes the index by only -0.020 to -0.012, and large moves come late when income is highest; Kenya shows similarly small single-move attribution.
Korea and Kenya experiment reports, Why sections [TWO_EXPERIMENT_REPORTS].
Horizon changes the strategy
The 10-year greedy stops investing in slow builds near its horizon, so the median index and strategy differ from the 20-year and 30-year runs.
Research notes: greedy by horizon [objective_distributions].
Tier movement is sensitive even without policy moves
Hold experiments show tier_jump warnings, with Korea hold recording up to 4 tiers gained within five years in 25 runs and Kenya hold 3 tiers in 1 run, so events and propagation alone can produce large tier changes.
Checks tier_jump warnings [checks].
Critique of a rule finding
F-cycle-001-04: I disagree with lowering chunk_large from 0.05 to 0.03 before decomposing the one-year moves; the move_without_shock check may include delivery from earlier multi-year build/buy/partner decisions, so chunk_large may not be the sole cause.
Critique of a rule finding
F-cycle-001-06: I disagree with dropping tier rank from the objective set before checking whether the lexicographic tier rank carries information in tail runs or hold-out nations; the current batch has only five nations and tier rank may still add value at the top.
Critique of a rule finding
F-cycle-001-07: I disagree with setting frontier_premium to 0.9 solely on saturation warnings; the sensitivity table shows frontier_premium changes the objective only 0.0145 at +-20 per cent, so saturation may be driven more by the frontier threshold or chunk sizes.
Critique of a rule finding
F-cycle-001-08: I disagree with scaling every shock by min(1, score/10) as a first fix; it changes the meaning of exposure for weak groups and may mask where resilience is actually missing. A floor or buffer-based absorption is preferable.
Parameter judgements
The AI reviewer's verdict on each parameter or family of parameters: too high, too low or about right, with what it would set instead. The 2026 starting scores are fixed inputs and are never judged.
| Parameter | Now | Verdict | Suggested | Why |
|---|---|---|---|---|
| frontier | 17.0 | Cannot tell | Elicit a plausible range, e.g. 14-20, and test both ends | The frontier parameter dominates the sensitivity sweep, so a single value cannot be judged from one batch.Sensitivity table shows max_rel_change = 1.000 for frontier at +-20 per cent [sensitivity]. |
| convexity and frontier_premium | convexity 2.0-2.5 by domain; frontier_premium 0.60 | Cannot tell | Test frontier_premium from 0.6 to 1.2 while holding frontier fixed | Saturation warnings suggest the combined cost of high scores may be too low, but the two parameters confound and frontier_premium alone has small sensitivity.Sensitivity table gives frontier_premium max_rel_change = 0.0145; saturation_high warnings affect 64-96 per cent of greedy runs in GBR, KOR, EST and BRA [sensitivity, checks]. |
| chunk_large | 0.05 | Too high | 0.03 if a diagnostic confirms that new large chunks, not stacked delivery from earlier years, cause the moves | It permits one-year moves above 3 points in the absence of shocks, which is implausible for most capability groups.move_without_shock warnings: KOR random 3.84 points, GBR mirror-peer 3.33 points, KOR greedy 3.20 points, GBR random 3.18 points [checks]. |
| shock magnitudes (low/high) | 0.60 / 1.20 | Too high | Do not lower globally; apply score-sensitive scaling or a floor to weak groups | The same global magnitudes produce disproportionate losses in low-rated nations and can drive groups to zero.Removing shocks raises Kenya hold index by 1.473, compared with 1.008 for Korea hold; saturation_low warns 31 per cent of Kenya hold runs end with a group at 0 [batch_summary, checks]. |
| decay rates by domain | technology 0.05 built / 0.09 bought; institutions 0.01 / 0.03; other domains between | About right | Leave unchanged pending backcast and stress tests | Hold-policy declines are moderate and ordered by domain class, with no sign of runaway decay.Hold medians decline from -0.31 to -1.10 index points over 20 years across five nations [batch_summary]; domain class decay values are in the domain table [domain_class_values]. |
| credit function coefficients | credit_eta = 1.00, headroom_base = 0.10, headroom_slope = 0.30 | Cannot tell | Retain until the multi-nation, multi-policy sensitivity sweep distinguishes dead versus context-dependent parameters | Only credit_eta shows a modest sensitivity, while headroom parameters show zero in a single KOR greedy sweep; this may not mean they are inert for other nations.Sensitivity table: credit_eta max_rel_change = 0.0087; headroom_base and headroom_slope = 0.0000 [sensitivity]. |
| absorption_max | 0.75 | About right | Leave unchanged | Its sensitivity is modest and the mechanism operates through buffer means, which is plausible.Sensitivity table: absorption_max max_rel_change = 0.0045 [sensitivity]. |
Master recommendations and what came of them
Every master recommendation, whether it was approved, and what the next cycle changed and found.
- M-cycle-001Proposed2026-10-20
Fix measurement and policy degeneracy before parameter reductions: backcast, pipeline-aware moves, objective demotion, shock scaling, and two parameter steps
Awaiting review.
Evolution of the engine
The same experiments with the same seeds, cycle by cycle, so a difference is the model's and not luck's. Select an experiment to show or hide it.
Checks by cycle
Checks, cycle-001
Invariants, plausibility, saturation, degeneracy, dead parameters, calibration, regression and performance. Every warning or failure is a candidate finding.
320 of 320 checks
Check | Category | Experiment | Status | Detail |
|---|---|---|---|---|
| tier_jump | plausibility | e3-gbr-hold | warn | largest gain in one domain within five years: 4 tiers (8 runs above 2) |
| move_without_shock | plausibility | e3-gbr-random | warn | largest one-year move in a group with no shock landing: 3.18 points; 3% of runs have one above 3 |
| tier_jump | plausibility | e3-gbr-random | warn | largest gain in one domain within five years: 4 tiers (4 runs above 2) |
| saturation_high | saturation | e3-gbr-random | warn | 65% of runs end with a group at 20 that did not start there (median 1 groups) |
| move_without_shock | plausibility | e3-gbr-greedy-marginal | warn | largest one-year move in a group with no shock landing: 3.20 points; 67% of runs have one above 3 |
| tier_jump | plausibility | e3-gbr-greedy-marginal | warn | largest gain in one domain within five years: 3 tiers (12 runs above 2) |
| saturation_high | saturation | e3-gbr-greedy-marginal | warn | 96% of runs end with a group at 20 that did not start there (median 8 groups) |
| move_without_shock | plausibility | e3-gbr-mirror-peer | warn | largest one-year move in a group with no shock landing: 3.33 points; 55% of runs have one above 3 |
| tier_jump | plausibility | e3-gbr-mirror-peer | warn | largest gain in one domain within five years: 4 tiers (8 runs above 2) |
| tier_jump | plausibility | e3-kor-hold | warn | largest gain in one domain within five years: 4 tiers (25 runs above 2) |
| move_without_shock | plausibility | e3-kor-random | warn | largest one-year move in a group with no shock landing: 3.84 points; 3% of runs have one above 3 |
| tier_jump | plausibility | e3-kor-random | warn | largest gain in one domain within five years: 5 tiers (82 runs above 2) |
| saturation_high | saturation | e3-kor-random | warn | 57% of runs end with a group at 20 that did not start there (median 1 groups) |
| move_without_shock | plausibility | e3-kor-greedy-marginal | warn | largest one-year move in a group with no shock landing: 3.20 points; 2% of runs have one above 3 |
| tier_jump | plausibility | e3-kor-greedy-marginal | warn | largest gain in one domain within five years: 6 tiers (100 runs above 2) |
| saturation_high | saturation | e3-kor-greedy-marginal | warn | 95% of runs end with a group at 20 that did not start there (median 8 groups) |
| tier_jump | plausibility | e3-kor-mirror-peer | warn | largest gain in one domain within five years: 4 tiers (36 runs above 2) |
| saturation_high | saturation | e3-kor-mirror-peer | warn | 76% of runs end with a group at 20 that did not start there (median 1 groups) |
| same_action | degeneracy | e3-kor-mirror-peer | warn | the most repeated move (group and mode) appears in a median 95% of rounds |
| tier_jump | plausibility | e3-est-random | warn | largest gain in one domain within five years: 3 tiers (4 runs above 2) |
Showing 1–20 of 320 checks · page 1 of 16
Research notes
What the latest batch taught that is not a defect: insights about the model and what it implies, kept apart from the fix list.
- Policy ranking by median index at 20 years is not the same for every nation: BRA: greedy_marginal > mirror_peer > random > hold; EST: greedy_marginal > mirror_peer > random > hold; GBR: greedy_marginal > random > mirror_peer > hold; KEN: greedy_marginal > mirror_peer > random > hold; KOR: greedy_marginal > mirror_peer > random > hold [baseline_comparison].
- What greedy is worth over holding, in median index points at 20 years: KEN +0.98, GBR +2.21, KOR +2.27, EST +2.36, BRA +2.38. Low-rated nations gain least, because income is calibrated to the cost of their position [objective_distributions].
- Removing the shock library raises the median index by 0.205 to 1.509 points at 20 years across E3 experiments: the cost of the scenario library's risks, as the model prices them [probe_attribution].
- Greedy by horizon (median index): KEN: 10y 7.25, 20y 7.38, 30y 7.52; KOR: 10y 17.41, 20y 17.98, 30y 18.08. The 10-year greedy stops investing in slow builds near its horizon, so horizons recommend different strategies [objective_distributions].
- Across the batch, the graph's busiest edges are Semiconductors & Advanced Computing to Artificial Intelligence (+0.51 points a run); Semiconductors & Advanced Computing to Economic Leverage & Dependence (+0.19 points a run); Economic Output & Productivity to Fiscal Capacity & Public Finance (+0.18 points a run) [top_edges].
Flagged for the framework review
Groups and measures the model cannot represent sensibly. They are findings about the National Capability Framework, not parameters to tune.
- Monetary & Currency Capability cannot be given a defensible cost, decay or vulnerability. cost_params and decay_params, defensible = false: Monetary credibility and the international use of a currency follow from history and the size of the economy; neither a cost per level nor a decay rate is defensible. Left at the class values.
- Economic Leverage & Dependence cannot be given a defensible cost, decay or vulnerability. cost_params and decay_params, defensible = false: Leverage is relational; a nation's level depends on others' dependence on it, which phase one freezes. No cost or vulnerability is defensible for a single-nation model.
- Nuclear Deterrence cannot be given a defensible cost, decay or vulnerability. cost_params and decay_params, defensible = false: Deterrence is close to binary and governed by treaty; a level scale, a yearly cost per level and a decay rate do not describe it. Kept expensive and unbuyable so the solver cannot treat it as a cheap source of points.
- Early Childhood Development cannot be given a defensible cost, decay or vulnerability. cost_params and decay_params, defensible = false: Early childhood investment pays out in adult skills ten to fifteen years later; a six-year delivery and a decay rate on the group's own score do not represent that lag. The graph's edges carry it instead.
- Diaspora & Global Communities cannot be given a defensible cost, decay or vulnerability. cost_params and decay_params, defensible = false: A diaspora is a demographic stock formed over generations; a budget does not build it, so no cost per level is defensible. Kept costly so that it is not a cheap target.
- Several capability groups lack a defensible cost, decay or vulnerability as investment targets. F-cycle-001-11 to -15 list Monetary & Currency Capability, Economic Leverage & Dependence, Nuclear Deterrence, Early Childhood Development, Diaspora & Global Communities [RECOMMENDATIONS_ALREADY_RAISED].
Every finding
All findings of all cycles, from the rules and from AI review. Open one for its evidence, hypothesis and risk.
23 of 23 findings
Finding | Class | Layer | Source | Cycle |
|---|---|---|---|---|
No backcast can be scored: there are no reconstructed historical starting states.F-cycle-001-01Proposed. Reconstruct nation_states rows for KOR 1990, EST 1995, ARE 2005, SGP 1990 and IRL 1995 (calibration) and CHL and POL (hold-out) from the GINC Research, Patents, Trade and Infrastructure indices, with provenance = reconstructed and a confidence grade; add scripted historical shock schedules and an E4 batch.
| data gap | Harness and data | Rules | 1 |
Greedy marginal is the best policy on every objective for 3 of the five E3 nations.F-cycle-001-02Proposed. Add a sequencing-aware comparison policy to the E3 batch (greedy with a two-round lookahead on the same value model) so that the bar a solver must beat is tested against a policy that can exploit lags.
| degenerate strategy | Harness and data | Rules | 1 |
The same move is repeated in most rounds in 3 experiments.F-cycle-001-03Proposed. Count levels still in delivery when mirror_peer measures a gap and when greedy measures the room left below 20 (pipeline/sim/policies).
| degenerate strategy | Harness and data | Rules | 1 |
Groups move more than 3 points in a year without any shock.F-cycle-001-04Proposed. Lower chunk_large from 0.05 to 0.03 in data/sim/seed/params_v0_1.yaml (model).
| implausible dynamics | Parameter | Rules | 1 |
Domains gain more than two tiers within five years, including under the hold policy, which makes no moves.F-cycle-001-05Proposed. Decide how the model should read tier movement: against the Index's published tier bands, with a wider tolerance, or not at all.
| methodology question | Methodology | Rules | 1 |
The tier rank picks the same best policy as the capability index for 5 of 5 nations, and ties policies for 3.F-cycle-001-06Proposed. Keep tiers as a reported outcome; use the capability index (the mean of domains of groups) and score_sum as the objectives policies optimise and are ranked by; review whether the tier rank should be dropped from the objective set.
| methodology question | Methodology | Rules | 1 |
Groups are driven to 20 and held there in many runs, across nations and policies.F-cycle-001-07Proposed. Raise frontier_premium from 0.6 to 0.9 (model).
| saturation | Parameter | Rules | 1 |
Groups of low-rated nations are driven to 0 by shocks and stay there.F-cycle-001-08Proposed. Scale a shock's points on each group by min(1, score / 10) in model.event_scale, so damage is proportional for weak groups.
| saturation | Mechanic | Rules | 1 |
23 of the model's scalar parameters move no objective by more than 0.5 per cent at +-20 per cent.F-cycle-001-09Proposed. Run the sensitivity sweep across three nations (KOR, EST, KEN), two policies (hold, greedy) and a stress scenario before removing any parameter; remove only those still inert.
| dead parameter | Harness and data | Rules | 1 |
frontier dominates the model's sensitivity.F-cycle-001-10Proposed. Put frontier to the expert network for an elicited range, and report results at both ends.
| sensitivity | Parameter | Rules | 1 |
Monetary & Currency Capability cannot be given a defensible cost, decay or vulnerability.F-cycle-001-11Proposed. None here: a question for the framework review.
| framework question | Framework | Rules | 1 |
Economic Leverage & Dependence cannot be given a defensible cost, decay or vulnerability.F-cycle-001-12Proposed. None here: a question for the framework review.
| framework question | Framework | Rules | 1 |
Nuclear Deterrence cannot be given a defensible cost, decay or vulnerability.F-cycle-001-13Proposed. None here: a question for the framework review.
| framework question | Framework | Rules | 1 |
Early Childhood Development cannot be given a defensible cost, decay or vulnerability.F-cycle-001-14Proposed. None here: a question for the framework review.
| framework question | Framework | Rules | 1 |
Diaspora & Global Communities cannot be given a defensible cost, decay or vulnerability.F-cycle-001-15Proposed. None here: a question for the framework review.
| framework question | Framework | Rules | 1 |
Showing 1–15 of 23 findings · page 1 of 2
Ledger
One row per cycle. The full ledger, the history as Parquet and a DuckDB database over every run are in the repository under loop/.
| Cycle | Date | Parameters | Graph | Runs | Checks | Findings | Implemented | AI review |
|---|---|---|---|---|---|---|---|---|
| 1 | 2026-10-20 | 0.1.0 → 0.1.0 | 0.2.0 | 2,401 | 251 69 0 | 23 | – | deepseek-v4-pro |
Stop condition: not evaluable: no backcast. The loop pauses for a human review when the hold-out backcast error stops improving for three cycles, or improves calibration while worsening the hold-out.