Method · exploratory · 2026-10-20

Evolution strategy (CMA-ES)

The covariance matrix adaptation evolution strategy treats a whole plan as one vector of numbers, samples a population of plans, scores each by playing it through the engine, keeps the best and resamples around them with a covariance that adapts to the shape of the good region. It needs no gradients, no tree and no discretisation of actions, and it does not mind discontinuities in the reward such as tier thresholds. Its limitation is that it finds open-loop plans; the receding-horizon mode re-optimises every round so that it can react.

2 batches · 5 experiments · 19 simulations

A second optimiser, to tell the model from the search

CMA-ES and the tree search look for pathways in completely different ways. Where they agree, the result is more likely a property of the model; where they disagree, the disagreement is a finding about the model or about one search's design. Neither result is a forecast or a recommendation, and every CMA-ES experiment also writes the standard tables so it can be compared with any other.

How the optimisation works here

Four steps a generation. The budget is generations, never wall-clock, and the optimiser is seeded from the simulation's own stream, so a simulation is reproducible from its seed.

1Sample
Draw a population of plans around the current mean, each a vector of numbers, with a covariance that says which directions to explore together.
2Score
Decode every plan into actions and play it through the engine to the horizon under the same set of shock seeds (common random numbers), so the plans are compared on the same futures. The fitness is the objective at the horizon, averaged over the seeds.
3Select
Keep the better half, weighted by rank, and move the mean towards them. The best plan so far is also scored on a fixed hold-out set of seeds, so the curve that is reported only rises.
4Adapt
Update the covariance to the shape of the region the good plans came from, and the step size to how far they moved. Stop at the generation budget, or when the hold-out fitness has not improved for thirty generations.

What a plan is. A genome decodes to a round's actions, always legal by construction: a softmax over the open (group, mode) pairs, at most five actions a round, every action within the large chunk and the total within greedy's own cap, so the optimiser never needs penalties and the action space is the same as greedy's and the tree search's, only with continuous chunk sizes.

Two modes. Open-loop optimises once from the position over the whole horizon and then plays the plan with the realised shocks, recording both what was planned and what was played. Receding-horizon re-optimises every round over the remaining horizon from the realised state, warm-started from the previous solution shifted by one round, and commits the first round: the mode to compare with the tree search, which is receding-horizon by construction.

Where it starts. The initial mean is greedy's own plan, encoded, so the search starts from a sensible plan rather than noise; whether that was used is recorded with every experiment.

Decision rule: 26 numbers: weights over four state features (value per credit, graph leverage, goal gap, distance below the Tier 1 bar) by mode, a mode bias, a domain bias, a horizon weight and a spend logit. The rule is applied to the realised state each round, so it reacts to shocks by construction; greedy is the point with weight one on value per credit. The default.
Domain shares: 28 numbers a round: logits over the 27 (domain, mode) shares and a spend logit. The credits a domain receives go to its group with the best value per credit in that mode. A fixed multi-year allocation.
Allocation shares: 220 numbers a round: logits over the 219 (group, mode) shares and a spend logit, with a separable (diagonal) covariance. Full control, at a cost.

Sanity

C0: the same two-action toy game the tree search's M0 uses, with a genome of one number a round decoded as B where it is positive. CMA-ES must recover the known optimum.

C0 passeshorizon 6: CMA-ES reached 15 against the optimum 15 (myopic 6) after 2 generations; sequence BBBBBB

Best total by generation: 13, 15.

Experiments

Each CMA-ES experiment's page carries the planner's panels beside the standard ones: the convergence curves, the planned against the realised allocation, and the three methods compared. Regret is the plan's objective minus greedy's at the horizon, paired by seed; 'against the tree search' is the same against the tree-search experiment with the same nation, horizon and objective.

Compare two experiments side by side.

CMA-ES C1, encoding comparison cmaes-c1

One nation, one objective, 20 rounds, open-loop: the three genome encodings at roughly equal compute (population times generations), 5 seeds each, with the greedy pair. Decides the default encoding.

CMA-ES development batch cmaes-dev

A tiny CMA-ES batch (one nation, four rounds, two seeds, a population of four for a few generations) to exercise the tables, the checks, the report and the site's CMA-ES pages while the real experiments run. Not evidence of anything.

The batch's summary table

# CMA-ES summary: cmaes-dev

Regret is the plan's objective minus greedy's at the horizon, paired by seed (positive: CMA-ES did better); 'vs MCTS' is the same against the tree-search experiment with the same nation, horizon and objective. Similarity is the mean cosine similarity of the allocation vectors by round.

| Experiment | Nation | Objective | Encoding | Mode | Dim | Pop | Gens | s/sim | Regret vs greedy | p10 | p90 | Ahead | vs MCTS | Ahead | Sim. greedy | Sim. MCTS | Noisy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| dev-est-cmaes-receding | EST | index | rule | receding | 26 | 4 | 3 | 0 | +0.032 | +0.012 | +0.053 | 100% | +0.052 | 100% | 0.29 | 0.24 | 88% |
| dev-est-cmaes-open-loop | EST | index | domain | open-loop | 112 | 4 | 4 | 0 | −0.013 | −0.029 | +0.004 | 50% | +0.007 | 50% | 0.50 | 0.39 | 100% |

Method agreement: CMA-ES against the tree search

## Method agreement: CMA-ES against the tree search (cmaes-dev, 20 October 2026)

### Estonia, index: dev-est-cmaes-receding against dev-est-mcts-index

- Regret against greedy at the horizon: CMA-ES +0.032 (ahead 100 per cent); tree search −0.020 (ahead 0 per cent of its seeds). CMA-ES against the tree search on the same seeds: +0.052, ahead in 100 per cent.
- Allocation similarity by round (cosine): with the tree search a mean of 0.24 (first five rounds 0.24), with greedy 0.29; the tree search with greedy 0.86.
- Rounds where the two methods disagree on the leading action: 4 of 4. round 1: CMA-ES led with buy Semiconductors & Advanced Computing (hold-out index 11.762); the tree search committed build Semiconductors & Advanced Computing in 1 simulations (mean index 11.692); round 2: CMA-ES led with partner Alliances & Security Cooperation (hold-out index 11.766); the tree search committed buy Semiconductors & Advanced Computing in 1 simulations (mean index 11.675); round 3: CMA-ES led with partner Diplomatic Statecraft & Global Networks (hold-out index 11.542); the tree search committed build Semiconductors & Advanced Computing in 1 simulations (mean index 11.430); round 4: CMA-ES led with partner Diplomatic Statecraft & Global Networks (hold-out index 11.649); the tree search committed build Semiconductors & Advanced Computing in 1 simulations (mean index 11.427)

Open-loop fragility: the plans under new shocks

# C5 open-loop fragility: cmaes-dev

Every open-loop plan replayed as scripted actions under 5 new shock seeds (1001 to 1005). Survival is the share of the plan's shock-free gain over the position that remains on average under the new shocks; the cost of not reacting is greedy's mean objective under the same new seeds minus the plan's.

| Experiment | Nation | Objective | Plans | Planned (mean) | Under new shocks (mean, p10 to p90) | Survival | Greedy under the same shocks | Cost of not reacting |
|---|---|---|---|---|---|---|---|---|
| dev-est-cmaes-open-loop | EST | index | 2 | 11.764 | 11.618 (11.466 to 11.765) | 47% | 11.632 | +0.014 |

What these results are, and are not

Plans the model rewards: Under its current, structural parameters, and under the optimiser's own settings: the encoding, the population, the generations, the shock seeds. Change any of them and the plan may change; that is what C1 and C2 test.
Noisy by nature: A plan's fitness is an average over a few shock futures. Where the noise over seeds is larger than the gap between the best and the mean of a population, the optimiser is choosing among plans it cannot tell apart; the checks say when that happens.
Open-loop plans do not react: An open-loop plan is played whatever the shocks do. C5 replays every such plan under new shocks to measure what that costs; the receding-horizon mode is the one to set against the tree search.
Kept apart on purpose: These pages sit in their own area so the planner's results are not read as the ordinary simulation results or the tree search's. Every CMA-ES experiment also writes the standard tables.

Reconciliation with the brief and the decisions approved: docs/cmaes-reconciliation.md in the repository.