Changelog

Changelog

All notable changes to GLADE are documented in this file.

The format is based on Keep a Changelog, and the project aims to follow Semantic Versioning. While the model remains under active development (pre-1.0), minor releases may introduce breaking changes to configuration and outputs.

Unreleased

Added

• FAOSTAT bulk data is now fetched from a pinned Zenodo mirror (the GLADE input-data mirror, shared with the land-cover extract) instead of FAO’s unversioned bulk endpoints, so re-downloads can no longer silently change the data vintage and fresh clones reproduce the calibrated baseline. The record is pinned by the new data.mirror.zenodo_record config key (which replaces data.land_cover.zenodo_record and participates in calibration provenance, so a vintage bump forces recalibration); maintainers refresh the mirror to a new FAO release with tools/mirror_faostat.py.

• Multiple cropping is now anchored to an observed baseline derived from MIRCA-OS v2 (new automated data source), using the available 2010, 2015, or 2020 release nearest baseline_year. A fixed, documented sequence catalog replaces dynamic combination discovery. Config entries may disable catalog sequences or add zero-baseline greenfield potential under a new name. Irrigated and rainfed observations are attributed separately, competing rotations share crop-area budgets, and outputs are derived per config so the spatial and climate inputs cannot be mixed between runs. crop_production_multi links participate in the land deviation penalty, crop growth cap, cost calibration, and validation-mode pinning like single-crop links. Harvested cycles are reconciled out of the single-crop FAOSTAT baselines so each cycle is counted once. A new multi_crop_cost.csv calibration artefact carries per-(combination, country) bundle corrections. The GAEZ growing-season compatibility gate is removed; observed feasibility comes from MIRCA plus the GAEZ multiple-cropping zone.

• New health.segment_formulation: relax_and_fix option (now the default): a two-pass LP scheme for the health module’s non-convex dose-response curves (solve the relaxation, pin each non-convex curve to the segment of its relaxed intake, re-solve warm-started) with a certified optimality gap checked against health.relax_and_fix_max_gap. If the certificate fails, the solve re-fixes the segments from the repaired solution and, as a last resort, automatically falls back to the exact sos1 MIP seeded with the repaired solution, instead of erroring. Health-enabled solves no longer contain integer variables: full-resolution scenarios solve with the open-source HiGHS solver in about 5 minutes (solving.options_highs: {solver: ipm, run_crossover: on}, no Gurobi license required) and about 30% faster than before under Gurobi, with results that agree across solvers and certify tighter than the previous 0.1% MIP gap. The exact MIP indicators remain available via health.segment_formulation: sos1.

• Interactive Carbon Price Dial: a web widget embedded in the documentation where GHG-price and value-per-life-year sliders drive live land-use maps, net-emissions, system-cost and diet readouts by evaluating the MLP surrogate directly in the browser. Includes constant- and flexible-diet modes, a grams/kcal diet toggle, per-region hover breakdowns of cropland and grazing land, and an advanced panel exposing the remaining surrogate inputs.

• New mlp surrogate method (now the default) with optional seed-ensemble averaging and per-target loss weighting, plus PCA-compressed spatial-field surrogate outputs that reconstruct per-region land-use maps. Surrogate modelling is documented on a dedicated docs page (docs/surrogate_modelling.rst).

• Calibration artefacts are now organized in per-config sets under data/curated/calibration/<source>/, selected via the new calibration.source config key. Each set carries a provenance.yaml stamp of the structural config it was calibrated against; workflow runs error on structural mismatch (downgradable via calibration.accept_provenance_mismatch). tools/calibrate --base <config> calibrates a dedicated set for a structurally divergent config.

• New alternative baseline-diet source diet.source: fbs, derived from FAOSTAT Food Balance Sheets: per-country food supply energy at model-basis densities, corrected for consumer waste. The default remains the GDD-IA source. No calibration artefact set is shipped for the FBS diet, so using it requires running tools/calibrate against your config first (see docs/calibration.rst).

• New diet.anchor_groups_to_gbd option that decouples GBD anchoring of the baseline diet’s risk-factor food groups from the health module. Defaults to the sentinel match_health (follow health.enabled); set true/false to control it independently. Previously anchoring was unconditional. See docs/current_diets.rst for a quantitative description of the difference and the refined-grain caveat. The baseline diet feeds calibration, so a gbd-anchored artefact set (the previous GBD-anchored artefacts) is committed alongside default and consumed by the health-enabled configs via calibration.source: gbd-anchored. Provenance stamps record the resolved anchoring, and tools/calibrate pins the base config’s resolved anchoring across all five calibration steps.

Changed

• The validation configs (config/validation.yaml, docs/config/doc_validation.yaml) and both tutorial configs under config/tutorial/ now leave the health module off, so they run without the manually-downloaded IHME GBD data. Their baseline diet is consequently no longer anchored to GBD intake exposure and they consume the default calibration artefact set, which shifts validation figures somewhat. Turn health back on with health.enabled: true plus calibration.source: gbd-anchored.

• tools/calibrate --check now answers by content instead of by file timestamp. Each artefact set carries a fingerprint.yaml hashing the step’s external inputs (curated, bundled and manually downloaded data, the rule files and scripts its DAG runs, and its config), so git checkout, touch and comment-only config edits no longer report anything, and re-running one step no longer marks its successors stale. New tools/calibrate --record re-stamps a set without solving, for when a code change provably cannot move the artefacts.

• config/default.yaml now has the canonical name default. All shipped configuration fields are validated as required; workflow code no longer supplies hidden fallback values for missing keys.

• planning_horizon now defaults to 2020, matching baseline_year, so an unmodified run solves the observed year the calibration artefacts are fit against. Configs that previously relied on the 2030 default (including gsa.yaml, gsa_fixed_diet.yaml and example.yaml) now solve at 2020 population and GDP levels; set planning_horizon: 2030 explicitly to keep projecting demand forward. The calibration, validation, tutorial and documentation configs no longer restate the year, as it now matches the default.

• Cost calibration is fit against the base config’s water model. It previously pinned water.data.availability: current_use regardless of the configuration it was calibrating, so every artefact set carried cost corrections fit under a different water system than the one they were applied in, while provenance.yaml recorded the base config’s setting. The shipped cost corrections shift by 3-12%; configurations that resolve water seasonally with AWARE scarcity tiers, where the water system actually binds, shift by around 30%.

• MIRCA-OS monthly calendar grids are now packed into one shared sparse preprocessing artefact and reuse exact region-cell coverage across crops. Repeated configuration builds avoid decoding the same dense global NetCDF grids, substantially reducing calendar build time and peak memory without changing its output.

• Spatial preprocessing and model construction now reuse region/class cell mappings, bound raster cache memory, and vectorize repeated aggregation and crop-link operations. This substantially reduces the time and peak memory needed to build a default model without changing its contents.

• Crop-yield and harvested-area preparation now compute exact region and resource-class cell coverage once per configuration and reuse it across crops, substantially reducing build time and peak memory without changing outputs.

• The water system has been rebuilt on a consumption basis. Irrigation previously drew from a single per-region growing-season store sized from Huang et al. withdrawals. It now draws from a regional pool anchored on WaterGAP 2.2e irrigation consumption, through a per-region delivery link whose efficiency eta_c is calibrated at build time against observed consumption, with availability and scarcity characterised by AWARE 2.0. The three water quantities the literature conflates (crop net requirement, consumption, withdrawal) are now distinct and separately reported. New automatic downloads: AWARE 2.0 and WaterGAP 2.2e (ISIMIP3a). The Water Footprint Network “sustainable” supply scenario and the water.supply_scenario key are removed, along with the WFN availability download/processing pipeline and its documentation figure; the source is now water.data.availability (aware or current_use), defaulting to aware. This is a results-affecting default change — the AWARE pool is a looser constraint than the previous binding present-day withdrawal cap.

• Water supply and demand can be resolved at intra-year periods (water.temporal_resolution, a divisor of 12), so a season whose surface cannot meet its demand draws groundwater endogenously instead of being rescued by annual averaging. Crop water demand is placed into periods by the observed MIRCA-OS irrigated crop calendar, retimed to WaterGAP’s monthly requirement. The default is 1 (annual), which is cheap but has a consequence worth stating plainly: at annual resolution the groundwater bands are nearly inert and reported depletion falls to near zero — an artefact of the resolution, not a finding. Studies about water should raise it.

• Surface water and renewable groundwater are characterised as one AWARE renewable resource: each basin’s CF curve spans the joint envelope (WaterGAP surface delivery plus renewable groundwater) and is split at the basin’s surface fraction — the lower slice is period-bound surface, the upper slice becomes annual per-region renewable-groundwater CF bands. This replaces the earlier draft’s flat renewable-groundwater band at the region’s scarcest surface CF, which saturated at the AWARE cutoff (CF 100) almost everywhere and drifted with the temporal resolution. Groundwater is always part of the aware supply (the water.supply.groundwater switch is removed; cap mining at solve time via groundwater_depletion.cap_mm3: 0 for a mining-free system); the current_use source emits no groundwater bands, since its observed-withdrawal pool already contains groundwater. Irrigation’s share of the groundwater-storage depletion trend is attributed by its share of all-sector potential groundwater consumption (new WaterGAP ptotusegw download), so basins mined by municipal or industrial pumping no longer zero irrigation’s renewable band. Water supply fidelity remains a single switch, water.supply.scarcity_tiers (convex AWARE scarcity curves, default off — each pool is one flat availability cap). Scarcity pricing or capping requires it and raises otherwise, since with collapsed curves there is no scarcity signal to price.

• New optional solve-time levers, both off by default: water_scarcity (pricing and/or capping accumulated AWARE scarcity) and groundwater_depletion (pricing and/or capping accumulated mining). With water_scarcity.nonrenewable_cf set, mined groundwater is charged at that CF under scarcity pricing and counts CF-fold against a scarcity cap (a joint constraint), so neither lever can be satisfied by free substitution into fossil groundwater. Analysis gains a water_metrics output with per-region withdrawal, scarcity, renewable groundwater and depletion.

• Model regions are now built basin-aware: GADM provinces are first split along AWARE hydrological basin boundaries, and each country is partitioned into regions balancing geography against basin scarcity (aggregation.regions.basin_scarcity_weight, default 2.0; 0 recovers the previous purely geographic clustering). A province straddling an abundant and a scarce basin is no longer pooled into one region, which used to average away exactly the sub-provincial scarcity that constrains irrigation. Every region is still either contained in one province or a union of whole provinces, so regions remain comparable to political units. The aggregation.regions.allow_cross_border option is removed; regions never cross country borders. This changes default region geometry for every config, so all processing/ artefacts are rebuilt and the tracked calibration sets must be regenerated. Requires the AWARE2.0 basin geopackage (new automatic download).

• The GDD-IA baseline-diet dataset is now retrieved automatically from Zenodo (10.5281/zenodo.20818140, CC-BY-4.0) instead of being obtained on personal request and placed under data/manually_downloaded/. It is now published as Springmann, M., Global dietary estimates for conducting health, environmental and economic impact assessments, Nature Food (2026), doi:10.1038/s43016-026-01388-z, and should be cited as such. The data is unchanged, so results are unaffected; any GDD-IA CSVs under data/manually_downloaded/ are now ignored and can be deleted. The record ships 1990-2020 in five-year steps; for intervening baseline_year values the workflow warns and uses the closest release. Retrieving it needs no account, so a default build now requires no manually-downloaded data at all.

• Tightened the default solve memory allocation (solving.mem_mb) to match the reduced memory usage of full-resolution solves.

• Reformulated the L1 deviation penalties (production, animal-feed, diet stability) from an absolute-value auxiliary variable with two inequality rows per link to an equivalent equality split into non-negative positive/negative deviation parts, and priced the zero-baseline land-conversion penalty directly on link flows. Together with a faster nodal-balance construction in the vendored PyPSA fork, this cuts full-resolution solve times by roughly a third (about 40% fewer constraint rows after presolve) with identical optima up to solver tolerance.

• Improved the optimisation model’s numerical conditioning to remove Gurobi’s “large matrix coefficient range” warning. The CH₄ and N₂O emission buses are now denominated in kilotonnes (previously tonnes) so their flow coefficients sit within a few orders of the CO₂ bus, and a new numerics config block clips physically-negligible coefficients at build time (sub-hectare areas, trace irrigation/carbon fluxes, rounding-level cost corrections). The former land.filtering thresholds now live under numerics. Emission totals and the objective are unchanged (to within solver tolerance); only reported CH₄/N₂O bus flows change units.

• Whole-grain definitions are aligned across diet sources: a new maize-whole food carries GBD’s whole-grain exposure in maize-staple regions, diet.fbs.whole_grain_shares is refit against GBD per-country whole-grain exposure, and GDD-IA cereal energy is re-split by each country’s FBS cereal composition (fixing starved whole-grain intake for Sahel coarse-grain staples). Both calibration artefact sets are refreshed accordingly.

• The health module is now disabled by default (health.enabled: false). With health off, the workflow no longer requires the manually-downloaded IHME GBD data and runs end to end without it; a clear startup error is raised if health (or GBD anchoring) is enabled but the data is absent.

• A default build now requires no credentials: land-cover data is fetched from a CC-BY-4.0 Zenodo mirror instead of the Copernicus Climate Data Store (dropping the CDS API key), and the USDA FoodData Central key is only needed when refreshing nutrition data (data.usda.retrieve_nutrition: true, off by default; the bundled data/curated/nutrition.csv is used otherwise).

• Upgraded the vendored solver stack: linopy to v0.8.0+glade2 (CSR-based matrix construction, frozen constraint storage) and PyPSA to v1.2.0+glade2 (vectorized dual assignment). Together these cut solver matrix assembly by ~60x and dual recovery from ~470 s to ~2 s on full-resolution solves.

Removed

• The MARS surrogate method; supported surrogates are now pce, rf, xgb and mlp.

• Unused configuration keys health.ssb_sugar_g_per_100g, data.gaez.climate_model_ensemble, and sensitivity_analysis.default_surrogate. Surrogate methods are selected explicitly in target and bundle names. Sensitivity scenarios must use the separate food_loss and food_waste factors instead of the removed food_loss_waste convenience key.

Fixed

• Enforced biofuel and industrial demand for vegetable oils no longer draws more of the oil than FAOSTAT reports. The demand links deflated the food-bus draw by the source crop’s moisture content, which is not the moisture of the commodity the food bus carries: palm oil was drawn at 2.5 times its reported non-food use (95 against 38 Mt globally), and soybean and rapeseed oil at about 1.1 times. Palm oil consequently ran a 26 Mt global shortage that the food-demand calibration absorbed by clipping its multiplier at the configured floor. Ethanol demand is unchanged. biofuel_baseline.csv and biogas_crop_demand.csv gain a dm_mt column carrying the dry matter delivered to the biomass bus alongside the source-bus draw.

• docs/config/doc_figures.yaml no longer pins water.data.availability to current_use, which contradicted the gbd-anchored calibration artefacts it consumes and aborted every documentation build on the provenance check. Documentation figures now use the default AWARE water supply, and the regional water availability map reports annual renewable availability, the quantity both availability sources provide.

• The feed, food_waste, food_demand and cost calibration steps no longer depend on the calibrated deviation penalty, which the stability step produces at the end of the chain. The first three pin production to actuals, so the penalty had nothing to act on; the cost step drives production stability through hard bounds, which never read the calibrated L1 costs. Regenerating the artefact sets confirms the dependency was inert: every artefact is unchanged. The calibration chain is a strict forward pass again, and tools/calibrate --check settles after a full run instead of reporting the first four steps stale.

• Workflow startup no longer builds a full copy of the configuration for every configured scenario when deciding whether health data is needed. On configs with generated scenario ensembles this dominated DAG construction: for gsa.yaml (16384 samples) Snakefile parsing drops from ~35 s to ~4 s, and every Snakemake invocation against such a config benefits.

• Solve-time calibration artefacts (feed corrections, exogenous feed and forage, food-demand multipliers, the calibrated deviation penalty) are now declared as inputs of solve_model and calibrate_deviation_penalty, so Snakemake reruns solves when they change.

• The feed calibration step no longer consumes the food-waste calibration artefact, which is produced by a later step in the chain. Feed was effectively fit against whichever vintage happened to be on disk, a full chain could never reach a fixed point, and tools/calibrate --check reported the feed step permanently stale. Feed-side artefacts move by 0.1-0.2%.

• Fixed a GAEZ data artefact where a handful of cells carry a negative net irrigation requirement, which flipped those crop links into spurious water producers. Negative requirements are now clipped to zero.

• Baseline biofuel/industrial and biogas demand is enforced again. Since 2026-05-20 the crops-with-supply safety check in add_biofuel_links ran before any crop production links existed, so every build silently dropped the entire fixed biofuel demand (~290 MtDM globally: maize and sugarcane ethanol plus palm, soybean and rapeseed oil). Baseline solves still looked right because production-stability anchoring mimics the demand, but under strong price signals (water or carbon pricing) the model could simply abandon bioenergy crops instead of meeting their demand. Models must be rebuilt for the fix to take effect; results solved on affected builds understate pressure on bioenergy feedstocks.

0.1.0 - 2026-06-15

First public release of GLADE (Global Land, Agriculture, Diet and Emissions), a global food-systems optimization model built on PyPSA and Snakemake.

Added

• Configuration-driven mixed-integer linear program covering the food supply chain from land and primary resources through crops, processing, livestock, trade, and human nutrition.

• Sub-national optimization regions created by clustering administrative units, connected through hub-based trade networks for crops, foods, and feeds.

• Spatially explicit crop production for 60+ crops with GAEZ-derived yield potentials, multi-cropping, irrigation, and rainfed/irrigated land classes.

• Livestock systems with grazing and feed-based pathways, including enteric fermentation, manure management, and manure-application emissions.

• Greenhouse-gas accounting (CO2, CH4, N2O aggregated to CO2-equivalent) for land-use change, spared-land sequestration, rice cultivation, fertilizer use, and residue incorporation, with configurable GWP factors.

• Nutritional and food-group constraints ensuring caloric and dietary adequacy per country, plus health-impact tracking by disease cluster.

• Reproducible Snakemake workflow with data retrieval, model build, scenario solve, analysis, and plotting targets, organized under results/{config}/.

• Five-stage calibration pipeline (feed, food waste, food demand, cost, production stability) with git-tracked artefacts and a tools/calibrate entrypoint.

• Manifest-based HPC cluster execution path for large scenario sweeps (e.g. global sensitivity analysis) without Snakemake DAG overhead.

• Automatic JSON-schema validation of configuration files.

• Comprehensive Sphinx documentation and tutorial notebooks, published to GitHub Pages.