API Reference¶
This section provides automatically generated documentation for the workflow scripts and utility modules.
Core Scripts¶
Model Building and Solving¶
Food systems optimization model builder.
Modular package for constructing PyPSA networks representing global food production, conversion, trade, and nutrition constraints.
Component Naming and Accessing Conventions¶
This module follows a consistent naming and attribute scheme for all PyPSA components. Never parse component names to extract metadata - always use columns.
Naming Scheme¶
Names use : as delimiter (uncommon in data values, safe for parsing if needed):
Pattern: {type}:{specifier}:{scope}
Buses:
crop:{crop}:{country} e.g., crop:wheat:USA
food:{food}:{country} e.g., food:bread:USA
feed:{category}:{country} e.g., feed:ruminant_forage:USA
residue:{item}:{country} e.g., residue:wheat_straw:USA
group:{group}:{country} e.g., group:cereals:USA
nutrient:{nutrient}:{country} e.g., nutrient:protein:USA
land:cropland:{region}_c{class}_{water} e.g., land:cropland:usa_east_c1_r
land:pasture:{region}_c{class} e.g., land:pasture:usa_east_c1
land:existing_cropland:{region}_c{class}_{water} e.g., land:existing_cropland:usa_east_c1_r
land:new:{region}_c{class}_{water} e.g., land:new:usa_east_c1_r
land:existing_grassland_convertible:{region}_c{class} e.g., land:existing_grassland_convertible:usa_east_c1
land:existing_grassland_marginal:{region}_c{class} e.g., land:existing_grassland_marginal:usa_east_c1
land:spared:{region}_c{class}_{water} e.g., land:spared:usa_east_c1_r
land:spared_existing_grassland_{type}:{region}_c{class} e.g., land:spared_existing_grassland_convertible:usa_east_c1
water:{region} e.g., water:usa_east
fertilizer:supply (global)
fertilizer:{country} e.g., fertilizer:USA
emission:{type} e.g., emission:co2, emission:ghg
biomass:{country} e.g., biomass:USA
fiber:{country} e.g., fiber:USA
health:cluster:{cluster:03d} e.g., health:cluster:001
Links:
produce:{crop}_{water}:{region}_c{class} e.g., produce:wheat_rainfed:usa_east_c1
produce:multi_{combo}_{water}:{region}_c{class}
produce:grassland:{region}_c{class}
pathway:{pathway}:{country} e.g., pathway:milling:USA
convert:{item}_to_{category}:{country} e.g., convert:wheat_to_ruminant_grain:USA
animal:{product}_{feed}:{country} e.g., animal:beef_grassfed:USA
consume:{food}:{country} e.g., consume:bread:USA
use:existing_land:{region}_c{class}_{water}
use:existing_to_pasture:{region}_c{class}
convert:new_land:{region}_c{class}_{water}
convert:new_to_pasture:{region}_c{class}
use:existing_grassland_{type}_to_pasture:{region}_c{class}
spare:land:{region}_c{class}_{water}
spare:existing_grassland_{type}:{region}_c{class}
distribute:fertilizer:{country}
incorporate:residue_{item}:{country}
aggregate:{from}_to_{to} e.g., aggregate:ch4_to_ghg
trade:{commodity}:{from}_{to}
biomass:{item}:{country}
biofuel:{item}:{country}
fiber:{item}:{country}
Stores:
store:group:{group}:{country} e.g., store:group:cereals:USA
store:nutrient:{nutrient}:{country} e.g., store:nutrient:protein:USA
store:water:{region} e.g., store:water:usa_east
store:fertilizer:{country} e.g., store:fertilizer:USA
store:emission:{type} e.g., store:emission:ghg
store:spared:{region}_c{class}_{water} e.g., store:spared:usa_east_c1_r
store:spared_existing_grassland_{type}:{region}_c{class}
store:fiber:{item}:{country}
store:yll:{cause}:cluster{cluster:03d} e.g., store:yll:ihd:cluster001
Generators:
supply:land_{type}:{region}_c{class}_{water} e.g., supply:land_existing_cropland:usa_east_c1_r
supply:fertilizer
supply:exogenous_{category}:{country} e.g., supply:exogenous_ruminant_forage:USA
supply:health:cluster{cluster:03d}
sink:biomass:{country}
slack:{type}:{scope} e.g., slack:water:usa_east
Carrier Column¶
Every bus, link, store and generator carries a carrier column; use it for type
identification. Carriers identify component type only – specific items
(crops, foods, products, regions, …) are stored in metadata columns alongside.
Buses:
crop_{crop},food_{food},feed_{category},residue_{item},group_{group},{nutrient},land_cropland,land_pasture,land_existing_cropland,land_existing_grassland_convertible,land_existing_grassland_marginal,land_new,spared_land,spared_grassland,multi_cropping_land_correction,water,fertilizer,co2,ch4,n2o,ghg,biomass,fiber_demand,healthLinks: -
crop_production: Crop production (usecropcolumn for specific crop) -crop_production_multi: Multi-cropping production (usecropcolumn for combination) -grassland_production: Grassland/pasture production (feed_categoryis alwaysruminant_forage) -animal_production: Animal product production (useproduct,feed_categorycolumns) -food_consumption: Food consumption (usefood,food_groupcolumns) -food_processing: Food processing pathways (usepathway,cropcolumns) -feed_conversion: Crop/food to feed conversion (usecrop,feed_categorycolumns) -trade_crop: Crop trade (usecropcolumn) -trade_food: Food trade (usefoodcolumn) -trade_feed: Feed trade (usefeed_categorycolumn) -biomass_crop: Crop to biomass (usecropcolumn) -biomass_byproduct: Byproduct to biomass (usefoodcolumn) -biomass_disposal: Food to biomass disposal (usefoodcolumn) -biofuel: Food/crop to biomass for biofuel demand (usecropcolumn) -fiber_demand: Food bus to per-country fiber bus -fertilizer_distribution: Fertilizer distribution -emission_aggregation: GHG emission aggregation -land_use,land_conversion,existing_to_pasture,new_to_pasture,existing_grassland_to_pasture,spare_land,spare_existing_grassland,residue_incorporationGenerators:
land_existing_cropland,land_existing_grassland_convertible,land_existing_grassland_marginal,fertilizer,land_slack,exogenous_feed,exogenous_forage_cal,exogenous_protein_cal,exogenous_roughage_cal(the last three are added at solve time by calibration generators),slack_positive_feed,slack_negative_feed,biomass,health
Custom Columns¶
All components have consistent domain-specific columns for filtering:
Buses:
country: str | NaN - country code (NaN for global/regional)region: str | NaN - region name (land / water / spared-land buses)resource_class: int | NaN - land quality class (land buses)water_supply: str | NaN - “irrigated” / “rainfed” (cropland / new / existing_cropland buses)land_type: str | NaN - “convertible” / “marginal” (existing-grassland buses)feed_category: str | NaN - feed category (feed buses)health_cluster: int | NaN - GBD cluster id (health buses)
Links:
country: str | NaN - country coderegion: str | NaN - region namecrop: str | NaN - crop namefood: str | NaN - food namefood_group: str | NaN - food group nameproduct: str | NaN - animal product namefeed_category: str | NaN - feed categoryresource_class: int | NaN - land quality classwater_supply: str | NaN - “irrigated” or “rainfed”land_type: str | NaN - e.g. “convertible” or “marginal” for grassland pools
Stores:
country: str | NaN - country codefood_group: str | NaN - food group namenutrient: str | NaN - nutrient nameregion: str | NaN - region name (spared-land stores)resource_class: int | NaN - land quality class (spared-land stores)water_supply: str | NaN - “irrigated” / “rainfed” (spared-land stores)land_type: str | NaN - grassland-pool type (spared-grassland stores)health_cluster: int | NaN - GBD cluster id (yll stores)cause: str | NaN - GBD cause name (yll stores)
Generators:
country: str | NaN - country coderegion: str | NaN - region name
Global Constraints:
country: str | NaN - country codefood_group: str | NaN - food group namenutrient: str | NaN - nutrient nameproduct: str | NaN - product namecrop: str | NaN - crop name
Accessing Components¶
Use regular pandas indexing with carrier and domain columns. Fail fast when
no components found:
# Get food group stores for a specific group
group_stores = n.stores.static[n.stores.static["carrier"] == f"group_{group}"]
if group_stores.empty:
raise ValueError(f"No stores found for food group '{group}'")
# Get crop production links for a specific country
crop_links = n.links.static[
(n.links.static["carrier"] == "crop_production") &
(n.links.static["crop"] == crop) &
(n.links.static["country"] == country)
]
if crop_links.empty:
raise ValueError(f"No production links for crop '{crop}' in '{country}'")
# Get all consumption links
consume_links = n.links.static[n.links.static["carrier"] == "food_consumption"]
Solving utilities for the food systems optimization model.
Modular package providing specialized constraint builders for the PyPSA linopy model during optimization.
Data Preparation¶
Process UN WPP population data for total and age-specific counts.
Pre-compute health data for SOS2 linearisation in the solver.
- class workflow.scripts.prepare_health_costs.RelativeRiskTable[source]¶
Bases:
dict[tuple[str,str,str],dict[str,ndarray]]Container mapping (risk, cause, age) to exposure grids and log RR values.
- workflow.scripts.prepare_health_costs.main()[source]¶
Main entry point for health cost preparation.
- Return type:
Basin-aware clustering of GADM level-1 provinces into model regions.
Provinces are first split along hydrological basin boundaries (overlay with the
AWARE basins), so a province straddling an abundant and a scarce basin can be
separated – otherwise pooling the two averages away the sub-provincial water
scarcity that drives groundwater use. The province-basin pieces are then
clustered per country into exactly target_count regions under a nesting
invariant: every model region is either contained in one GADM province (a large
province is split into sub-regions) or a union of whole provinces (small
provinces are merged), never a mix of partial pieces across provinces – so
regions stay cleanly comparable to political units.
Within each country the regions are balanced on geography and basin scarcity
(basin_scarcity_weight sets their relative influence; 0 recovers plain
geographic clustering). A reconciliation step splits the largest splittable
region until the exact target is reached, since a province allocated more
sub-regions than it has basin pieces would otherwise under-produce.
- workflow.scripts.build_regions.cluster_country(pieces, k, scarcity_weight, method, random_state)[source]¶
Assign a local region id (0..k-1) to each province-basin piece of a country.
pieceshas one row per (province, basin) piece with columnsprov,px,py(equal-area centroid),areaandcf(basin scarcity). Provinces large enough for >=2 regions are split: their pieces are clustered (on geography + weighted scarcity) into sub-regions that stay within the province. All remaining provinces are grouped into whole-province regions. Every region is therefore either contained in one province or a union of whole provinces, and the total is exactlyk.
- workflow.scripts.build_regions.cluster_regions(pieces, target_count, scarcity_weight, method='kmeans', random_state=0)[source]¶
Basin-aware clustering of province-basin pieces into
target_countregions.piecesis the overlay of GADM level-1 provinces with AWARE basins, with columnsGID_0(country),prov(GADM province id),cf(basin scarcity) and geometry. Per-country region counts are allocated proportionally to area; each country is then partitioned (seecluster_country) so every region is either contained in one province or a union of whole provinces. Basin scarcity separates regions within a province so a scarce sub-basin is not pooled with an abundant one.
Yield and Land Processing¶
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
- workflow.scripts.compute_resource_classes.read_raster_cells_float(path, cell_ids, expected_shape)[source]¶
- workflow.scripts.compute_resource_classes.yield_multiplier(crop, *, use_actual_yields, conversions)[source]¶
- workflow.scripts.compute_resource_classes.scale_yield(raw, crop, *, use_actual_yields, conversions, moisture)[source]¶
- workflow.scripts.compute_resource_classes.compute_max_yield_score(yield_paths, pairs, expected_shape, *, use_actual_yields, conversions, moisture)[source]¶
- workflow.scripts.compute_resource_classes.compute_regional_harvested_area_score(yield_paths, harvested_paths, pairs, region_raster, regions_gdf, expected_shape, *, conversions, moisture, mapping_path, production_path, fdd_shares_path, non_food_crops)[source]¶
- workflow.scripts.compute_resource_classes.classify_by_region(score, region_raster, quantiles)[source]¶
Assign positive scores to unweighted within-region quantile bins.
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
- workflow.scripts.aggregate_class_areas.load_scaled_fraction(path, *, target_shape=None, target_transform=None, target_crs=None)[source]¶
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
Water Resources¶
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
Build region-level irrigation water availability and a convex water-scarcity supply curve from the AWARE2.0 dataset (Seitfudem et al. 2025, WaterGAP2.2e).
AWARE2.0 supplies, per native basin and month:
NatAvail(naturalised availability),EWR(environmental reserve),basin_area, andAMD_final(availability minus demand, m3/m2/month) inAWARE20_Intermediate_Variables.xlsx;the irrigation-sector water demand
2019_agri_pHWCinAWARE20_Native_CFs.xlsx;native-basin polygons in
AWARE20_Native_CFs_geospatial.gpkgwhose feature id equalsBasin_ID.
The published AWARE characterisation factor (CF) is marginal:
CF = AMD_world_avg / AMD, clipped to [0.1, 100] (and 100 where AMD <= 0).
That is only valid for small inventories. The model re-decides all irrigation,
so we reconstruct the non-marginal curve: as the model draws a volume V from
a basin’s agricultural pool, the basin’s AMD falls and the CF rises. Anchoring to
the published AMD_final (which already carries AWARE’s hydrological
corrections) and adding back the agriculture the model re-decides gives, per
basin-month:
pool = max(area * AMD_final + agri_pHWC, 0) [m3/month] AMD0 = pool / area [m3/m2/month, no-agriculture AMD] CF(x) = clip(AMD_world_avg / (AMD0 * (1 - x)), 0.1, 100)
where x in [0, 1] is the fraction of the pool drawn. At x corresponding
to AWARE’s 2019 irrigation draw this reproduces the published CF. GLADE then
replaces the pool capacity with WaterGAP’s joint renewable envelope –
surface delivery plus renewable groundwater, one resource in AWARE’s
methodology (availability is basin discharge including baseflow; CF application
is source-agnostic) – while retaining the AWARE CF curve. The convex CF curve
is discretised into sub-segments (closed-form average CF per segment) and split
at each basin’s surface fraction: the lower slice is the period-bound surface
delivery, the upper slice the annual renewable-groundwater buffer (the
marginal, costlier-to-access renewable source). Surface segments merge across
basins into a per region-month merit-order supply curve binned into
N_TIERS tiers; groundwater segments merge across basins and months into
per-region annual bands. The lowest-CF (most abundant) water is drawn
first, so a plain LP reproduces the convex integral with no integrality.
Monthly resolution. This script keeps the full monthly signal: it emits one
convex tier curve per region and month (12 curves per region). The temporal
resolution the model actually solves at is chosen downstream in
compose_water_supply (water.temporal_resolution), which groups whole
months into equal periods and re-merges the monthly curves. Keeping this stage
month-resolved lets the temporal resolution change without re-running the
(expensive) AWARE basin overlay. No demand-based capping is applied here: the
seasonal bind (a month’s surface use cannot exceed that month’s availability) is
enforced by the LP once the period buses exist, not baked into an annual scalar.
Outputs (all keyed by model region):
monthly_region_water.csv: renewable pool per region-month (m3);region_growing_season_water.csv: annual availability per region;region_water_tiers.csv:region, month, tier, capacity_mm3, marginal_cf– the per-month convex surface supply curves consumed bycompose_water_supply;region_renewable_gw_tiers.csv:region, tier, capacity_mm3, marginal_cf– the annual renewable-groundwater CF bands, on the same curve;
All volumes are WaterGAP’s: its regional totals are allocated directly to the
intersecting AWARE basins before the CF tiers are built. AWARE contributes the
scarcity (CF) curve. The eta_c consumption anchor comes from
build_region_watergap.py on the same basis.
- workflow.scripts.build_region_water_aware.load_basin_pool(intermediate_path, native_cfs_path)[source]¶
Return per-basin agricultural pool and no-agriculture AMD.
Index is
Basin_ID; columns arearea(m2) plus, for each month,pool_{Mon}(m3/month) andamd0_{Mon}(m3/m2/month).
- workflow.scripts.build_region_water_aware.build_basin_region_cells(basins_path, regions, basin_ids)[source]¶
Return model-region intersections of the AWARE basins.
The AWARE geospatial layer encodes
Basin_IDas the feature id; only basins present inbasin_ids(those with pool data) are retained. Thesharecolumn is the intersection’s area share of its native basin and is used to apportion AWARE’s basin-level AMD pool. Its geometry supports a separate, direct overlay of WaterGAP surface delivery.- Parameters:
basins_path (
str)regions (
GeoDataFrame)basin_ids (
Index)
- Return type:
GeoDataFrame
- workflow.scripts.build_region_water_aware.bin_merit_tiers(seg, keys, n_tiers)[source]¶
Merge CF segments per key group into equal-volume merit-order tiers.
segholds columnskeys + [cf, volume](m3). Segments are sorted in ascending-CF order per group and binned inton_tiersequal-volume tiers, each carrying its volume-weighted mean CF. Output columns arekeys + [tier, capacity_mm3, marginal_cf].
- workflow.scripts.build_region_water_aware.build_region_curves(long, n_subsegments, n_tiers)[source]¶
Surface tiers per (region, month) and renewable-GW bands per region.
longholds one row per (region, month, basin) with columnsregion,month,volume(m3 of the joint renewable envelope in that month),amd0(m3/m2/month) andsurface_frac(the surface share of the row’s volume). Each row is expanded inton_subsegmentssegments of the convex CF curve over its full volume; each segment is then split at the row’s surface fraction – the lower (more abundant) slice is the period-bound surface delivery, the upper slice the annual renewable-groundwater buffer, reflecting groundwater as the basin’s marginal renewable source. Surface segments merge into per (region, month) tiers; groundwater segments merge across months into per-region annual bands. Summed capacities reproduce the surface and groundwater envelopes exactly (the split is volume-conserving).
- workflow.scripts.build_region_water_aware.aggregate_watergap_by_basin(pirruse_path, pirrusegw_path, continental_area_path, basin_cells, reference_start, reference_end)[source]¶
Aggregate WaterGAP irrigation delivery to region-basin cells.
Returns two
(region, basin_id, month)-indexed series: surface deliverymax(pirruse - pirrusegw, 0)and groundwater deliverypirrusegw, on WaterGAP’s native grid. Aggregating them directly to the AWARE basin intersections preserves WaterGAP’s spatial allocation within each model region before the AWARE CF tiers are built.
- workflow.scripts.build_region_water_aware.scale_pool_to_watergap(cells, surface_m3, renewable_gw_m3, basin_surface_m3, basin_gw_m3)[source]¶
Rescale AWARE pools to the joint WaterGAP renewable envelope.
AWARE
availabilityis basin river discharge, which counts through-flow discharge as divertible and hugely overstates the surface water accessible to irrigation in groundwater-dependent basins (e.g. the Ogallala: an AWARE pool ~100x the surface WaterGAP’s detailed allocation actually supplies). Its monthly shape is unregulated discharge timing, while WaterGAP’shistsocruns operate every GRanD reservoir >= 0.5 km3: the monthly profile of its irrigation surface consumption (pirruse - pirrusegw) is regulated, demand-timed delivery. We keep AWARE’s scarcity structure – the per-basin CF curve (a function ofamd0, not volume) – but replace the volume with WaterGAP’s.AWARE treats renewable water as ONE resource (its availability is basin discharge including baseflow; CF application is source-agnostic), so the curve’s draw domain is the joint renewable envelope: surface delivery plus renewable groundwater. Each cell row gets
region_pool= surface + renewable-GW target andsurface_frac= the surface share of it, along which the curve is later split into the period-bound surface slice (drawn first) and the annual renewable-groundwater slice (the marginal, buffered source, drawn after). Regional WaterGAP totals are the conservation anchors (surface per region-month; renewable GW annual, spread by the overlay’s basin-month profile); the direct overlay supplies within-region basin shares. Delivery without a mapped AWARE basin is retained on an explicit ceiling-CF tier.surface_m3is indexed by(region, month),renewable_gw_m3byregion(annual), and the two basin overlays by(region, basin_id, month).- Parameters:
cells (
DataFrame)surface_m3 (
Series)renewable_gw_m3 (
Series)basin_surface_m3 (
Series)basin_gw_m3 (
Series)
- Return type:
DataFrame
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
Aggregate WaterGAP 2.2e (ISIMIP3a) fields to model regions: the irrigation surface-water availability that caps the AWARE scarcity curve, and the renewable and non-renewable (mined) groundwater bands.
All fields are the standard ISIMIP3a WaterGAP2.2e output, obsclim climate /
histsoc (with human water use) setup, gswp3-w5e5 forcing, monthly
1901-2019, 0.5 degree. Irrigation-sector, source-split water use is published
directly, on both a withdrawal and a consumption basis:
continentalarea(km2): WaterGAP’s static continental area, including land and surface-water bodies but excluding ocean. It is the required volume conversion area for WaterGAP’s flux and storage fields.groundwstor(mm): groundwater storage compartment. Its negative long-term trend is groundwater depletion / mining (Doll et al. 2014).pirruse(kg m-2 s-1 = mm/s): potential irrigation water consumption (the evapotranspired portion), all sources.pirrusegw: the part ofpirrusesupplied from groundwater.ptotusegw: potential groundwater consumption of all sectors; the denominator of irrigation’s share of groundwater abstraction.
The model works on a consumption basis (crops draw beneficial ET, delivered from
the consumption-basis pool), so the use (consumption) variables are the right
ones. From them:
irrigation surface availability =
pirruse - pirrusegw(per region and month). This is WaterGAP’s assessment of how much of irrigation’s consumptive demand its detailed water allocation supplies from surface water. It replaces AWARE’s basin-discharge availability (which counts through-flow river discharge as divertible and so hugely overstates the accessible surface in groundwater-dependent basins such as the Ogallala). Crucially it is kept monthly: WaterGAP’shistsocruns operate every GRanD reservoir >= 0.5 km3 (Hanasaki scheme), so the monthly timing ofpirruse - pirrusegwis regulated, demand-timed delivery – reservoirs carry wet-season discharge into the irrigation season inside WaterGAP. AWARE’s monthly shape is unregulated discharge timing and strands that delivery in the wet months. The AWARE scarcity (CF) curve is kept; the per region-month volumes are rescaled to this envelope inbuild_region_water_aware.py.mined groundwater = the groundwater-storage decline. The trend reflects all users, so irrigation’s part is attributed by its share of potential groundwater consumption (
pirrusegw / ptotusegw, same basis and window); a basin mined by municipal or industrial pumping does not zero irrigation’s renewable band.renewable groundwater =
max(pirrusegw - mined_irrigation, 0): the recharged part of irrigation groundwater consumption.
Outputs (keyed by model region):
region_watergap_surface.csv:region, month, surface_consumption_mm3– monthly climatological irrigation surface consumption, the availability envelope for the AWARE curve;region_groundwater_depletion.csv:region, mined_mm3, irrigation_gw_share, mined_irrigation_mm3, renewable_gw_mm3– the renewable-groundwater volume anchor (and mining diagnostics) consumed bybuild_region_water_aware.pyandcompose_water_supply.py;region_agri_consumption.csv:region, agri_consumption_m3– annual total irrigation consumption (pirruse), the demand anchor foreta_cand the mining ceiling. Replaces the AWARE 2019agri_pHWCanchor so that every volume (supply envelope, groundwater bands, demand anchor) comes from one WaterGAP simulation and window; AWARE then contributes the scarcity (CF) valuation and its native basin geometry.region_watergap_demand.csv:region, month, irrigation_consumption_mm3– the monthly resolution of the samepirruseclimatology: WaterGAP’s demand-timed irrigation requirement (net of effective precipitation). Used to retime crop-calendar demand shares so that region-month demand totals are consistent with the supply envelope above (build_mirca_crop_calendar.py).
- Reference:
Doll et al. (2014). Global-scale assessment of groundwater depletion and related groundwater abstractions. Water Resources Research, 50, 5698-5720. Muller Schmied et al. (2024). WaterGAP v2.2e. Geosci. Model Dev., 17, 8817.
- workflow.scripts.build_region_watergap.load_continental_area(path, lat, lon)[source]¶
Load WaterGAP’s continental cell area as m2 on the requested grid.
- workflow.scripts.build_region_watergap.compute_depletion_raster(groundwstor_path, continental_area_path, trend_start, trend_end)[source]¶
Return per-cell groundwater depletion (m3/yr), latitudes and longitudes.
Depletion is the negative linear trend of annual-mean groundwater storage over
[trend_start, trend_end](inclusive), converted from mm/yr to a volume via cell area. Cells with a non-negative trend (stable or recovering storage) contribute zero.
- workflow.scripts.build_region_watergap.compute_monthly_flux_raster(path, variable, continental_area_path, reference_start, reference_end)[source]¶
Return a WaterGAP flux variable as per-cell monthly volumes (m3/month).
variableis a monthly water-use flux (kg m-2 s-1 = mm/s), e.g.pirruseorpirrusegw. Averaged into a 12-month climatology over[reference_start, reference_end](inclusive), clipped at zero per cell-month (negative cells are net returns / recharge) and converted to volumes via month length and cell area. Shape (12, nlat, nlon).
- workflow.scripts.build_region_watergap.aggregate_to_regions(values_m3, lat, lon, regions_gdf)[source]¶
Coverage-weighted sum of a per-cell volume (m3/yr) into regions.
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
Compose the scenario-agnostic water-supply tables from the per-month
availability curves, grouping months into the model’s temporal periods
(water.temporal_resolution).
The upstream availability stage (build_region_water_aware /
process_huang_irrigation_water) emits a convex tier curve per region and
month. This stage groups whole months into temporal_resolution equal
periods (month m -> period (m - 1) * T // 12), re-merges the monthly
curves within each period into a single convex merit-order surface curve, and
produces two tables the model build consumes:
region_water_tiers.csv(region, period, tier, capacity_mm3, marginal_cf, source): the per-period surface supply. Surface is period-bound – a river cannot be pumped next season without a reservoir – so each period’s cap binds its own draw.region_groundwater_bands.csv(region, source, band, capacity_mm3, marginal_cf): the annual per-region groundwater bands, emitted for theawareavailability source. Groundwater is an aquifer, an annual buffer that can be pumped in any period, so it attaches to a single per-region groundwater bus in the model build and is shared across periods rather than split per period. The renewable bands (source = groundwater_renewable) carry the AWARE CF curve slice computed inbuild_region_water_aware(the upper part of the joint renewable envelope); the non-renewable / mined band (source = groundwater_nonrenewable) is a generous non-binding aquifer ceiling carrying a real pumping cost that orders it last. Because groundwater is additive rather than a relabel of surface, mining emerges endogenously wherever surface plus renewable groundwater fall short – it is not capped at renewable availability. Thecurrent_usesource emits no bands: its Huang withdrawal pool already contains the groundwater-supplied part, so additive bands would double-count it.
Keeping the month grouping here means changing the temporal resolution does not re-run the expensive basin overlay.
supply.scarcity_tiers finishes the tables: keep the convex scarcity curves
(per-period surface and annual renewable groundwater), or collapse each pool to
one flat cf = 0 tier/band – a simple availability cap for studies where water
is not the focus.
The monthly and growing-season availability tables are copied through unchanged.
- workflow.scripts.compose_water_supply.month_to_period(months, temporal_resolution)[source]¶
Map calendar months (1..12) to period indices (0..T-1) in equal blocks.
Tmust divide 12 (enforced by the config schema), so each period spans12 // Tconsecutive months.
- workflow.scripts.compose_water_supply.aggregate_months_to_periods(monthly_tiers, temporal_resolution)[source]¶
Group per-month tiers into per-period convex curves.
monthly_tiershas columnsregion, month, tier, capacity_mm3, marginal_cf. Returnsregion, period, tier, capacity_mm3, marginal_cfwith the merged convex curve re-binned toN_TIERSequal-volume tiers per region-period. Fully vectorised: the segments are sorted into merit order, an equal-volume tier index is assigned from the grouped cumulative volume, and each tier’s marginal CF is the volume-weighted mean of its segments.- Parameters:
monthly_tiers (
DataFrame)temporal_resolution (
int)
- Return type:
DataFrame
- workflow.scripts.compose_water_supply.collapse_single(tiers)[source]¶
One flat renewable tier per region-period (cf = 0): a simple hard cap.
- Parameters:
tiers (
DataFrame)- Return type:
DataFrame
- workflow.scripts.compose_water_supply.build_groundwater_bands(renewable_gw_tiers, surface_tiers, agri_consumption_mm3, ceiling_factor, scarcity_tiers)[source]¶
Annual per-region groundwater bands (renewable + non-renewable).
All bands are annual: an aquifer integrates recharge over the year and can be pumped in any period, so – unlike surface, which is period-bound – they attach to a single per-region groundwater bus in the model build, shared across all periods.
groundwater_renewable: the AWARE CF bands of the renewable-groundwater slice of the joint renewable envelope (renewable_gw_tiers, frombuild_region_water_aware). Withscarcity_tiersoff they collapse to one flat cf = 0 band per region, mirroring the surface collapse.groundwater_nonrenewable: a generous non-binding aquifer ceiling (ceiling_factor * Cwith C the annual consumption anchor, falling back to the region’s total surface capacity where C is zero), cf 0, ordered last by its pumping cost. The volume actually mined is set endogenously by how far surface plus renewable groundwater fall short of demand.
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
Shared intra-year water-period helpers.
The model resolves water in T equal intra-year periods
(config["water"]["temporal_resolution"]). Months (0-based) map to periods by
month * T // 12 (equal blocks), matching compose_water_supply. These
helpers place a crop’s irrigation demand into those periods by day-overlap of its
growing season, and are used by both the single-crop path
(build_model.crops) and the multi-cropping Stage-2 split
(build_multi_cropping), so the two stay consistent.
- workflow.scripts.water_periods.month_overlaps(start_days, length_days)[source]¶
(N, 12)matrix of days each season overlaps each calendar month.Vectorised over the
Nseasons; wrap-around past new year is handled by also intersecting each season with the+365-day copy of every month (a season is clipped to at most one year, so two copies suffice). Rows with a missing or non-positive season are all-zero.
- workflow.scripts.water_periods.month_to_period_matrix(water_periods)[source]¶
(12, T)one-hot map from calendar month to intra-year period.
(N, T)fraction of each season’s irrigation demand per period.The requirement is apportioned by the days the growing season spends in each month, aggregated to periods. A missing or degenerate season falls back to an even split across periods. Fully vectorised over the
Nseasons.
(N, 12)observed monthly demand shares forcropoverregions.calendaris the long MIRCA-OS calendar table (region, crop, month, share). Regions absent for this crop get an all-zero row (GAEZ fallback downstream). The 12 columns are calendar months 1..12.
(N, T)period demand shares from an observed monthly calendar.monthly_sharesis(N, 12)of observed per-month growing-area shares (MIRCA-OS). Rows with a positive monthly profile are binned into theTperiods; rows that are all-zero (no observed calendar for that season) fall back to the GAEZ growing-season splitperiod_demand_shares(start, length). Fully vectorised. Returns the shares and the(N,)boolean mask of rows with an observed calendar.
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
Build per (region, crop) monthly irrigated water-demand shares from the MIRCA-OS 2015 monthly growing-area grids, retimed to WaterGAP’s monthly irrigation requirement.
The water supply envelope is WaterGAP’s reservoir-regulated monthly delivery, but crop water demand was placed in the year by GAEZ growing seasons – the potential, yield-maximising calendar, which systematically disagrees with the observed cropping calendar in the major irrigated systems (Indus, Nile, Gangetic plain). That demand-calendar mismatch strands deliverable surface water in months the model does not demand it and mines groundwater in the months it does.
MIRCA-OS publishes the observed calendar spatialised: monthly growing-area grids per crop and sub-crop (5 arcmin, ha per cell-month, planting to maturity). The 2015 vintage is used deliberately: the 2020 crop calendar misplaces the entire northwest-India wheat belt (~16.5 Mha) into the monsoon window, and MIRCA2000 v1.1 splits the same belt 50/50 between monsoon and rabi windows; the 2015 calendar carries the correct rabi timing (verified against Sacks et al. 2010 and USDA FAS calendars). Calendars are climatological, so the 2015 timing is valid for the 2020 baseline areas.
Growing-area months are not requirement months, however: a crop’s net
irrigation requirement is modulated by evapotranspiration minus effective
precipitation within its season (it collapses during the monsoon and peaks in
the dry shoulder months), while the growing-area profile weights every month
of the season equally – including dormant winter-wheat months and rain-fed
monsoon months. WaterGAP’s pirruse carries the requirement timing on the
same simulation and basis as the supply envelope. The two are combined by
iterative proportional fitting (retime_shares_to_demand): per region, the
crop x month prior (MIRCA area shares weighted by each crop’s annual
irrigation water) is scaled so that region-month column totals match the
WaterGAP monthly requirement shape while per-crop annual totals and the
structural zeros of the observed calendar are preserved exactly. Wheat can
shift within its rabi window but never into the monsoon.
Sub-crops of the same class (Wheat1 + Wheat2, Rice1..3) are
summed: all cycles of a crop feed the same regional water buses, so only the
summed monthly profile matters. The source grids are packed once into a shared
sparse artefact. Exact region/cell coverage is computed once per configuration
and reused across crops, then the regional totals are normalised into monthly
shares. Only irrigated grids are used – rainfed links carry no water.
A calendar-only supplement mapping adds MIRCA classes excluded from the
multi-cropping concordance (sugar cane, pulses, fodder) so that the large
irrigators they represent are placed by observed timing instead of the flat
GAEZ fallback; a class mapping to several GLADE crops gives each the same
profile with the class area split evenly for weighting.
Output mirca_crop_calendar.csv: region, crop, month, share, area_ha
with one row per (region, mapped GLADE crop, month) where the region grows
the crop under irrigation in MIRCA-OS. share is the retimed demand share
(summing to 1 over the year); area_ha is the raw MIRCA-OS growing area
(provenance, not rescaled). Consumers (single-crop links in
build_model.crops and the multi-cropping cycle split in
build_multi_cropping) fall back to the GAEZ growing season where a
(region, crop) is absent.
- workflow.scripts.build_mirca_crop_calendar.get_extract_buffer(shape)[source]¶
Return the reused float64 exact_extract buffer for
shape.
- workflow.scripts.build_mirca_crop_calendar.region_coverage_entries(grid, regions, xmin, ymin, xmax, ymax, crs_wkt)[source]¶
Extract exact region/cell coverage once for the common MIRCA grid.
- workflow.scripts.build_mirca_crop_calendar.aggregate_packed_crop_by_region(packed, label_indices, grid, region_names, region_rows, region_cell_ids, coverage, extract_values)[source]¶
Aggregate packed subcrop areas with one reusable monthly grid.
Normalise per-(region, crop) monthly areas into demand shares.
monthly_areahas columnsregion, crop, month, area_ha. Regions with an annual total belowmin_area_haare dropped (no reliable calendar signal); the remaining shares sum to 1 over the 12 months.- Parameters:
monthly_area (
DataFrame)min_area_ha (
float)
- Return type:
DataFrame
- workflow.scripts.build_mirca_crop_calendar.water_weights(yield_paths, crops)[source]¶
Per-(region, crop) annual irrigation water requirement rates (m3/ha).
Reads the irrigated GAEZ yield tables (
{crop}_i.csv) for the requested crops and returns the suitable-area-weighted mean ofwater_requirement_m3_per_haover resource classes, as a DataFrame with columnsregion, crop, req_m3_per_ha. Crops without an irrigated yield table are absent (they build no irrigated links, so they carry no weight).
Retime per-(region, crop) monthly shares to a region-month demand shape.
Iterative proportional fitting on each region’s crop x month demand matrix: the prior is
share * weight(MIRCA growing-area placement scaled by each crop’s annual irrigation water), the row marginals are the crop weights (annual totals preserved), and the column marginals are the region’s WaterGAP monthly requirement shape scaled to the same total. Multiplicative updates preserve structural zeros, so demand moves within each crop’s observed season, never outside it. Months WaterGAP serves that no crop grows in are left unmet (the column scaling skips empty columns); the final row normalisation keeps every profile summing to exactly 1.shares: columnsregion, crop, month, share(full 12-month profiles).weights: (region, crop)-indexed annual water volumes.demand: columnsregion, month, irrigation_consumption_mm3. Rows without a positive weight, and regions whose annual demand is belowmin_demand_mm3(no reliable requirement signal), pass through unchanged. Returnsshareswith thesharecolumn retimed.
Process Huang et al. gridded irrigation water withdrawal data.
Aggregates monthly gridded irrigation water withdrawal, stored as a depth (mm/month at 0.5 degree resolution), converting it to a volume via grid-cell area before aggregating to model regions.
This script produces the availability tables in the shared schema of the “aware” source, including a single zero-CF supply tier per region-month that reproduces a hard cap at current withdrawals, so the two availability sources can be used interchangeably.
- Reference:
Huang et al. (2018). Reconstruction of global gridded monthly sectoral water withdrawals for 1971-2010 and analysis of their spatiotemporal patterns. Hydrology and Earth System Sciences, 22, 2117-2133. https://doi.org/10.5194/hess-22-2117-2018
- workflow.scripts.process_huang_irrigation_water.compute_month_overlaps(start_day, length_days)[source]¶
Return array of day overlaps per month for given season.
- workflow.scripts.process_huang_irrigation_water.aggregate_gridded_to_regions(data_array, lon, lat, regions_gdf)[source]¶
Aggregate a gridded array to regions by summation.
- Parameters:
- Return type:
DataFrame- Returns:
DataFrame with ‘region’ and ‘value’ columns.
- workflow.scripts.process_huang_irrigation_water.load_crop_growing_seasons(crop_files)[source]¶
Load area-weighted crop growing seasons from yield files.
- workflow.scripts.process_huang_irrigation_water.compute_region_growing_water(region_month_water, crop_seasons, regions)[source]¶
Compute growing-season weighted water availability.
but uses ‘water_available_m3’ column from monthly data.
- workflow.scripts.process_huang_irrigation_water.process_huang_irrigation(nc_path, regions_path, crop_files, reference_year=2010)[source]¶
Process Huang et al. irrigation NetCDF to regional water data.
- Parameters:
- Returns:
DataFrame with monthly region water (region, month, water_available_m3)
DataFrame with growing season water (shared availability schema)
- Return type:
Tuple of
Utility Modules¶
SPDX-FileCopyrightText: 2026 Koen van Greevenbroek
SPDX-License-Identifier: GPL-3.0-or-later
- workflow.scripts.raster_utils.calculate_all_cell_areas(src, *, repeat=True)[source]¶
Return per-pixel area in hectares for a geographic (lon/lat) raster.
- Parameters:
src (
DatasetReader) – Raster opened with rasterio, expected in lon/lat coordinates.repeat (
bool) – When True (default) repeat the per-row areas across columns, yielding a 2D array matching the raster shape. When False, return the 1D per-row areas without repeating, which is useful when the caller can rely on broadcasting to avoid materialising the full 2D matrix.
- Return type:
- workflow.scripts.raster_utils.scale_fraction(arr)[source]¶
Scale array to 0..1 if stored as 0..100 or 0..10000; clip to [0,1].
- workflow.scripts.raster_utils.raster_bounds(transform, width, height)[source]¶
Calculate bounding box from raster transform and dimensions.
Visualization Scripts¶
Plot a Plate Carrée map of resource classes by grid cell.
- workflow.scripts.plotting.plot_resource_classes_map.plot_resource_classes_map(classes_path, regions_path, output_path)[source]¶
- Parameters:
classes_path (
Path)regions_path (
Path)output_path (
Path)
- Return type:
- workflow.scripts.plotting.plot_crop_production_map.crop_groups_from_config(config)[source]¶
Build crop-to-group and group-color mappings from
plotting.crop_groups.- Returns:
Mapping from crop name to group display name. crop_group_colors: Ordered mapping from group display name to hex color.
- Return type:
crop_to_group
- Parameters:
config (
dict)
Plot objective breakdown and visualize health risk factors by region.
- class workflow.scripts.plotting.plot_health_impacts.HealthInputs(risk_breakpoints, cluster_cause, cause_log_breakpoints, cluster_summary, clusters, cluster_risk_baseline)[source]¶
Bases:
object- Parameters:
risk_breakpoints (
DataFrame)cluster_cause (
DataFrame)cause_log_breakpoints (
DataFrame)cluster_summary (
DataFrame)clusters (
DataFrame)cluster_risk_baseline (
DataFrame)
- risk_breakpoints¶
- cluster_cause¶
- cause_log_breakpoints¶
- cluster_summary¶
- clusters¶
- cluster_risk_baseline¶
- __init__(risk_breakpoints, cluster_cause, cause_log_breakpoints, cluster_summary, clusters, cluster_risk_baseline)¶
- Parameters:
risk_breakpoints (
DataFrame)cluster_cause (
DataFrame)cause_log_breakpoints (
DataFrame)cluster_summary (
DataFrame)clusters (
DataFrame)cluster_risk_baseline (
DataFrame)
- class workflow.scripts.plotting.plot_health_impacts.HealthResults(cause_costs, risk_costs, intake, cluster_population)[source]¶
Bases:
object- Parameters:
- cause_costs¶
- risk_costs¶
- intake¶
- cluster_population¶
- workflow.scripts.plotting.plot_health_impacts.compute_health_results(n, inputs, risk_factors, value_per_yll, tmrel_g_per_day, food_groups_df)[source]¶
Compute health costs from optimized network, relative to TMREL intake levels.
- Parameters:
- Return type:
- workflow.scripts.plotting.plot_health_impacts.compute_baseline_risk_costs(n, inputs, risk_factors, value_per_yll, tmrel_g_per_day, food_groups_df)[source]¶
Compute baseline health costs by risk factor and by cause, relative to TMREL intake levels.
Health costs represent the monetized burden from deviations from optimal (TMREL) intake. Both total costs and individual risk factor contributions are measured relative to TMREL.
- Return type:
tuple[DataFrame,DataFrame]- Returns:
(risk_costs_df, cause_costs_df) where risk_costs has columns (cluster, risk_factor, cost) and cause_costs has columns (cluster, cause, cost, log_total, rr_total, coeff).
- Parameters:
- workflow.scripts.plotting.plot_health_impacts.build_cluster_risk_tables(risk_costs_df, cluster_population)[source]¶
- workflow.scripts.plotting.plot_health_impacts.compute_total_health_costs_per_capita(cause_costs, cluster_population)[source]¶
Compute total health costs per capita for each cluster.
The total cost per cluster is the sum across causes (health costs are additive across different health outcomes).
- workflow.scripts.plotting.plot_health_impacts.plot_health_map(gdf, cluster_lookup, per_capita_by_risk, output_path, top_risks, *, diverging=True, value_label='Health cost per capita (bnUSD)', total_per_capita=None)[source]¶
- workflow.scripts.plotting.plot_health_impacts.build_health_region_table(gdf, cluster_lookup, cost_by_risk, per_capita_by_risk)[source]¶
Plot global food consumption by food group per person per day.