Changelog¶

Unreleased¶

New Features¶

group support in range_probability — the stat now honours the group field (month, season, year) like the other ops, computing probabilities within each group and returning a group dimension. Previously group was accepted and silently discarded: the call returned the ungrouped all-record probability, but the pipeline still appended the group to the output variable name, so the result looked grouped and was not. Denominators are computed once per variable and shared across all ranges defined on it.
group support in pcount — the stat now honours the group field, reporting the percentage of valid samples within each group and returning a group dimension. The denominator is the number of timesteps belonging to the group, not the length of the whole record, so a variable with no gaps reads 100 in every group.

Bug Fixes¶

statdir with more than one function raised MergeError — every stat names its output after the variable it reduced, so funcs: [mean, max] produced two datasets that both contained hs and the merge of the two sector results failed with conflicting values for variable 'hs'. The stat was therefore usable only with the single function it exists to avoid. Delegated outputs are now suffixed with the function that produced them (hs_mean, hs_max), including when only one function is requested, so a name never depends on the length of funcs. The pipeline no longer appends _statdir on top of a name that already carries the name of the delegated stat, which would have given hs_mean_statdir_direc; the written names are hs_mean_direc and hs_max_direc — identical to those of the equivalent per-function calls with nsector set, and they resolve through attributes.yml like any other output. Single-function statdir output previously written as hs_statdir will be written as hs_mean_direc; when appending to an existing store the old variable is not removed automatically.
nsector on a statdir call binned the data twice — nsector is a pipeline-level field, so it was consumed by the pipeline's own sectoriser and never reached the stat, which then applied its own default of 4 sectors inside each of the outer ones. The documented eight-sector statdir example failed with conflicting sizes for dimension 'direction'. The sector settings are now forwarded to stats that sectorise internally instead of being applied around them, so nsector: 8 yields the eight sectors asked for.
statdir accepts group again — it forwarded group to its delegated stats through **kwargs but declared no group parameter of its own, so the new signature-based guard classed it as unable to group and rejected the call. group is now an explicit parameter passed through to every delegated stat, and a call naming a stat that cannot honour it (funcs: [mean, rpv] with group: month) fails up front, naming the offender, rather than silently returning an ungrouped result for it.
output.chunks was silently ignored — the pipeline read the output chunking as self.config.output.__dict__.get("chunks", {}), but OutputConfig declared no chunks field, and Pydantic had already dropped the key while validating the config. A user who set chunks: under output: got the default chunking of the finalised dataset — typically a single chunk per variable — with no warning, however large the grid. chunks is now a declared dict[str, int] field, read by normal attribute access, and reaches finalise, which applies it to the dimensions present in the output and clears the stale chunk encoding inherited from the source.
Grouped variables were written with no attributes at all — the pipeline appends the group to the output variable name, so a group: month call on mean writes hs_mean_month. Attribute resolution split that name on _ and read the last token as the statistic, found month in neither the stats nor the data_vars map, and gave up: the variable was delivered with no units, no standard_name and no long_name, while its ungrouped sibling hs_mean was fully described. The group suffix — and the _direc suffix a directionally sectorised call appends after it — are now stripped before the statistic and the parent variable are looked up, so hs_mean_month and hs_mean_month_direc both resolve through hs_mean and carry units: m. A grouped variable additionally has its long_name prefixed with the matching adjective, taken from an explicit mapping rather than built by string arithmetic (month → monthly, season → seasonal, year → annual; the previous, unreachable attempt at this would have produced "seasonly"). Only a trailing group token counts as a group, so a parent variable whose own name contains a group word is not mangled. The override that keeps a probability or a percentage dimensionless still applies on top, so hs_gt_1p5_range_probability_month remains units: "1". Values are unchanged, but grouped fields already written carry no metadata and will not gain any until they are rewritten or re-consolidated.
count no longer inherits the units of its parent variable — hs_count was written with units: m although it is a number of samples. count is now declared in the stat_units section of attributes.yml as "1", the CF/UDUNITS spelling of a dimensionless quantity, alongside dist, whose joint histograms are raw counts and had exactly the same problem. Values are unchanged; only metadata is affected.
Grouped pcount output was the ungrouped value — pcount accepted group and silently discarded it, yet the pipeline still appended the group to the output variable name, so a group: month config wrote the whole-record coverage under a name like hs_pcount_month, with no month dimension. Now that the stat honours group the name matches the data. Any grouped pcount field produced before this change is wrong and should be regenerated.
Requesting group on a stat that cannot honour it is now an error — rpv, hmo and distribution3_timestep absorbed group into **kwargs and ignored it while the pipeline appended it to the output variable name, producing an ungrouped result wearing a grouped name. Such a call now raises before any data is loaded, naming the stat and listing the stats that do support grouping. Support is determined by inspecting the stat's signature for an explicit group parameter, so it cannot drift out of sync with the code. This is a behaviour change callers may notice: a config that previously ran to completion with a meaningless group on one of these stats will now fail, and the fix is to drop the group field from that call. Grouping is deliberately not implemented for the three — a per-group extreme-value fit uses a fraction of the samples and is not statistically defensible, and grouping is not well defined for a Welch spectrum or for a histogram accumulated over windows that do not align with time groups.
rpv timestep derivation no longer assumes nanosecond time — the sampling interval was computed as float(np.diff(time)[0]) / 3.6e12, which raises TypeError under NumPy 2 for any datetime64 resolution other than nanoseconds (and would silently mis-scale if it did not). It now goes through pd.Timedelta, matching the fix already applied in exceedance.
uorb no longer returns a value outside the water column — the evaluation point is a height above the seabed, z_bed, and with reference: surface it is derived as depth - z, so z: 15 in 3 m of water asked for a point 12 m below the bed. The transfer function cosh(k z_bed)/sinh(k h) is even in z_bed, so that point silently returned the value for its mirror image 12 m above the bed — well clear of the surface and larger than the true surface value (Hs = 2 m, Tm02 = 9 s in 5 m of water gave 2.04 m/s against a true surface value of 1.46 m/s). The same applied to reference: bed with z greater than the depth. Points with z_bed < 0 or z_bed > h are now NaN, tested per timestep and per cell so the mask follows the bathymetry; the seabed (z_bed = 0) and the still-water surface (z_bed = h) remain valid. Values inside the water column are unchanged. Any uorb_sfc_* field already computed at a fixed depth below the surface is affected wherever the water is shallower than that depth and should be regenerated.
Statistics no longer inherit the units of their parent variable — hs_pcount was written with units: m although it is a percentage, and hs_gt_1p5_range_probability carried units: m plus the standard_name and long_name of a wave height although it is a dimensionless fraction in [0, 1]. attributes.yml gains a stat_units section declaring the units of statistics that do not preserve those of their parent (pcount: "%", and "1" for probability, exceedance and nonexceedance), merged from the metadata config like the other sections. The key is matched against any _-separated token of the output variable name, so grouped names such as hs_gt_1p5_range_probability_month resolve too, and names inherited from the parent are dropped rather than left describing the wrong quantity. pcount and range_probability now also set correct attributes at the source, the latter with a long_name spelling out the range (probability of hs >= 1.5 m). Values are unchanged; only metadata is affected.
Durational exceedance is now actually grouped — exceedance/nonexceedance with both duration and group set previously computed the ungrouped result and broadcast it across the group dimension, giving every month the same value. Runs are now identified on the continuous record and then grouped, so events are neither truncated at group boundaries nor spliced across the gaps a grouper introduces (a month grouper places every January back to back).
Runs touching the ends of the record are no longer missed — run detection used scipy.signal.find_peaks, which only reports plateaus bounded by lower values on both sides, so a qualifying spell at the very start or end of the record was dropped (a record entirely above the threshold returned a probability of 0.0). Detection is now based on run boundaries and includes them. Durational exceedance values may increase slightly as a result.
Timestep derivation no longer assumes nanosecond time — the sampling interval was computed as float(np.diff(time)[0]) / 3.6e12, which raises TypeError under NumPy 2 for any datetime64 resolution other than nanoseconds (and would silently mis-scale if it did not). It now goes through pd.Timedelta.

Breaking Changes¶

Unknown fields under output are now rejected — OutputConfig inherited Pydantic's default of dropping keys it does not know, which is how output.chunks came to be written in configs, accepted by validation and never applied. Any misspelling had the same failure mode: chunk: for chunks:, or consolidated: for consolidate:, left the option unset while the run reported success. The model now forbids extras, so such a config fails validation naming the offending key. A config carrying a stray key under output that used to run will now fail, and the fix is to correct or remove the key — the setting it was meant to express was never in force.
Fractional exceedance thresholds are named with p — exceedance/nonexceedance output variables now write the decimal point as p, so a threshold of 1.5 gives hs_1p5 rather than hs_1.5, keeping the name usable as an identifier and by attribute access. Integral thresholds are unchanged (2.0 still gives hs_2). Configs using fractional thresholds will write new variable names; when appending to an existing store the old dotted variables are not removed automatically.
Default range_probability labels are named with p — a data_ranges entry with no label built its output name from the raw bounds, so a fractional bound produced hs_1.5_to_max, a name that is not a valid identifier and is awkward in path-like contexts. The decimal point is now written as p (hs_1p5_to_max), matching exceedance; the formatting is a single helper shared by both ops so the two cannot drift apart. Integral bounds are unchanged (hs_1_to_2), and entries that set label explicitly are unaffected. Configs relying on the default label with a fractional bound will write new variable names; when appending to an existing store the old dotted variables are not removed automatically.

2.3.0 (2026-06-26)¶

New Features¶

updir output upload — the updir field on the output config is now implemented. After writing, the output (Zarr store or NetCDF file) is uploaded to the given remote directory under its basename via fsspec, so it works for gs://, s3://, and local targets (e.g. outfile: ./scratch/hs.zarr + updir: gs://bucket/stats → gs://bucket/stats/hs.zarr). Useful for deployments that compute to fast local disk and then publish the result. Skipped when outfile is already remote.
Inline config via $CONFIG — gridstats run can now read its YAML config from the $CONFIG environment variable instead of a path argument, for deployment tools that inject the config inline (e.g. Argo on k8s). The path argument takes precedence when both are present.
hmo sampling frequency control — the hmo stat accepts an explicit fs (Hz) parameter to override the sampling frequency. When omitted, fs is now inferred from the mean sampling interval over the whole record (rounded to 3 decimal places) rather than the first interval alone, making it robust to small timing jitter in model output whose nominal cadence wanders by a few percent. Numeric (seconds) and datetime64 time coordinates are both supported.

2.2.0 (2026-05-04)¶

New Features¶

mode stat — new generic histogram-based mode stat. Returns the centre of the most-occupied bin along the reduction dimension. Accepts explicit bins edges and an optional weight_var for energy- or Hs-weighted modes. Supports the same group parameter as all other aggregations. Particularly suited for ordinal data such as the Douglas Sea Scale — pass half-integer edges (e.g. [-0.5, 0.5, ..., 9.5]) to get one bin per integer degree. Dask-compatible via apply_ufunc with vectorize=True.
Output masking — new mask: block on the output config applies a spatial mask to all output variables before writing. Two types are supported: notnull (keep points where a source variable is non-null) and threshold (keep points satisfying a numerical condition). Both accept an optional isel: dict to reduce the source variable to a 2-D slice before the mask is computed, and xarray broadcasting ensures the mask is applied correctly across any extra dimensions (time, quantile, direction).

Documentation¶

Douglas scales — corrected scale descriptions in the douglas_sea docs table (e.g. "Rippled" → "Calm (rippled)", "Wavelets" → "Smooth") and added a full degree/description/range table to the douglas_swell docs, including a note on degree 9.
Crossing seas — rewrote the crossing_seas docs section: added a direction convention warning (all inputs must use the same coming-from or going-to convention), guidance on threshold choices (Li 2016 defaults and operational variants), and the full Li (2016) reference with DOI.

Bug Fixes¶

Douglas sea scale — fixed an off-by-one error where all degrees were shifted one lower than the correct Douglas classification (e.g. a 3 m sea was reported as degree 4 "moderate" instead of degree 5 "rough"). Degree 9 (phenomenal, Hs > 14 m) was also never assigned.
Crossing seas — changed hs_threshold default from 0.0 to 0.5 m. A zero threshold allowed the function to flag crossing seas in near-calm conditions where partition directions are undefined and noisy. The Li (2016) criterion is meaningful only when both systems are energetically significant; 0.5 m is the standard operational floor.

Improvements¶

Douglas sea scale — replaced the 9-iteration xr.where loop with a single xr.apply_ufunc(np.digitize, …) call. Produces a flat dask graph (one task per chunk instead of nine) and runs ~1.5× faster on eager arrays; gains are larger under dask where the stacked task graph was a scheduling bottleneck.
Douglas swell scale — DOUGLAS_SWELL_BINS replaced with DOUGLAS_SWELL_INTERVALS, a degree-keyed dict of pd.Interval pairs matching the original onstats definition. Degree 9 ("confused") is documented as a crossing-seas condition that cannot be assigned from height/wavelength thresholds alone; it was also unreachable in the original (pd.Interval(inf, inf) is never satisfied by any finite value).
Douglas swell scale — replaced the 9-iteration xr.where loop with xr.apply_ufunc(_classify_swell) backed by two np.digitize calls and a 3×3 LUT. Eager compute cost is unchanged (2D classification has similar work to the loop), but the dask task graph shrinks ~7× (54 → 8 tasks per chunk), reducing graph construction and scheduler overhead on large multi-chunk hindcasts.

2.1.0 (2026-05-01)¶

New Features¶

Derived variables — new top-level gridstats.derived subpackage computes secondary variables (wind speed/direction, current speed/direction, wave parameters, wave orbital velocity, sky conditions) directly from source data before stats are applied.
Modal direction — new modal_direction stat implemented as a vectorised ufunc.
Zarr append / consolidate — new append and consolidate output options enable parallel Zarr writes across tiled or distributed runs.
Custom dataset metadata — new global_attrs field on the output config block lets you inject arbitrary CF-compliant global attributes into the output dataset.

Bug Fixes¶

Derived variables were silently dropped when data_vars was set on a call; this is now fixed so derived variables survive variable selection.

Improvements¶

Info-level logging added for loader calls and derived variable definitions, making it easier to trace pipeline execution.
Docs restructured: configuration, loaders, and derived variables each have their own section with per-topic pages; MathJax support added for equation rendering.

2.0.0 (2026-03-25)¶

A near-complete rewrite that renamed the library from onstats to gridstats and replaced the legacy architecture with a modern, well-typed stack.

Breaking Changes¶

Library renamed from onstats to gridstats; all imports must be updated.
Configuration schema rewritten using Pydantic v2 — existing YAML configs require migration (see the Configuration docs).
slice_dict replaced by explicit sel / isel fields on source config.
Source config now uses a discriminated union: set urlpath for xarray or catalog + dataset_id for intake — the old type: field is gone.
CLI commands changed: gridstats run config.yml and gridstats list-stats (powered by Typer instead of the old argparse CLI).

New Features¶

Pipeline orchestrator — new Pipeline class handles loading, stat dispatch, tiling, directional sectorisation, and output in a single object.
Registry with entry-point plugins — @register_stat / @register_loader decorators; third-party packages can contribute stats or loaders via the gridstats.stats / gridstats.loaders entry-point groups.
Spatial tiling — tiles: on any call keeps peak memory bounded for large grids.
flox integration — use_flox per-call toggle (default True); automatically disabled for quantile to avoid ~2× memory overhead.
open_kwargs on XarraySourceConfig — pass arbitrary kwargs to xarray.open_dataset.
CF attributes — gridstats/attributes.yml maps variable names to CF standard_name / long_name / units; applied automatically on output.
Packaging migrated to pyproject.toml; mkdocs documentation site published to GitHub Pages; GitHub Actions CI/CD added.

1.2.0 (2022-06-26)¶

New Features¶

CF attribute support — variable attributes (standard_name, long_name, units) can now be declared in config and are written to the output dataset.
Global attributes — pipeline-level global_attrs for dataset-level metadata.

Bug Fixes¶

Fixed crash with non-uniform chunk sizes during output writing.

1.1.0 (2022-05-17)¶

New Features¶

Probability stats — new module with exceedance and non-exceedance probability functions (pexc, pcount).
Allow transposing, fixing dtype, and setting chunks on output variables.
Dask cluster can now be enabled or disabled individually per call.
chunks can be specified when opening datasets from an intake catalog.

Bug Fixes¶

Fixed chunking when applying a land mask.
Fixed bug in distribution function output.

Internal Changes¶

Removed deprecated KMZ stats code.

1.0.0 (2021-11-25)¶

New Features¶

Return period values (RPV) — new rpv stats module supporting groupby time aggregations.
Distribution stats — histogram / empirical distribution functions.
Count function — count non-NaN observations.
Directional sectorisation — nsector / directional bin support for computing stats per direction sector.
Stepwise decorator — compute stats in time chunks to bound peak memory on large datasets.
Dask cluster — explicit cluster setup with configurable worker count (including all / half shortcuts).
Logging throughout the pipeline.
Refactored to a CLI entry point.

0.1.0 (2021-06-18)¶

First release on PyPI.