Energy and mass-balance modelling of glaciers is a key tool for climate impact
studies of future glacier behaviour. By incorporating many of the physical
processes responsible for surface accumulation and ablation, they offer more
insight than simpler statistical models and are believed to suffer less from
problems of stationarity when applied under changing climate conditions.
However, this view is challenged by the widespread use of parameterizations
for some physical processes which introduces a statistical calibration step.
We argue that the reported uncertainty in modelled mass balance (and
associated energy flux components) are likely to be understated in modelling
studies that do not use spatio-temporal cross-validation and use a single
performance measure for model optimization. To demonstrate the importance of
these principles, we present a rigorous sensitivity and uncertainty
assessment workflow applied to a modelling study of two glaciers in the
European Alps, extending classical best guess approaches. The procedure
begins with a reduction of the model parameter space using a global
sensitivity assessment that identifies the parameters to which the model
responds most sensitively. We find that the model sensitivity to individual
parameters varies considerably in space and time, indicating that a single
stated model sensitivity value is unlikely to be realistic. The model is most
sensitive to parameters related to snow albedo and vertical gradients of the
meteorological forcing data. We then apply a Monte Carlo multi-objective
optimization based on three performance measures: model bias and mean
absolute deviation in the upper and lower glacier parts, with glaciological
mass balance data measured at individual stake locations used as reference.
This procedure generates an ensemble of optimal parameter solutions which are
equally valid. The range of parameters associated with these ensemble members
are used to estimate the cross-validated uncertainty of the model output and
computed energy components. The parameter values for the optimal solutions
vary widely, and considering longer calibration periods does not
systematically result in better constrained parameter choices. The resulting
mass balance uncertainties reach up to 1300 kg m

Surface energy and mass balance models are valuable tools for estimating the
response of glaciers to meteorological forcing

All glacier surface mass and energy balance models contain a degree of parameterization of physical relationships. These parameters are either optimized to fit observations, or chosen based on previously established empirical relationships, or are a mix thereof. Uncertainty surrounding the transferability of parameterizations in both space and time poses a critical limitation on the usefulness of such models for regional upscaling of glacier behaviour or forward projections of global glacier behaviour under changing climate conditions.

Early energy balance studies typically apply models at a single point in
space for which local physical relations can be readily established
empirically, or direct measurements are available to tune the
parameterizations

In studies of spatially distributed glacier mass balance

A way forward may be found in multi-objective optimization of glacier energy
balance modelling, first applied in a glaciological context by

The sequential approach used in this study can be classified in three steps. First, data management and model setup in beige, the simulations (blue) first use a global sensitivity analysis to reduce the parameter space followed by a multi-objective optimization. All simulations are performed independently for three summers on two glaciers. The data analysis (green) is done independently for sensitivity, parameter and model uncertainty analyses.

Mass balance models are required to be transferable in space and time in
order to estimate run-off on a larger scale or the impact of a changing
climate

It can be expected that models with more parameters have greater variation in
their solutions. Reduction of free parameters for optimization based on a
sensitivity analysis is therefore a helpful tool to reduce both the effect of
parameter correlation and computational expense

Model simulations are performed at the stake locations shown as points; points marked in black are only used in the optimization, while green points indicate the seven stakes on each glacier that were also used in the sensitivity analysis. Detailed maps are available in the Supplement (Figs. S1–S2).

Model sensitivity and model uncertainty are often evaluated together, and
assessments of varying robustness have been presented in the literature. For
example,

In this study, we present a model calibration and uncertainty assessment
workflow built upon a combination of these ideas. Our aim is to bring
awareness that uncertainty estimates of physically-based models with many
free parameters are likely to be under-estimated when applied in different
settings (geographical and or temporal) than those for which the model was
calibrated. Using an established distributed energy and mass balance model

In this paper, we will use the term “model uncertainty” to describe the difference between any modelled quantity and its counterpart in reality (“the truth”). An uncertainty value is a measure of how much trust can be given to a modelled quantity: in practice, model uncertainty can be estimated based on observations, and in any modelling activity which includes parameter calibration model uncertainty must be estimated separately from the calibration procedure (cross-validation). For quantities without equivalent in reality (e.g. model parameters), we use the term “uncertainty” to refer to the fact that their true value is really unknown, and that this uncertainty in the parameters is also conveyed in the model uncertainty. When we speak from “model sensitivity”, we mean the variance of the model output as function of the variance of an input quantity (e.g. forcing data, model parameters). A model sensitivity analysis does not require observations. In our paper, we restrict our sensitivity analysis to the internal model parameters, not to the input meteorological variables.

Two glaciers in the eastern European Alps were selected as test sites in this
study (Fig.

At HEF the AWS is located on a small plateau within a rock slope north of the
upper tongue area of the glacier at an altitude of 3025 m a.s.l. The
horizontal distance of this AWS from the glacier is about 300 m and it
provides all meteorological data required for the model except for
precipitation. Precipitation data were taken from the gauge operated by the
Bavarian Academy of Sciences at Vernagtbrücke, 3.5 km east of HEF at an
elevation of 2600 m a.s.l., and were scaled to the elevation of the AWS on
the basis of precipitation gradients derived from 11 totalizing rain gauges
in the vicinity of the glacier

The energy and mass balance model used in this study is a process-based model
that has been applied in a range of glacier environments

Variance-based sensitivity testing methods work in a probabilistic framework
judging sensitivity by relative variances of model input and output

The influence of an individual parameter can be examined by the main effect
(

The sensitivity indices for the simple model

The ranges for the 22 different parameters used in the sensitivity
study. Most parameterizations are explained in the
Appendix

The estimation of the sensitivity indices follows the algorithm from

The parameter sensitivity results from the GSA are also used as a tool to
reduce the number of free parameters in the model by identifying those
parameters which have only a marginal influence on the model output

A multi-objective optimization allows for more than one optimal solution in
the calibration procedure, and offers a way to assess a range of plausible
parameter sets that we will use later on for model predictions. The
multi-objective optimization used here follows previous approaches in
hydrology and glaciology

The figure displays a 2-dimensional Pareto space which comprises a
2-dimensional Pareto front. The solutions on this front (black solid line)
are referred to as the non-dominated set of solutions. In comparison, all
other solutions within the solution space are inferior in at least one
objective relative to the Pareto front. Classic single objective
optimization yields the points

In this study the multi-objective optimization is based on a Monte Carlo
simulation. The non-sensitive parameters from the GSA were fixed to their
median value from the range used in the GSA
(Table

The amount of sensitive stakes per year for

The focus of this GSA is not on the absolute sensitivity towards single
parameters, but rather to reduce the dimension of the parameter space.
Therefore, the following discussion is limited to two classes: parameters to
which the model is sensitive (

At Hintereisferner, 11 out of 22 parameters are identified as sensitive
(Fig.

The sensitivities show spatial and temporal variability, which can be
explained by the varying mass balance conditions of the respective year (mean
specific summer/annual mass balance with 2012:

On the smaller Langenferner, 6 of the 22 parameters were identified as
sensitive (Fig.

Five objective functions are used to analyse the model performance.
The minimum value for every function and each year are given in kg m

First we consider the best model performance with respect to each individual
objective function tested (Table

In all cases a model simulation with very low bias (

The multi-objective optimization, using BIAS, MAD

Each individual member of the Pareto set for HEF in 2012 is
displayed with a different colour and the compromise solution highlighted (red
triangle or red line). The different panels are the 2-dimensional projections
of the Pareto space onto the

The parameter values of those optimal solutions span the entire allowed space
apart for some of those relating to snow albedo which span (almost) the whole
parameter space in all years for both glaciers, and show no obvious
tendencies towards a certain albedo range (Sect. 3). For HEF in 2012, snow
albedo values cluster in the higher range (0.52–0.6) for firn and
(0.86–0.9) for fresh snow (Fig.

To investigate the transferability of the optimized mass balance model
settings, all the optimal solutions of the Pareto set of one glacier summer
mass balance case were applied to the five other summer and glacier cases.
While each Pareto set was identified based on the multi-objective
optimization, the transferability study uses only the Euclidean distance
towards the utopian point as a quantification tool. The individual optimal
parameter settings for HEF 2012 for example yield quite varying performances
for the other summers (Fig. S4a). While the performance on the same glacier
(HEF) is reasonably good for 2012 (200–800 kg m

The cross-validation (Fig.

The energy balance components for 8 out of 18 selected stake locations
close to the central flow line, displayed in different colours for HEF 2012.
Solid bars represent the fluxes of short-wave radiation (SW), long-wave
radiation (LW), turbulent heat fluxes (

Analysis of the energy balance components associated with Pareto set
solutions offers a qualitative means of verifying that the identified optimal
parameter settings are in line with expected physical processes at the
glacier surface. The energy balance components calculated by the model are
expected to vary depending on the parameter settings of an optimal ensemble,
which have been demonstrated to span almost the whole parameter space. This
variation in energy balance components is indicative of the uncertainty in
the modelled energy fluxes (we say “indicative”, because the true
uncertainty can only be assessed using observations, which are not available
here). Figure

The variation of the averaged energy components over the stakes for HEF 2012
are given in Fig.

The energy balance components averaged over all stakes has less uncertainty than on the point scale for HEF 2012. The objective functions are all integrated over the whole glacier and thus the uncertainty is lower. Glacier wide the short-wave radiation is the largest component with the largest absolute uncertainty as well, followed by the turbulent fluxes. The long-wave balance and the penetrating short-wave radiation provide a net cooling effect for the surface.

The energy balance components are averaged over all stake locations.
The uncertainty is given in respect to the minimum and maximum of the
ensemble. The short-wave radiation (SW

The total contribution of the energy balance components averaged over the
glacier are listed in
Table

The largest uncertainties in our study are associated with the short-wave
radiation as a result of the albedo parameterization, which relies on five
model parameters. Alternative albedo parameterizations are also known to be a
source of substantial uncertainty

The long-wave radiation shows a lower uncertainty in this study than in

The turbulent fluxes are associated with the second largest uncertainty in
this study, which is in agreement with other studies that found larger
uncertainties in the radiative forcing

Heat supply from rain is negligible in our study, which is in agreement with
other studies on alpine glaciers

The larger glacier, Hintereisferner, has more sensitive parameters and the variation over the stakes is larger than at Langenferner, as a result of more distinct climate regions on the longer tongue of the larger glacier. This is also true for the uncertainty of energy balance components, with the exception of the net solar radiation, which is comparable on both glaciers. Short-wave radiation is the most uncertain of the energy balance components, due to the albedo parameterization, which accounts for the change in albedo over time, but does not account for any possible spatial variation in temperature or grainsize-dependent albedo decay rates. We have shown that the model has difficulties optimizing the upper and lower part of the glacier simultaneously, as a result of the variable parameter values of physical quantities like albedo. The large spread of our ensemble is a result of trade-off solutions between the real albedo at any time and any location and the temporally and spatially averaged parameterization applied. Other parameterizations that are assumed as constant in space and/or time, or only indirectly affected by temperature and altitude dependencies, are also subject to similar trade-off effects. Although the physical relations may not be the same at all times and the lower tongue area may be quite different from the upper glacier, this does not mean that the model performance is worse on the larger glacier (HEF) with more variation in a quantitative matter (Table 3), but rather that the solutions of the Pareto front show more variation in the parameter settings. This analysis clearly identifies the issue of governing parameters and parameterizations not being constant in space and time as the main problem of distributed energy balance modelling. The most readily appreciable example in this regard is ice albedo, which is often lower near the terminus due to debris and dust accumulation and water saturation of the glacier surface.

To improve this we suggest two potential approaches: (1) although optimizing all key parameters serves a purpose for a broad
range of applications, fixing low sensitivity parameters to common values, which are not optimized, results
in a type of a simplification of the model that reduces over-fitting and
potentially increases the stability and comparability of the energy balance
model over short-timescales. The overall performance of such a model will be
lower because the tuning possibilities have been restricted, but better
estimates of the model uncertainties for out-of-sample periods can be
generated. (2) Parameters or parameterizations could be allowed to vary in
space and/or time. This could be achieved either by increasing the
measurements and data availability or increasing the model complexity. More
complex albedo schemes are available, for example, for snowpack models like
Crocus

The approaches in this study are helpful tools to combine these suggestions.
A clear understanding of the model sensitivity, independent of the
optimization of the model is necessary to decide on the importance of certain
parameters. It gives the option of fixing parameters and focusing on the key
processes. We have shown that multi-objective optimization is a valid tool to
assess uncertainties in the model. The objectives used are all based on the
same data (i.e. stake data). This allowed us to show the uncertainty that is
just associated with treating the available data in a different way without
requiring additional measurements. The model can readily be optimized to
minimise bias or meet any single value objective; therefore, model
performance based on single best-fit approaches should be treated with
caution. Furthermore, a single solution may suffer significantly from
parameter over-fitting and is not representative in its parameter settings
for other plausible solutions. The
chosen objectives show that there is inter-annual variation in the
performances of the upper and lower section of the glacier in our cases. The
curved nature of the Pareto front highlights that simultaneous optimization
of both areas is difficult for the model. Parameters are just not constant in
either space or time, so the model uncertainty increases when the model is
applied to other time periods or on another glacier. The model uncertainty is
in the range of 1000 kg m

Neither meteorological forcing on the point scale nor mass balance
measurements are free of errors, and the related model uncertainties were not
formally disentangled from other uncertainties in this study.

The analysis presented here indicates that while mass and energy balance models help us to understand the physical processes on the glacier, the necessity for parameterizations within these models introduces considerable, variable uncertainty to the model output. Calibration of surface mass balance models is complex and uncertainty studies are helpful to understand those models, and it is not advisable to draw general conclusions from such modelling efforts without first fully understanding the inherent model sensitivity and the properties of the uncertainty of the calculated mass balance and associated energy fluxes in detail.

Based on a well developed mass and energy balance model, applied to two well-studied glaciers in the European Alps, this study gives a robust estimate of the model uncertainty and discusses the advantages of parameter space reduction and multi-objective optimization in glaciological modelling.

Using a variance-based global sensitivity method, model sensitivity to the free model parameters was identified, independent of the calibration data. Model sensitivity to specific parameters is both site- and time- specific, and this should be acknowledged in wider applications of such models. By separating the parameters into two sensitivity categories, the model parameters to be optimized can be reduced. Those that the model output is sensitive to were subject to a multi-objective optimization, while non-sensitive parameters were fixed to literature values.

The multi-objective optimization was based on three objectives related to
stake mass balance data measured using the glaciological method. We used the
model bias over all stakes and the mean absolute deviation over the upper and
lower part of the glaciers. It proved difficult to optimize model performance
in the upper and lower section of the glacier simultaneously. The bias over
all stakes, which was used as a proxy for the cumulative mass balance, can be
minimized easily, and this should be considered when optimizing for a single
best fit against single values. The ensemble of optimal solutions shows a
wide spread of parameter settings within the physically reasonable range.
This implies that the common approach of a single best optimized parameter
set is subject to over-fitting and may significantly differ from other
equally plausible solutions, meaning that they are not representative by
default. Furthermore, our results show that the constraint of plausible
parameters is only marginally linked to the sensitivity, with very sensitive
parameters also taking multiple optimal values. This implies that keeping
these parameters constant in space and time increases the model uncertainty.
The overall model uncertainty (not accounting for uncertainties related to
meteorological forcing data) is in the range of 1000 kg m

Parameter uncertainty is connected with uncertainty in the energy balance
components, which, in the cases studied here, reached 30 % averaged over
the glacier and 50 % at individual stake locations. In our study the most
uncertain energy balance components are the net short-wave radiation and the
turbulent fluxes, reasserting the findings of other studies

Overall, the findings of this study highlight that understanding the sensitivity and uncertainty of surface energy and mass balance models is complex, and simplistic assessments, in particular single best guess approaches, of model performance are likely to overstate model capabilities. Further studies such as this, incorporating more models, glaciers and years would help to constrain the degree to which results from such models can be considered reliable for regional applications and for projections of glacier mass balance.

The code of the mass balance model can be requested from Thomas Mölg (thomas.moelg@fau.de). Pareto construction scripts and the updated solar module can be requested directly from Tobias Zolles (tobias.zolles@uib.no).

The mass balance and meteorological data used in this paper are available at
Zenodo;

The mass and energy balance model used here consists of coupled surface and
subsurface components. The model computes mass balance as the sum of solid
precipitation, surface deposition, internal accumulation (refreezing of
liquid water in snow), change in englacial liquid water storage, subsurface
and surface melt and sublimation. This approach is based on the surface
energy balance of a glacier in the following form:

The calculation of the incoming long-wave radiation is based on
Stefan–Boltzmann law

The outgoing long-wave radiation follows the Stefan–Boltzmann law as in
Eq. (

The latent heat flux (

The albedo parameterization is based on

The conductive heat flux (

The calculation of the penetration of short-wave radiation is based on

The surface accumulation is directly related to the precipitation. The model
has two threshold values for all liquid and all solid precipitation

The parameterization of the short-wave radiation is based on the calculation
of the cloudiness, in the form of the effective cloud cover fraction

The calculation of the incoming short-wave radiation on every point of the
glacier is based on the assumption of homogeneous cloudiness
(

The calculation of solar radiation incorporates the free parameter

Furthermore, the change in the calculation of direct and diffuse components
from linear with cloudiness to a linear increase of the fraction are better
suited to represent the site radiation. This is in agreement with the
measured radiation by

The supplement related to this article is available online at:

TZ conducted the simulations and the data analysis and wrote the main part of the manuscript. FM was involved in defining the study and contributed to the statistical analysis. WG was involved in the model set-up and the adaptation of the solar module. SG was responsible for the mass balance measurements and data acquisition. LN contributed to the paper design and writing. All authors contributed to finalizing the paper.

The authors declare that they have no conflict of interest.

The computational results presented have been achieved (in part) using the HPC infrastructure LEO of the University of Innsbruck. We thank Rainer Prinz for supplying the snow, density and DEM grids for the energy balance model. We thank the Hydrographisches Amt Bozen and the Hydrographischer Dienst Tirol for funding the mass balance measurements. Lindsey Nicholson was supported by the Austrian Science Fund Grant number V309. Edited by: Tobias Sauter Reviewed by: Matthieu Lafaysse and one anonymous referee