The Observation Range Adjusted Method: A Novel Approach to Accounting for Observation Uncertainty in Model Evaluation

The Observation Range Adjusted Method: A Novel Approach to Accounting for Observation Uncertainty in Model Evaluation

June 27, 2024

Model evaluations are performed by comparing a modelled quantity with an observation of that quantity and any deviation from this observed quantity is considered an error. We know that all observing systems have uncertainties, and multiple observational products for the same quantity can provide equally plausible “truths”. Thus, model errors depend on the choice of observation used in the evaluation exercise. We propose a method that considers models to have no error when they lie within the range of observations, and hence are indistinguishable from observations. Errors occur when models are outside the observational range. Errors calculated in this way can be used within traditional statistics to calculate the Observation Range Adjusted (ORA) version of that statistic. The ORA statistics highlight the measurable errors of models, provide more robust model performance rankings, and identify areas of the model where further model development is likely to lead to consistent model improvements.

Read more: Evans, J & Imran, H.. (2024). The Observation Range Adjusted Method: A Novel Approach to Accounting for Observation Uncertainty in Model Evaluation. Environmental Research Communications. 10.1088/2515-7620/ad5ad8. 

Back to Resources