This page provides a brief introduction to verification and validation (V&V) for scientific models – mostly to solidify the terminology we’ll be using. We highly recommend reading our open access publication in JAMES for a detailed discussion of V&V and its application in LIVVkit [Kennedy2017]. For a very in depth discussion of V&V for scientific computing, we recommend [Oberkampf2010].

Introduction to V&V

Verification and validation (V&V) is a set of techniques that are used to build and quantify confidence in something. That something can be a numerical model or a piece of software, and building confidence in each of these will require the use of specialized techniques.

Model V&V

Numerical models are typically used by scientists to make predictions about the physical world. V&V of a numerical model consists of:

Numerical verification

The process of comparing the approximate numerical prediction against an analytical prediction, or in the case of a sufficiently complex problem, against highly accurate community benchmark solutions. This is inherently a math problem [Oberkampf2010] [Roache1998] that is represented by the question, “Are we solving the equations correctly?”

Physical validation

The process of comparing the numerical predictions against natural phenomena. This is inherently a physics problem [Oberkampf2010] [Roache1998] that is represented by the question, “Are we using the right physics?”

Software V&V

Software, on the other hand, is a tool made by a set of developers which is to be used for a set of specific tasks by a user. V&V of a piece of software consists of:

Code verification

The process of determining the software’s implementation, and it’s associated data, accurately represent the developers’ specifications. This is inherently an engineering problem [Oberkampf2010] that is represented by the question, “did we build what we wanted?”

Performance validation

The process of determining how well the software is able to be used for its intended task. This is inherently an design problem [Oberkampf2010] that is represented by the question, “did we build what the users wanted?”

Scientific model V&V

Any scientific model (e.g., an ice sheet model) is a blend of the underlying numerical models used to make the prediction, and the software that is used to drive the numerical models. Therefore, validation and verification of any scientific model requires the use of both (numerical) model V&V and software V&V. These two types of V&V techniques should be viewed as a complimentary process, as the numerical and software code are intrinsically linked and live within the same code-base.

On confidence and credibility

Confidence is (or at least should be) needed for users and developers to trust their modeling and simulation (M&S) results, and to transmit them to the wider scientific community, policy/decision makers, and other stakeholders.

However, for those results to be used people outside the core user and development community, the model must have an adequate level of credibility – it much be trusted outside of the core community. This is especially important when the results or predictions are used to inform policies that may affect affect the economic and general well-being of citizens.

Importantly, confidence within the core community is not enough. Credibility relies on the M&S results being reproducible, the model being well-described and thoroughly tested, and all the information relating to the development and testing of the model being transparent and easily discoverable [Kennedy2017].