MOTIVATE Publications

Publications from the MOTIVATE project.

Model Validation

Model validation is based on comparison to experimental data from camera-based instruments.
Starting Point: CEN Workshop Agreement
A recent CEN guideline recommends comparison by using data reduction techniques such as image decomposition. In MOTIVATE this procedure is applied to an aircraft sub-component test in an industrial environment.
/documents/1583244/0/Validation_I-Beam/93009348-5834-4a06-a944-256853605885?t=1509014968800
Predicted (top left) & measured (bottom left) y-direction strain field in percentage strain in the web of an I-beam subject to three-point bending; corresponding plot (right) of normalised shape descriptors with a band of acceptability shown by the dashed lines (from Hack et al. J. Strain Analysis 51(2016)5 ).

We have developed novel methods for the comparison of predictive numerical models with full-field experimental data in order to achieve a robust, quantitative validation of the simulation. In particular an understanding has been gained of the uncertainties in simulation and experimental data and their influence on predictions and measurements.

Criteria allowing an easy comparison and interpretation of data have been embedded in novel correlation methods that allow confidence in simulations to be established, supported by the quantifcation of 'the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model', as the ASME definition of validation reads. Work at ULIV has demonstrated that feature vectors, obtained via image decomposition, can be used to generate relative error metrics. This preliminary work is being integrated with the prior work embedded in the CEN guideline to produce a validation methodology.

Validation flowchart

The MOTIVATE Validation Flowchart, reproduced below, forms the basis of the MOTIVATE Protocol which can be downloaded from Zenodo, together with a GUI that guides the user through the validation process step by step. A key novel feature is the evaluation of historical data.  In addition, the development of the model takes priority, while physical testing is performed only if required. The processes involved are shown as coloured boxes.  This includes the decision sequence required to evaluate historical data for use in the validation process and the quantitative validation process described in the recently published CEN guide. To allow for an unbiased comparison of the data sets, it is recommended to implement a “double blind” procedure, in which interaction of the testing and modelling teams are only allowed as far as to clarify and agree on test object, boundary conditions and load cases. Then, data are generated by experiment (blue box) and by simulation (orange box) independently, and the results are transferred to an independent validation team who is in charge of the Quantitative Comparison (red box).  It should be noted that, prior to starting the validation process, it will be necessary for decision-makers to state their expectations so that appropriate decision criteria can be adopted, for instance requirements for measurement uncertainty.  While this activity of the decision-makers was not within the brief of the MOTIVATE project, their review at the end of the process is shown for completeness and comparison with the existing ASME flowchart


/documents/1583244/0/Flowchart/2e9c2451-be6a-43c0-8dae-7dd04e2a9d5b?t=1599819961677
Proposed new flowchart for the process of validating simulations that allows the use of historical data. While the two strands for simulation and dedicated validation experiment known from the corresponding ASME flowchart make part of the graph, the proposed flowchart opens more options to reach a validation statement.

Blog posts

Slowly crossing the valley of death

The valley of death ... is the gap between discovery and application, or between realization of an idea in a laboratory and its implementation in the real-world. ... Our work on quantitative comparisons of data fields from physical measurements and computer predictions is about three-quarters of the way across the valley.  We published a paper in December (see Dvurecenska et al, 2020) on its application to a large panel from the fuselage of an aircraft based on work we completed as part of the MOTIVATE project. ...

This blog post appeared on January 27th, 2021, on realizeengineering.blog

The blind leading the blind

Three years after it started, the MOTIVATE project has come to an end. The focus of the project has been about improving the quality of validation for predictions of structural behaviour in aircraft using fewer, better physical tests.  We have developed an enhanced flowchart for model validation, a method for quantifying uncertainty in measurements of deformation in an industrial environment and a toolbox for quantifying the extent to which predictions from computational models represent measurements made in the real-world.  In the last phase of the project, we demonstrated all of these innovations on the fuselage nose section of an aircraft. ...

This blog post appeared on May 27th, 2020, on realizeengineering.blog
Alleviating industrial uncertainty

Want to know how to assess the quality of predictions of structural deformation from a computational model and how to diagnose the causes of differences between measurements and predictions?  The MOTIVATE project has the answers; that might seem like an over-assertive claim but read on and make your own judgment. ...

This blog post appeared on May 13th, 2020, on realizeengineering.blog

Fake facts & untrustworthy predictions

I need to confess to writing a misleading post some months ago entitled ‘In Einstein’s footprints?‘ on February 27th 2019, in which I promoted our 4th Workshop on the ‘Validation of Computational Mechanics Models‘ that we held last month at Guild Hall of Carpenters [Zunfthaus zur Zimmerleuten] in Zurich.  I implied that speakers at the workshop would be stepping in Einstein’s footprints when they presented their research at the workshop, because Einstein presented a paper at the same venue in 1910.  However, ...

This blog post appeared on December 4th, 2019, on realizeengineering.blog

Same problems in a different language

I spent a lot of time on trains last week. ... The common thread, besides the train journeys, is the Fidelity And Credibility of Testing and Simulation (FACTS).  My research group is working on how we demonstrate the fidelity of predictions from models, how we establish trust in both predictions from computational models and measurements from experiments that are often also ‘models’ of the real world.  The issues are similar whether we are considering the structural performance of aircraft ..., the impact of agro-chemicals ..., or the performance of fusion energy and the impact of a geological disposal site ...

This blog post appeared on October 30th, 2019, on realizeengineering.blog

On the trustworthiness of multi-physics models

... The process of validating computational models of engineering infrastructure is moving slowly towards establishing an internationally recognised standard...  We have guidelines that recommend approaches for different parts of the validation process...  Under the auspices of the MOTIVATE project, we are gathering experts in Zurich on November 5th, 2019 to discuss the challenges of validating multi-physics models, establishing credibility and the future use of data from experiments...

This blog post appeared on October 2nd, 2019, on realizeengineering.blog

Archive video footage from EU projects
... We are in the last twelve months of the MOTIVATE project and we have started producing video clips about the technology that is being developed.  So, if you missed my presentations at the conference in the US then you can watch the videos online using the links below . We have been making videos describing the outputs of our EU project for about 20 years; so, if you want to see some vintage footage of me twenty years younger then watch a video from the INDUCE project that was active from 1998 to 2001....

MOTIVATE videos: Introduction; Industrial calibration of DIC measurements using a calibration plate or using an LCD screen

 This blog post appeared on June 5th, 2019, on realizeengineering.blog
In Einstein's footprints?

... I have been involved in organising a number of workshops in Glasgow, London, Munich and Shanghai over the last decade.  The next one will be in Zurich in November 2019 in Guild Hall of Carpenters (Zunfthaus zur Zimmerleuten) where Einstein lectured in November 1910 to the Zurich Physical Society ‘On Boltzmann’s principle and some of its direct consequences‘.  Our subject will be different: ‘Validation of Computational Mechanics Models’; ...

This blog post appeared on February 27th, 2019, on realizeengineering.blog

Industrial uncertainty

Last month I spent almost a week in Zurich. ... I was there for the mid-term meeting of the MOTIVATE project and to conduct some tests and demonstrations in the laboratories of our host, EMPA, the Swiss Federal Laboratories for Materials Science and Technology.  Two of our project partners, Dantec Dynamics GmbH based in Ulm, Germany, and the Athena Research Centre in Patras, Greece, have developed methods for quantifying the uncertainty present in measurements of deformation made in an industrial environment using digital image correlation ...

This blog post appeared on December 12th, 2018, on realizeengineering.blog

Spontaneously MOTIVATEd
Some posts arise spontaneously, stimulated by something that I have read or done, while others are part of commitment to communicate on a topic related to my research... This blog post on a new validation metric and flowchart appeared on June 27th, 2018, on realizeengineering.blog
Brave New World
In the brave new world of digital engineering, some engineers are attempting to conceive of a world in which experiments have become obsolete because we can rely on computational modelling to simulate engineering systems. ... This blog post on a world without experiments appeared on January 10th, 2018, on realizeengineering.blog
Getting smarter
Garbage in, garbage out (GIGO) is a perennial problem in computational simulations of engineering structures... This blog post is on how predictions from the simulation have to be tested against reality in order to establish confidence. It appeared on August 2nd, 2017, on realizeengineering.blog

Project Overview

Welcome to MOTIVATE
/documents/1583244/0/EC_flag/d19f0289-0c08-494c-9842-678172cf3050?t=1509629404583

MOTIVATE is an Innovation Action within the European Commission's Horizon 2020 Clean Sky 2 program under Grant Agreement No. 754660, supported by the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number 17.00064.

Our team involves Airbus Operations SAS as Topic Manager as well as University of Liverpool (the Coordinator), Empa, Dantec Dynamics GmbH and Athena Research and Innovation Center as Beneficiaries.

The goal of the project is to deploy a CEN validation method to numerical results from FE simulations of a subcomponent test based on measurements using Digital Image Correlation. Results of this test at an Airbus site  as well as preliminary tests at Empa will be documented on this website. An introductory video is found on Empa TV.

The work is based on a Support Action that was conducted in the FP7 project VANESSA. We have made significant progress on methods for DIC calibration and model validation, notably on measuring the quality of data comparison. A special session at the BSSM international conference in Belfast on September 11th 2019 as well as a Knowledge Exchange workshop in Zurich on November 5th 2019 were held to present and discuss the latest relevant findings. The project results have been summarized on CORDIS.

Now if you are new to the field of validation or want to consult related documents, you will find relevant publications and presentations on this website.

Clean Sky 2 Joint Undertaking
/documents/1583244/1592646/CS2JU/5973e973-3cba-404e-8a78-b17138cf72e0?t=1511523236597

Clean Sky is the largest European research programme developing innovative, cutting-edge technology aimed at reducing CO2, gas emissions and noise levels produced by aircraft. Funded by the EU’s Horizon 2020 programme, Clean Sky contributes to strengthening European aero-industry collaboration, global leadership and competitiveness.

News

The MOTIVATE project team held its final project meeting  on April 30th, 2020. Due to the Corona pandemic they gathered on-line to review the project outcomes which include a novel validation flowchart; digital tools to implement it; a validation metric to quantify the extent to which a model represents an experiment; and methodologies to estimate DIC measurement uncertainty in an industrial environment.

 


Contacts

Prof Dr Eann Patterson
Project Coordinator
University of Liverpool

Eszter

Dr Eszter Szigeti
Topic Manager
Airbus UK

 

 


Flowchart (Version 1.0)

Thumbnail Uploaded by Erwin Hack, 9/11/20 12:26 PM

Version 1.0

Last Updated by Erwin Hack
9/11/20 12:26 PM
Status: Approved
Download (149k) Get URL.
Version History
Version Date Size  
1.0 3 Years Ago 149k