Starting Point: CEN Workshop Agreement
We have developed novel methods for the comparison of predictive numerical models with full-field experimental data in order to achieve a robust, quantitative validation of the simulation. In particular an understanding has been gained of the uncertainties in simulation and experimental data and their influence on predictions and measurements.
Criteria allowing an easy comparison and interpretation of data have been embedded in novel correlation methods that allow confidence in simulations to be established, supported by the quantifcation of 'the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model', as the ASME definition of validation reads. Work at ULIV has demonstrated that feature vectors, obtained via image decomposition, can be used to generate relative error metrics. This preliminary work is being integrated with the prior work embedded in the CEN guideline to produce a validation methodology.
The MOTIVATE Validation Flowchart, reproduced below, forms the basis of the MOTIVATE Protocol which can be downloaded from Zenodo, together with a GUI that guides the user through the validation process step by step. A key novel feature is the evaluation of historical data. In addition, the development of the model takes priority, while physical testing is performed only if required. The processes involved are shown as coloured boxes. This includes the decision sequence required to evaluate historical data for use in the validation process and the quantitative validation process described in the recently published CEN guide. To allow for an unbiased comparison of the data sets, it is recommended to implement a “double blind” procedure, in which interaction of the testing and modelling teams are only allowed as far as to clarify and agree on test object, boundary conditions and load cases. Then, data are generated by experiment (blue box) and by simulation (orange box) independently, and the results are transferred to an independent validation team who is in charge of the Quantitative Comparison (red box). It should be noted that, prior to starting the validation process, it will be necessary for decision-makers to state their expectations so that appropriate decision criteria can be adopted, for instance requirements for measurement uncertainty. While this activity of the decision-makers was not within the brief of the MOTIVATE project, their review at the end of the process is shown for completeness and comparison with the existing ASME flowchart
E.A. Patterson, I. Diamantakos, K. Dvurecenska, R.J. Greene, E. Hack, G. Lampeas, M. Lomnitz, and T. Siebert, Validation of a structural model of an aircraft cockpit panel: An industrial case study, Journal of Strain Analysis for Engineering Design
T. Siebert, E. Hack, G. Lampeas, E.A. Patterson, and K. Splitthof, Uncertainty Quantification for DIC Displacement Measurements in Industrial Environments, Experimental Techniques 45 (2021) 685–694
- K. Dvurecenska, I. Diamantakos, E. Hack, G. Lampeas, E.A. Patterson, T. Siebert, The validation of a full-field deformation analysis of an aircraft panel – a case study, Journal of Strain Analysis for Engineering Design 56 (2021) 265-272
- K. Dvurecenska, E. Patelli, E.A. Patterson, Probabilistic metric for validation based on strain field data, BSSM's 14th Int. Conf. on Advances in Experimental Mechanics, Belfast, UK, 10-12 Sep 2019
- E. Hack, K. Dvurecenska, G. Lampeas, E.A. Patterson, T. Siebert, E.Szigeti, Incorporating historical data in a validation process, BSSM's 14th Int. Conf. on Advances in Experimental Mechanics, Belfast, UK, 10-12 Sep 2019
- A. Dean, W.J.R. Christian, Generalised decomposition of strain fields in complex components, BSSM's 14th Int. Conf. on Advances in Experimental Mechanics, Belfast, UK, 10-12 Sep 2019
- G. Lampeas, E.A. Patterson, E. Hack, Round robin results for CWA16799:2014 – Validation of computational solid mechanics models, BSSM's 14th Int. Conf. on Advances in Experimental Mechanics, Belfast, UK, 10-12 Sep 2019
- G. Lampeas, I. Diamantakos, T. Siebert, E. Hack, E. Patterson, Uncertainty quantification of Digital Image Correlation measurements based on projected speckle patterns, EASN 2019 Conference, Athens, Greece, 04–06 Sep 2019
- A. Alexiadis, R.L. Burguete, K. Dvurecenska, E. Hack, G. Lampeas, E.A. Patterson, T. Siebert and E. Szigeti, A Novel Flow-chart for Model Validation: Is it Conceivable to Validate without New Measurements, SEM Annual Conference, Reno, USA, 03–06 June 2019
- E. Hack, K. Dvurecenska, G. Lampeas, E.A. Patterson, T. Siebert, and E. Szigeti, Steps towards industrial validation experiments, ICEM 2018, Brussels, BE, 01-05 July 2018, Proceedings 2 (2018) 391
- K. Dvurecenska, E. Hack, G. Lampeas, T. Siebert and E.A. Patterson, Comparative study of orthogonal decomposition of surface deformation in composite automotive panel, ECCM18 - 18th European Conference on Composite Materials, Athens, HE, 24-28th June 2018
- K. Dvurecenska, E. Patelli, E.A. Patterson, What’s the probability that a simulation agrees with your experiment?, Photomechanics 2018, Toulouse, FR, 19–22 March 2018, Book of Abstracts pp.52-53
Slowly crossing the valley of death
The valley of death ... is the gap between discovery and application, or between realization of an idea in a laboratory and its implementation in the real-world. ... Our work on quantitative comparisons of data fields from physical measurements and computer predictions is about three-quarters of the way across the valley. We published a paper in December (see Dvurecenska et al, 2020) on its application to a large panel from the fuselage of an aircraft based on work we completed as part of the MOTIVATE project. ...
This blog post appeared on January 27th, 2021, on realizeengineering.blog.
The blind leading the blind
Three years after it started, the MOTIVATE project has come to an end. The focus of the project has been about improving the quality of validation for predictions of structural behaviour in aircraft using fewer, better physical tests. We have developed an enhanced flowchart for model validation, a method for quantifying uncertainty in measurements of deformation in an industrial environment and a toolbox for quantifying the extent to which predictions from computational models represent measurements made in the real-world. In the last phase of the project, we demonstrated all of these innovations on the fuselage nose section of an aircraft. ...This blog post appeared on May 27th, 2020, on realizeengineering.blog.
Alleviating industrial uncertainty
Want to know how to assess the quality of predictions of structural deformation from a computational model and how to diagnose the causes of differences between measurements and predictions? The MOTIVATE project has the answers; that might seem like an over-assertive claim but read on and make your own judgment. ...
This blog post appeared on May 13th, 2020, on realizeengineering.blog.
Fake facts & untrustworthy predictions
I need to confess to writing a misleading post some months ago entitled ‘In Einstein’s footprints?‘ on February 27th 2019, in which I promoted our 4th Workshop on the ‘Validation of Computational Mechanics Models‘ that we held last month at Guild Hall of Carpenters [Zunfthaus zur Zimmerleuten] in Zurich. I implied that speakers at the workshop would be stepping in Einstein’s footprints when they presented their research at the workshop, because Einstein presented a paper at the same venue in 1910. However, ...
This blog post appeared on December 4th, 2019, on realizeengineering.blog.
Same problems in a different language
I spent a lot of time on trains last week. ... The common thread, besides the train journeys, is the Fidelity And Credibility of Testing and Simulation (FACTS). My research group is working on how we demonstrate the fidelity of predictions from models, how we establish trust in both predictions from computational models and measurements from experiments that are often also ‘models’ of the real world. The issues are similar whether we are considering the structural performance of aircraft ..., the impact of agro-chemicals ..., or the performance of fusion energy and the impact of a geological disposal site ...
This blog post appeared on October 30th, 2019, on realizeengineering.blog.
On the trustworthiness of multi-physics models
... The process of validating computational models of engineering infrastructure is moving slowly towards establishing an internationally recognised standard... We have guidelines that recommend approaches for different parts of the validation process... Under the auspices of the MOTIVATE project, we are gathering experts in Zurich on November 5th, 2019 to discuss the challenges of validating multi-physics models, establishing credibility and the future use of data from experiments...
This blog post appeared on October 2nd, 2019, on realizeengineering.blog.
Archive video footage from EU projects
In Einstein's footprints?
... I have been involved in organising a number of workshops in Glasgow, London, Munich and Shanghai over the last decade. The next one will be in Zurich in November 2019 in Guild Hall of Carpenters (Zunfthaus zur Zimmerleuten) where Einstein lectured in November 1910 to the Zurich Physical Society ‘On Boltzmann’s principle and some of its direct consequences‘. Our subject will be different: ‘Validation of Computational Mechanics Models’; ...
This blog post appeared on February 27th, 2019, on realizeengineering.blog.
Last month I spent almost a week in Zurich. ... I was there for the mid-term meeting of the MOTIVATE project and to conduct some tests and demonstrations in the laboratories of our host, EMPA, the Swiss Federal Laboratories for Materials Science and Technology. Two of our project partners, Dantec Dynamics GmbH based in Ulm, Germany, and the Athena Research Centre in Patras, Greece, have developed methods for quantifying the uncertainty present in measurements of deformation made in an industrial environment using digital image correlation ...
This blog post appeared on December 12th, 2018, on realizeengineering.blog.
Brave New World
Welcome to MOTIVATE
MOTIVATE is an Innovation Action within the European Commission's Horizon 2020 Clean Sky 2 program under Grant Agreement No. 754660, supported by the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number 17.00064.
Our team involves Airbus Operations SAS as Topic Manager as well as University of Liverpool (the Coordinator), Empa, Dantec Dynamics GmbH and Athena Research and Innovation Center as Beneficiaries.
The goal of the project is to deploy a CEN validation method to numerical results from FE simulations of a subcomponent test based on measurements using Digital Image Correlation. Results of this test at an Airbus site as well as preliminary tests at Empa will be documented on this website. An introductory video is found on Empa TV.
The work is based on a Support Action that was conducted in the FP7 project VANESSA. We have made significant progress on methods for DIC calibration and model validation, notably on measuring the quality of data comparison. A special session at the BSSM international conference in Belfast on September 11th 2019 as well as a Knowledge Exchange workshop in Zurich on November 5th 2019 were held to present and discuss the latest relevant findings. The project results have been summarized on CORDIS.
Clean Sky 2 Joint Undertaking
Clean Sky is the largest European research programme developing innovative, cutting-edge technology aimed at reducing CO2, gas emissions and noise levels produced by aircraft. Funded by the EU’s Horizon 2020 programme, Clean Sky contributes to strengthening European aero-industry collaboration, global leadership and competitiveness.
The MOTIVATE project team held its final project meeting on April 30th, 2020. Due to the Corona pandemic they gathered on-line to review the project outcomes which include a novel validation flowchart; digital tools to implement it; a validation metric to quantify the extent to which a model represents an experiment; and methodologies to estimate DIC measurement uncertainty in an industrial environment.