计算流体力学验证与确认研究进展

姓名
邮箱
手机号码
标题
留言内容
验证码

doi:10.6052/1000-0992-23-012

1.
中国空气动力研究与发展中心, 四川绵阳 621000
2.
北京应用物理与计算数学研究所, 北京 100094
3.
航空工业西安航空计算技术研究所, 西安 710000

基金项目:本文工作得到国家数值风洞工程和国家自然科学基金委员会−中国工程物理研究院NSAF联合基金U2230208资助.

详细信息

作者简介:
陈江涛, 中国空气动力研究与发展中心副研究员, 主要从事CFD验证与确认及不确定度量化工作. 主持和参与国家数值风洞（NNW）工程验证与确认理论方法专题、共用技术项目、国家自然科学基金重点项目等10余项重大项目. 牵头制定CFD验证与确认标准, 填补国内领域空白. 出版验证与确认领域专著1部, 译著3部, 发表相关论文40余篇
吴晓军, 中国空气动力研究与发展中心研究员, 国家数值风洞（NNW）工程副总设计师, 验证与确认系统总设计师, 计算空气动力学CFD专委会委员. 长期从事飞行器气动布局设计、飞行器气动关键技术攻关和CFD验证与确认等研究. 先后主持完成国家和省部级重大项目56项, 获省部级科技进步二等奖5项, 三等奖2项, 发表学术论文50余篇, 译著3部
陈坚强, 中国空气动力研究与发展中心研究员, 空天飞行空气动力科学与技术全国重点实验室主任, 长期从事计算流体力学方法研究、高速复杂流动及工业CFD软件研发等领域的研究工作. 担任国家数值风洞（NNW）工程总设计师, 国家重大研发计划重点专项项目首席科学家. 现任中国空气动力学会副理事长, 计算流体力学专委会主任委员, 《Advances in Aerodynamics》创刊主编及多个杂志编委. 主持研发的自主可控NNW套装软件面向全国发布和开源, 应用于120余家航空航天工业部门和研究院所. 获部委级科技进步一等奖2项, 二等奖8项, 三等奖5项, 发表学术论文160余篇, 出版专著1部

通讯作者:
huang7766@sina.com
chenjq@cardc.cn

中图分类号:O355
计量
- 文章访问数:1336
- HTML全文浏览量:671
- PDF下载量:550
- 被引次数:0
出版历程
- 收稿日期:2023-03-16
- 录用日期:2023-06-05
- 网络出版日期:2023-06-06
- 刊出日期:2023-09-30

Advances in verification and validation in computational fluid dynamics

1.
China Aerodynamics Research and Development Centers, Mianyang Sichuan 621000, China
2.
Institute of Applied Physics and Computational Mathematics, Beijing 100094, China
3.
Xi’an Aeronautics Computing Technique Research Institute, AVIC, Xi’an 710000, China

More Information

Corresponding author:huang7766@sina.com;chenjq@cardc.cn

摘要

摘要:计算流体力学 (CFD) 在重大工程领域发挥了日益重要的作用, 可信度是制约其进一步大规模工程应用的关键因素. 国内外普遍认同验证与确认是CFD可信度评价和保证的必经途径. 通过系统的验证与确认, 可以有效识别代码中潜在的编程错误, 保证数值求解的可靠性, 客观评价模型在预期用途内的适用性, 在必要时提高模型的预测能力. 本文围绕着什么是验证与确认, 怎么做验证与确认这两个核心问题, 从基本概念、实施流程、主要方法、标模试验和平台工具等角度介绍了国内外在CFD验证与确认领域的研究进展, 重点对误差估计和不确定度量化方法展开介绍. 文章最后对现阶段验证与确认研究解决实际工程问题的不足进行了评述和总结, 对未来重点研究方向进行了展望.
- 验证与确认/
- 误差估计/
- 不确定度量化/
- 标模试验
Abstract:Computational fluid dynamics (CFD) has played an increasingly important role in major engineering fields, and its credibility is the key constraint to its further extensive engineering application. It is widely accepted home and abroad that verification and validation is the only way to evaluate and guarantee the credibility of CFD. Through systematic verification and validation, the potential programming errors can be effectively identified, the reliability of numerical solving process can be guaranteed, the adequacy and prediction capability of mathematical models in the intended use can be objectively evaluated and improved when necessary. In this paper, with regard to two key issues, ‘‘what is verification and validation’’ and ‘‘how to perform verification and validation’’, the research progress of verification and validation in CFD is introduced from the aspects including basic concept, implementation processes, main methods, calibration model experiments and platform tools, with focusing on numerical error estimation and uncertainty quantification. At the end, the shortcomings of current research are reviewed and the key research directions are prospected.
- verification and validation/
- error estimation/
- uncertainty quantification/
- calibration model experiment

HTML全文

图 1第三届AIAA高升力预测会议计算结果汇总(Rumsey et al. 2019)

下载: 全尺寸图片幻灯片

图 2国家数值风洞工程验证与确认系统架构

下载: 全尺寸图片幻灯片

图 3验证与确认的基本过程 (ASME 2019)

下载: 全尺寸图片幻灯片

图 4验证与确认流程图

下载: 全尺寸图片幻灯片

图 5高超声速巡航导弹确认层级示例 (Oberkampf & Trucano 2000)

下载: 全尺寸图片幻灯片

图 6不确定度量化的关键活动 (NASA 2016b)

下载: 全尺寸图片幻灯片

图 7面积度量方法示意图 (夏侯唐凡等2022)

下载: 全尺寸图片幻灯片

图 8验证与确认感兴趣的物理问题(Hallissy et al. 2014)

下载: 全尺寸图片幻灯片

图 9CHN-T1标模高速风洞油流试验结果

下载: 全尺寸图片幻灯片

表 1使用GCI估计数值离散误差时p和F_s的取值 (Roache 1998)

$ \left\| {\dfrac{{\hat p - {p_{\rm{f}}}}}{{{p_{\rm{f}}}}}} \right\| $	F_s	p
$ \leqslant 0.1 $	1.25	$ {p_{\rm{f}}} $
$ \gt 0.1 $	3.0	min(max(0.5, $ \hat p $),$ {p_{\rm{f}}} $)

下载: 导出CSV

参考文献 (144)

[1]	陈坚强. 2021. 国家数值风洞(NNW)工程关键技术研究进展. 中国科学:技术科学,51: 1326-1347 (Chen J Q. 2021. Advances in the key technologies of Chinese national numerical windtunnel project.Scientia Sinica Technologica,51: 1326-1347).doi:10.1360/SST-2020-0334 Chen J Q. 2021. Advances in the key technologies of Chinese national numerical windtunnel project.Scientia Sinica Technologica,51: 1326-1347).doi:10.1360/SST-2020-0334
[2]	陈树生, 刘丽媛, 闫超, 等. 2017. CFD软件自动化验证确认云平台设计与实现. 航空学报,38: 120209 (Chen S S, Liu L Y, Yan C, et al. 2017. Design and realization of automated testing cloud platform for CFD verification and validation.Acta Aeronautica et Astronautica Sinica,38: 120209). Chen S, Liu L, Yan C, et al. 2017. Design and realization of automated testing cloud platform for CFD verification and validation.Acta Aeronautica et Astronautica Sinica,38: 120209).
[3]	梁益华, 杨永, 朱朝. 2004. CFD 可信度分析平台 WiseCFD. 第十二届全国计算流体力学会议论文, 775-780 Liang Y H, Yang Y, Zhu C. 2004. CFD credibility analysis platform WiseCFD. XT, 775-780.
[4]	刘智益, 王晓东, 康顺. 2013. 叶顶间隙尺度的不确定性对压气机性能影响的CFD模拟. 工程热物理学报,34: 628-631 (Liu Z Y, Wang X D, Kang S. 2013. CFD simulation of uncertain tip clearance effect on compressor performance.Journal of Engineering Thermophysics,34: 628-631). Liu Z Y, Wang X D, Kang S. 2013. CFD simulation of uncertain tip clearance effect on compressor performance.Journal of Engineering Thermophysics,34: 628-631).
[5]	邵帅, 李明, 王年华, 等. 2018. 基于非结构/混合网格模拟黏性流的高阶精度DDG/FV 混合方法. 力学学报,50: 1470-1482 (Shao S, Li M, Wang N H, et al. 2018. High-order DDG/FV hybrid method for viscous flow simulation on unstructured/hybrid grids.Chinese Journal of Theoretical and Applied Mechanics,50: 1470-1482). Shao S, Li M, Wang N H, et al. 2018. High-order DDG/FV hybrid method for viscous flow simulation on unstructured/hybrid grids.Chinese Journal of Theoretical and Applied Mechanics,50: 1470-1482).
[6]	宋赋强, 阎超, 马宝峰, 等. 2018. 锥导乘波体构型的气动特性不确定度分析. 航空学报,39: 121519 (Song F Q, Yan C, Ma B F, et al. 2018. Uncertainty analysis of aerodynamic characteristics for cone-derived waverider configuration.Acta Aeronautica et Astronautica Sinica,39: 121519). Song F Q, Yan C, Ma B F. 2018. Uncertainty analysis of aerodynamic characteristics for cone-derived waverider configuration.Acta Aeronautica et Astronautica Sinica,39: 121519 ).
[7]	汤涛, 周涛. 2015. 不确定性量化的高精度数值方法和理论. 中国科学:数学,45: 891-928 (Tang T, Zhou T. 2015. Recent developments in high order numerical methods for uncertainty quantification.Scientia Sinica:Mathematica,45: 891-928).doi:10.1360/N012014-00218 Tang T, Zhou T. 2015. Recent developments in high order numerical methods for uncertainty quantification.Scientia Sinica: Mathematica,45: 891-928).doi:10.1360/N012014-00218
[8]	王年华, 张来平, 赵钟, 等. 2017. 基于制造解的非结构二阶有限体积离散格式的精度测试与验证. 力学学报,49: 627-637 (Wang N H, Zhang L P, Zhao Z, et al. 2017. Accuracy verification of unstructured second-order finite volume discretization schemes based on the method of manufactured solutions.Chinese Journal of Theoretical and Applied Mechanics,49: 627-637). Wang N H, Zhang L P, Zhao Z, et al. 2017. Accuracy verification of unstructured second-order finite volume discretization schemes based on the method of manufactured solutions.Chinese Journal of Theoretical and Applied Mechanics,49: 627-637).
[9]	王瑞利, 林忠, 袁国兴. 2010. 科学计算程序的验证和确认. 北京理工大学学报,30: 353-360 (Wang R L, Lin Z, Yuan G X. 2010. Verification and validation in scientific computing code.Transactions of Beijing Institute of Technology,30: 353-360). Wang R L, Lin Z, Yuan X G. 2010. Verification and validation in scientific computing code.Transactions of Beijing Institute of Technology,30: 353-360).
[10]	王运涛, 刘刚, 陈作斌. 2019. 第一届航空CFD可信度研讨会总结. 空气动力学报,37: 247-261 (Wang Y T, Liu G, Chen Z B. 2019. Summary of the first aeronautical computational fluid dynamics credibility workshop.Acta Aerodynamica Sinica,37: 247-261). Wang Y T, Liu G, Chen Z B. 2019. Summary of the first aeronautical computational fluid dynamics credibility workshop.Acta Aerodynamica Sinica,37: 247-261.
[11]	邬晓敬, 张伟伟, 宋述芳, 等. 2015. 翼型跨声速气动特性的不确定性及全局灵敏度分析. 力学学报,47: 587-595 (Wu X J, Zhang W W, Song S F, et al. 2015. Uncertainty quantification and global sensitivity analysis of transonic aerodynamics about airfoil.Chinese Journal of Theoretical and Applied Mechanics,47: 587-595).doi:10.6052/0459-1879-14-372 Wu X, Zhang W, Song S, et al. 2015. Uncertainty quantification and global sensitivity analysis of transonic aerodynamics about airfoil.Chinese Journal of Theoretical and Applied Mechanics,47: 587-595).doi:10.6052/0459-1879-14-372
[12]	熊芬芬, 王瑞利, 吴晓军, 等. 2023. 不确定性量化方法及应用. 北京: 科学出版社.
[13]	夏侯唐凡, 陈江涛, 邵志栋, 等. 2022. 随机和认知不确定性框架下的CFD 模型确认度量综述. 航空学报,43: 025716 (Xiahou T F, Chen J T, Shao Z D, et al. 2022. Model validation metrics for CFD numerical simulation under aleatory and epistemic uncertainty.Acta Aeronautica et Astronautica Sinica,43: 025716). (Xiahou T F, Chen J T, Shao Z D, et al. 2022. Model validation metrics for CFD numerical simulation under aleatory and epistemic uncertainty.Acta Aeronautica et Astronautica Sinica,43: 025716).
[14]	肖思男, 吕震宙, 王薇. 2018. 不确定性结构全局灵敏度分析方法概述. 中国科学:物理学力学天文学,48: 014601 (Xiao S N, Lv Z Z, Wang W. 2018. A review of global sensitivity analysis for uncertainty structure.Scientia Sinica Physica, Mechanica & Astronom1ca,48: 014601). Xiao S N, Lv Z Z, Wang W. 2018. A review of global sensitivity analysis for uncertainty structure.Scientia Sinica Physica, Mechanica & Astronom1ca,48: 014601).
[15]	肖钊, 韩旭, 杨刚. 2014. 基于区间技术的模型确认方法及应用. 机械工程学报,50: 177-184 (Xiao Z, Han X, Yang G. 2014. Model validation method and its application based on the interval techniques.Journal of Mechanical Engineering,50: 177-184). Xiao Z, Han X, Yang G. 2014. Model validation method and its application based on the interval techniques.Journal of Mechanical Engineering,50: 177-184).
[16]	赵辉, 胡星志, 张健, 等. 2019. 湍流模型系数的不确定度对翼型绕流模拟的影响. 航空学报,40: 122581 (Zhao H, Hu X Z, Zhang J, et al. 2019. Effects of uncertainty in turbulence model coefficients on flow over airfoil simulation.Acta Aeronautica et Astronautica Sinica,40: 122581). Zhao H, Hu X Z, Zhang J, et al. 2019. Effects of uncertainty in turbulence model coefficients on flow over airfoil simulation.Acta Aeronautica et Astronautica Sinica,40: 122581).
[17]	赵炜, 陈江涛, 肖维, 等. 2020. 国家数值风洞工程(NNW)验证与确认系统关键技术研究进展. 空气动力学学报,38: 1165-1172 (Zhao W, Chen J T, Xiao W, et al. 2020. Advances in the key technologies of verification and validation system of national numerical windtunnel project,.Acta Aerodynamics Sinica,38: 1165-1172). Zhao W, Chen J T, Xiao W, et al. 2020. Advances in the key technologies of verification and validation system of national numerical windtunnel project.Acta Aerodynamics Sinica,38: 1165-1172).
[18]	章超, 刘骁, 陈江涛, 等. 2020. 烧蚀热响应计算中的不确定性传播分析方法研究. 宇航学报,41: 1401-1409 (Zhang C, Liu X, Chen J T, et al. 2020. Study on uncertainty propagation anslysis method in ablative thermal response calculation.Journal of Astronautics,41: 1401-1409).doi:10.3873/j.issn.1000-1328.2020.11.005 Zhang C, Liu X, Chen J T, et al. 2020. Study on uncertainty propagation anslysis method in ablative thermal response calculation.Journal of Astronautics,41: 1401-1409).doi:10.3873/j.issn.1000-1328.2020.11.005
[19]	张伟, 王小永, 于剑, 等. 2018. 来流导致的高超声速气动热不确定度量化分析. 北京航空航天大学学报,44: 1102-1109 (Zhang W, Wang X Y, Yu J, et al. 2018. Uncertainty quantification analysis in hypersonic aerothermodynamics due to freestream uncertainties.Journal of Beijing University of Aeronautics and Astronautics,44: 1102-1109).doi:10.13700/j.bh.1001-5965.2017.0303 Zhang W, Wang X Y, Yu J, et al. 2018. Uncertainty quantification analysis in hypersonic aerothermodynamics due to freestream uncertainties.Journal of Beijing University of Aeronautics and Astronautics,44: 1102-1109).doi:10.13700/j.bh.1001-5965.2017.0303
[20]	中国科学院. 2018. 新型飞行器中的关键力学问题. 北京: 科学出版社.
[21]	AIAA (American Institute of Aeronautics and Astronautics). 1998. Guide for the verification and validation of computational fluid dynamics simulations, AIAA-G-077-1998.
[22]	AIAA. 1999. Assessment of experimental uncertainty with application to wind tunnel testing, AIAA S-071A-1999.
[23]	AIAA. 2003. Assessing experimental uncertainty supplement to AIAA S-071A-1999, AIAA G-045-2003.
[24]	Arendt P D, Apley D W, Chen W. 2012. Quantification of model uncertainty: calibration, model discrepancy, and identifiability.Journal of Mechanical Design,134: 100908.doi:10.1115/1.4007390
[25]	ASME (American Society of Mechanical Engineers). 2005. Test uncertainty. ASME PTC 19.1-2005.
[26]	ASME. 2006. Guide for verification and validation in computational solid mechanics. ASME V&V 10-2006.
[27]	ASME. 2009. Standard for verification and validation in computational fluid dynamics and heat transfer.
[28]	ASME. 2012. An illustration of the concepts of verification and validation in computational solid mechanics. ASME V&V 10.1-2012.
[29]	ASME. 2019. Standard for verification and validation in computational solid mechanics. ASME V&V 10-2019.
[30]	Andrianov G, Burriel S, Cambier S, et al. 2007. OpenTURNS, an open source initiative to treat uncertainties, Risks'N statistics in a structured industrial approach. In: Proceedings of the ESREL’2007 Safety and Reliability Conference, Stavenger: Norway.
[31]	Avdonin A, Polofke W. 2019. Quantification of the impact of uncertainties in operating conditions on the flame transfer function with nonintrusive polynomial chaos expansion.Journal of Engineering for Gas Turbines and Power,141: 011020.doi:10.1115/1.4040745
[32]	Babuska I, Nobile F, Tempone R. 2007. A stochastic collocation method for elliptic partial differential equations with random input data.SIAM J Numer Anal,45: 1005-1034.doi:10.1137/050645142
[33]	Bhattacharyya B. 2022. Uncertainty quantification of dynamical systems by a POD-kriging surrogate model.Journal of Computational Science,60: 101602.doi:10.1016/j.jocs.2022.101602
[34]	Blatman G, Sudret B. 2010. An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis.Probab Eng Mech,25: 183-97.doi:10.1016/j.probengmech.2009.10.003
[35]	Blatman G, Sudret B. 2011. Adaptive sparse polynomial chaos expansion based on least angle regression.Journal of Computational Physics,230: 2345-67.doi:10.1016/j.jcp.2010.12.021
[36]	Blottner F G. 1990. Accurate Navier-Stokes results for the hypersonic flow over a spherical nosetip.Journal of Spacecraft&Rockets,27: 113-122.
[37]	Brehm C, Hader C, Fasel H F. 2015. A locally stabilized immerse boundary method for the compressible Navier-Stokes equations.Journal of Computational Physics,295: 475-504.doi:10.1016/j.jcp.2015.04.023
[38]	Bungartz H J, Griebel M. 2004. Sparse grids.Acta Numer,13: 1-123.doi:10.1017/S0962492904000169
[39]	Burg C, Murali V K. 2004. Efficient code verification using the residual formulation of the method of manufactured solution. AIAA Paper, 2628.
[40]	Cary A W, Chawner J R, Earl P N, et al. 2021. CFD vision 2030 roadmap: progress and perspectives. AIAA Aviation Forum.
[41]	Casey M, Wintergerste T, Innotec S. 2000. Ercoftac special interest group on quality and trust in industrial CFD: Best practice guidelines. Brussels: European Research Community on Flow, Tutbulence and Combusion.
[42]	Choudhary A, Roy C J, Dietiker J F, et al. 2014. Code verification for multiphase flows using the method of manufactured solutions.International Journals of Multiphase Flow,80: 150-163.
[43]	DoD. 1994. DoD Directive No. 5000.59: Modeling and simulation (M&S) management.
[44]	DoD. 1996. DoD Instruction 5000.61: Modeling and simulation (M&S) verification, validation, and accreditation (VV&A). Defense Modeling and Simulation Office, Office of the Director of Defense Research and Engineering.
[45]	DoD. 1997. DoD Modeling and simulation glossary. from www. msco. mil.
[46]	DoD. 2008. Department of defense standard practice: documentation of verification, validation, and accreditation (VV&A) for models and simulations. US Washington, DC, Department of defense.
[47]	Duan Q, Sorooshian S, Gupta V. 1992. Effective and efficient global optimization for conceptual rainfall-runoff models.Water Resour. Res.,28: 1015-1031.doi:10.1029/91WR02985
[48]	Dunn M C, Shotorban B, Frendi A. 2011. Uncertainty quantification of turbulence model coefficients via Latin hypercube sampling method.Journal of Fluids Engineering,133: 2913-2921.
[49]	Eça L, Hoekstra M. 2014. A procedure for the estimation of the numerical uncertainty of CFD calculations based on grid refinemen tstudies.Journal of Computational Physics,262: 104-130.doi:10.1016/j.jcp.2014.01.006
[50]	Eca L, Klaij C M, Vaz G, et al. 2016. On code verification of RANS solvers.Journal of Computational Physics,310: 418-439.doi:10.1016/j.jcp.2016.01.002
[51]	Eça L, Vaz G, Pereira F S, et al. 2019. Numerical and parameter uncertainties: Are they independent?// Verification and Validation Symposium, VVS 2019.
[52]	Eça L, Pereira F S, et al. 2020. On the role of discretization errors in the quantification of parameter uncertainties. Proceedings of ASME 2020 verification and validation symposium, VVS2020-8825, virtual, online.
[53]	Adams B M, Bauman L E, Bohnhoff W J, et al. 2009. Dakota, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis: Version 5.4 user's manual. Sandia Technical Report SAND2010-2183.
[54]	Ferson S, Oberkampf W L. 2009. Validation of imprecise probability models.International Journal of Reliability and Safety,3: 3-22.doi:10.1504/IJRS.2009.026832
[55]	Ferziger J H, Peric M. 1996. Further discussion of numerical errors in CFD.International Journal for Numerical Methods in Fluids,23: 1263-1274.doi:10.1002/(SICI)1097-0363(19961230)23:12<1263::AID-FLD478>3.0.CO;2-V
[56]	Ferziger J H. 1998. A note on numerical accuracy.International Journal for Numerical Methods in Fluids,8: 995-996.
[57]	Ferziger J H, Peric M. 2002. Computational methods for fluid dynamics/ 3rd edn. Berlin, Springer-Verlag.
[58]	Fishman G S. 1996. Monte carlo: concepts, algorithms, and applications. New York: Springer-Verlag.
[59]	Gan Y J, Duan Q Y, Gong W, et al. 2014. A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model.Environmental Modelling & Software,51: 269-285.
[60]	Ghanem R, Spanos P. 1992. Stochastic finite elements: A spectral approach. New York: Springer-Verlag.
[61]	Ghia U, Bayyuk S, Roy C, et al. 2010. The AIAA code verification project-test cases for CFD code verification. AIAA Paper, 125.
[62]	Golub G H, Van Loan C F. 1996. Matrix computations, 3rd edn. Baltimore. The Johns Hopkins University Press.
[63]	Grier B, Alyanak E, White M, et al. 2014. Numerical integration techniques for discontinuous manufactured solutions.Journal of Computational Physics,278: 193-203.doi:10.1016/j.jcp.2014.08.031
[64]	Grier B, Figliola R, Alyanak E, et al. 2015. Discontinuous solutions using the method of manufactured solutions on finite volume solvers.AIAA J,53: 2369-2378.doi:10.2514/1.J053725
[65]	Hallissy B P, Hariharan N S, Laiosa J P, et al. 2014. CREATETM-AV quality assurance: best practices for validating and supporting computation-based engineering software. AIAA Paper, 0918.
[66]	Hebert S, Luke E A. 2005. Honey, I shrunk the grids! A new approach to CFD verification studies. AIAA Paper, 685.
[67]	Heeg J, Wieseman C D, Chwalowsiki P. 2016. Data comparisons and summary of the second aeroelastic prediction workshop. AIAA 2016-3121.
[68]	Hirsch C. 2008. The development of a framework for CFD validation and best practice: the QNET-CFD knowledge base.Chinese Journal of Aeronautic,19: 105-113.
[69]	Huang H, Giacobello M. 2022. Uncertainty quantification in store separation analysis using kestrel, Design of experiments and surrogate modelling. AIAA2022-1318. AIAA Scitech 2022 Forum San Diego, CA & Virtual.
[70]	IEEE. 1984. IEEE standard dictionary of electrical and electronics terms. ANSI/IEEE Std 100-1984, New York.
[71]	Jiang X, Mahadevan S. 2007. Bayesian risk-based decision method for model validation under uncertainty.Reliability Engineering & System Safety,92: 707-718.
[72]	Kawai S, Shimoyama K. 2014. Kriging-model-based uncertainty quantification in computational fluid dynamics. 32nd AIAA Applied Aerodynamics Conference.
[73]	Kennedy M C, O'Hagan A. 2001. Bayesian calibration of computer models.Journal of the Royal Statistical Society:Series B,3: 425-464.
[74]	Kirkpatrick S, Gelatt C D, Vecchi A. 1983. Optimization by simulated annealing.Science,220: 671-680.doi:10.1126/science.220.4598.671
[75]	Lax P D. 1954. Weak solutions of nonlinear hyperbolic equations and their numerical computation.Commun. Pure Appl. Math.,7: 159-193.doi:10.1002/cpa.3160070112
[76]	Ling Y, Mahadevan S. 2013. Quantitative model validation techniques: new insights.Reliability Engineering & System Safety,111: 217-231.
[77]	Liu D, Litvinenko A, Schillings C, et al. 2017. Quantification of airfoil geometry-induced aerodynamic uncertainties--comparison of approaches.Society for Industrial and Applied Mathematics,5: 334-352.
[78]	Liu S, Wang Y, Qin N, et al. 2020. Quantification of airfoil aerodynamic uncertainty due to pressure-sensitive paint thickness.AIAA J,58: 1432-1440.doi:10.2514/1.J058801
[79]	Liu W, Belytschko T, Mani A. 1986a. Probabilistic finite elements for nonlinear structural dynamics.Comput Methods Appl Mech Engrg,56: 61-81.doi:10.1016/0045-7825(86)90136-2
[80]	Liu W, Belytschko T, Mani A. 1986b. Random field finite elements.Internat J Numer Methods Engrg,23: 1831-1845.doi:10.1002/nme.1620231004
[81]	Liu X, Furrer D, Kosters J, et al. 2018. Vision 2040: A roadmap for integrated, multiscale modeling and simulation of materials and systems. NASA/CR-2018-219771.
[82]	Liu Y, Chen W, Arendt P, et al. 2011. Toward a better understanding of model validation metrics.ASME Journal of Mechanical Design,133: 071005.doi:10.1115/1.4004223
[83]	Loeven A, Bijl H. 2008. Airfoil analysis with uncertain geometry using the probabilistic collocation method. 49th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Schaumburg, IL, AIAA 2008-2070.
[84]	Mahaffy, J. 2007. Best practice guidelines for the use of CFD in nuclear reactor safety applications. OECD, Nuclear Energy Agency, Technical Report, NEA/CSNI/R(2007)5.
[85]	Marelli S, Sudret B. 2014. UQLab: A framework for uncertainty quantification in matlab.Int Conf on Vulnerability, Risk Analysis & Management, 2554-2563.
[86]	Marini M, Paoli R, Grasso F, et al. 2002. Verification and validation in computational fluid dynamics: The flownet databese experience.Bulletin of the JSME,45: 15-22.
[87]	Mariotti A, Salvetti M V, Omrani P S, et al. 2016. Stochastic analysis of the impact of freestream conditions on the aerodynamics of a rectangular 5: 1 cylinder.Computers & Fluids,136: 170-192.
[88]	Marshall D D. 2011. A scientific software verification library based on the method of manufactured solutions. AIAA Paper, 615.
[89]	NRC (National Research Council). 2012. Assessing the reliability of complex models: mathematical and statistical foundations of verification, validation, and uncertainty quantification.The National Academies Press,Washington: D.C.
[90]	Murali V K, Burg C. 2002. Verification of 2D Navier-Stokes codes by the method of manufactured solutions. AIAA Paper, 3109.
[91]	NASA (National Aeronautics and Space Administration). 2016a. Standard for models and simulations. NASA-STD-7009A, superseding NASA-STD-7009, 2008.
[92]	NASA. 2016b. Simulation credibility advances in verification, validation and uncertainty quantification. NASA/TP-2016-219422 JANNAF/GL-2016-0001.
[93]	Navah F, Nadarajah S. 2016. On the verification of high-order CFD solvers.VII European Congress on Computational Methods in Applied Sciences and Engineering,Crete: Greece.
[94]	Niederreiter H. 1992. Random number generation and quasi-monte carlo methods. Philadelphia: SIAM.
[95]	Niederreiter H, Hellekalek P, Larcher G, et al. 1998. Monte carlo and quasi-monte carlo methods. New York: Springer-Verlag.
[96]	Norman R, Philip D. 2006. The Navier-Stokes equations: a classification of flows and exact solutions. Cambridge University Press.
[97]	Oberkampf W L, Roy C J. 2010. Verification and validation in scientific computing. Cambridge University Press.
[98]	Oberkampf W L, Smith B L. 2017. Assessment criteria for computational fluid dynamics model validation experiments.ASME J. Verif. , Validation Uncertainty Quantif.,2: 031002.doi:10.1115/1.4037887
[99]	Oberkampf W L, Trucano T G. 2000. Validation methodology in computational fluid dynamics. Fluids 2000 Conference AIAA 2000-2549.
[100]	Park I, Amarchinta H K, Grandhi R V. 2010. A bayesian approach for quantification of model uncertainty.Reliab Eng Syst Saf,95: 777-785.doi:10.1016/j.ress.2010.02.015
[101]	Nair P B, Keane A J. 2002. Stochastic reduced basis methods.AIAA J,40: 1653-64.doi:10.2514/2.1837
[102]	Platteeuw P, Loeven G, Bijl H. 2008. Uncertainty quantification applied to the k-epsilon model of turbulence using the probabilistic collocation method. 49th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Schaumburg, IL. Reston, Virginia: AIAA.
[103]	Raisee M, Kumar D, Lacor C. 2015. A non-intrusive model reduction approach for polynomial chaos expansion using proper orthogonal decomposition.Int J Numer Methods Eng,103: 293-312.doi:10.1002/nme.4900
[104]	Ricci P, Riva F, Theiler C, et al. 2015. Approaching the investigation of plasma turbulence through a rigorous verification and validation procedure: A practical example.Physics of Plasma,22: 055704.doi:10.1063/1.4919276
[105]	Richard G H, David C M, Jonathan W N. 2015. V&V framework. Sandia National Labs. Rept. SAND2015-7455, Albuquerque, NM.
[106]	Riley M E, Grandhi R V. 2011. Quantification of model-form and predictive uncertainty for multi-physics simulation.Computers and Structures,89: 1206-1213.doi:10.1016/j.compstruc.2010.10.004
[107]	Roache P J. 1998. Verification and validation in computational science and engineering. Albuquerque, NM, Hermosa Publishers.
[108]	Roy C J, Oberkampf W L. 2011. A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing.Computer Methods in Applied Mechanics and Engineering,200: 2131-2144.doi:10.1016/j.cma.2011.03.016
[109]	Roy C, Blottner F G. 2003. Methodology for turbulence model validation: application to hypersonic flows.Journal of Spacecraft and Rockets,40: 313-325.doi:10.2514/2.3966
[110]	Roy C J, Nelson C C, Smith T M, et al. 2004. Verification of Euler/Navier-Stokes codes using the method of manufactured solutions.International Journal for Numerical Methods in Fluids,44: 599-620.doi:10.1002/fld.660
[111]	Rumpfkeil M P, Hanazaki K, Beran P S. 2017. Construction of multi-fidelity locally optimized surrogate models for uncertainty quantification. 19th AIAA Non-Deterministic Approaches Conference.
[112]	Rumsey C L, Slotnick J P, Sclafani A J. 2019. Overview and summary of the third AIAA high lift prediction workshop.Journal of Aircraft,56: 621-644.doi:10.2514/1.C034940
[113]	Saltelli A, Tarantola S, Campolongo F, et al. 2004. Sensitivity analysis in practise: a guide to assessing scientific models. John Wiley & Sons, Ltd, Chichester, West Sussex.
[114]	Schlesinger S. 1979. Terminology for model credibility.Simulation,32: 103-104.doi:10.1177/003754977903200304
[115]	Schaefer J A, Cary A W, Mani M, et al. 2017. Uncertainty quantification and sensitivity analysis of sa turbulence model coefficients in two and three dimensions. 55th AIAA Aerospace Sciences Meeting.
[116]	Sentz K, Ferson S. 2002. Combination of evidence in dempster-shafer theory. Contemporary Pacific.
[117]	Shafer G. 1976. A mathematical theory of evidence. Princeton University Press.
[118]	Shu C W. 1998. Essentially non-oscillatory and weighted essentially non-oscillatory schemes for hyperbolic conservation laws.Institute for Computer Applications in Science and Engineering(ICASE) ,1697: 325-432.
[119]	Slotnick J, et al. 2014. CFD vision 2030 study: A path to revolutionary computational aerosciences. NASA/CR-2014-218178.
[120]	Smolyak S. 1963. Quadrature and interpolation formulas for tensor products of certain classes of functions.Dokl Akad Nauk SSSR,4: 1042-1045.
[121]	Sod G. 1978. A survey of several finite difference methods for systems of nonlinear hyperbolic conservation laws.J. Comput. Phys.,27: 1-31.doi:10.1016/0021-9991(78)90023-2
[122]	Stein M. 1987. Large sample properties of simulations using latin hypercube sampling.Technometrics,29: 143-151.doi:10.1080/00401706.1987.10488205
[123]	Swiler L P, Eldred M S. 2009. Efficient algorithms for mixed aleatory-epistemic uncertainty quantification with application to radiation-hardened electronics. Sandia National Labs, Rept. SAND2009-5805, Albuquerque, NM.
[124]	Swiler L P, Mayes R L, Eldred M S. 2009. Epistemic uncertainty in the calculation of margins. 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper, 2249, Palm Springs, CA.
[125]	Thomas J L, Diskin B, Rumsey C L. 2008. Toward verification of unstructured-grid solvers.AIAA J,46: 3070-3079.doi:10.2514/1.36655
[126]	Tinoco E N, Brodersen O P, Keye S, et al. 2018. Summary data from the sixth AIAA CFD drag prediction workshop: CRM cases.Journal of Aircraft,55: 1352-1379.doi:10.2514/1.C034409
[127]	Tong C. 2005. Psuade user's manual.Lawrence Livermore National Laboratory(LLNL) ,Livermore: CA.
[128]	Venditti D A and Darmofal D L. 2000. Adjoint error estimation and grid adaptation for functional outputs: application to quasi-one-dimensional flow.Journal of Computational Physics,164: 204-227.doi:10.1006/jcph.2000.6600
[129]	Vidanovic N, Rasuo B, Damljanovic D, et al. 2014. Validation of the CFD code for determination of aerodynamic characteristics of nonstandard AGARD-B calibration model.Thermal Science, 18.
[130]	Voyles I T, Roy C J. 2015. Evaluation of model validation techniques in the presence of aleatory and epistemic input uncertainties. AIAA 2015-1374. AIAA SciTech, Kissimmee, Florida.
[131]	Wang C, Duan Q Y, Gong W, et al. 2014. An evaluation of adaptive surrogate modeling based optimization with two benchmark problems.Environ. Model. Softw,60: 167-179.doi:10.1016/j.envsoft.2014.05.026
[132]	Wang C, Duan Q Y, Charles H T, et al. 2016. A GUI platform for uncertainty quantification of complex dynamical models.Environmental Modelling & Software,76: 1-12.
[133]	Wang N, Yao W, Zhao Y, et al. 2018. A new interval area metric for model validation with limited experimental data.ASME Journal of Mechanical Design,140: 1-11.
[134]	Wang Y J, Zhang S D. 2016. Uncertainty quantification of numerical simulation of flows around a cylinder using non-intrusive polynomial chaos.Chinese Physics Letters,33: 090501.doi:10.1088/0256-307X/33/9/090501
[135]	Loh W. 1996. On Latin hypercube sampling.Ann Statist,24: 2058-2080.
[136]	Wiener N. 1938. The homogeneous chaos.Amer J Math,60: 897-936.doi:10.2307/2371268
[137]	Wilson G E, Boyack B E. 1998. The role of the PIRT process in experiments, code development and code applications associated with reactor safety analysis.Nuclear Engineering and Design,186: 23-37.doi:10.1016/S0029-5493(98)00216-7
[138]	Woods C N, Starkey R P. 2015. Verification of fluid-dynamic codes in the presence of shocks and other discontinuities.Journal of Computational Physics,294: 312-328.doi:10.1016/j.jcp.2015.03.055
[139]	Xiu D, Karniadakis G E. 2002. The wiener-askey polynomial chaos for stochastic differential equations.SIAM J Sci Comput,24: 619-644.doi:10.1137/S1064827501387826
[140]	Xiu D, Hesthaven J S. 2005. High-order collocation methods for differential equations with random inputs.SIAM J Sci Comput,27: 1118-1139.doi:10.1137/040615201
[141]	Xiu D. 2007. Efficient collocational approach for parametric uncertainty analysis.Commun Comput Phys,2: 293-309.
[142]	Zhang D X. 2002. Stochastic methods for flow in porous media. New York: Academic Press.
[143]	Zhu H Y, Wang G, Liu Y, et al. 2020. Numerical investigation of transonic buffet on supercritical airfoil considering uncertainties in wind tunnel testing.International Journal of Modern Physics B,34: 2040083.doi:10.1142/S0217979220400834
[144]	Zio E, Apostolakis G E. 1996. Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories.Reliab Eng Syst Saf,54: 225-241.doi:10.1016/S0951-8320(96)00078-6