2016 NGWA Groundwater Summit

Evaluation of excluding deficient models on multi-model analyses using AICc and KIC information criteria

Tuesday, April 26, 2016: 11:40 a.m.
Confluence Ballroom B (The Westin Denver Downtown)
Judith Schenk , Colorado School of Mines, Golden, CO
Eileen P. Poeter , International Ground-Water Modeling Center, Department Geology and Geological Engineering, Colorado School of Mines, Golden, CO

AICc and KIC (Akaike second order and Kashyap information criteria) were compared in experiments where hydraulic conductivity and recharge were optimized for a set of experimental models using different boundary conditions and calibration data sets. Use of experimental models allows us to know “true” conditions, and thus evaluate procedures for improving model predictions. Multi-model analysis was conducted for each full model set and for a model subset which excluded deficient models. Objective criteria were used to identify deficient models. Model-averaged predictions based on AICc and KIC were compared to determine the impact of removal of deficient models. Using full model sets, AICc results were generally more precise than KIC, but less accurate such that some AICc model-averaged predictions did not include the true prediction within the confidence region. This condition persisted even with the removal of deficient models. Use of KIC to model-average predictions results in an extremely wide confidence region for some model sets, but precision was improved for most sets when deficient models were removed. The confidence region based on KIC increased for one experimental set after deficient models were removed. With a reduced model set, KIC results were generally more accurate but less precise than AICc. One exception was a model set where KIC was more precise, but in that case the true predictions were not contained within the KIC confidence region. Some model sets included only two or three models after deficient models were removed and in those cases AICc and KIC results were nearly identical.  In conclusion, for these experiments, the removal of deficient models did not change the quality of AICc model-averaged predictions, but KIC performance was improved. This is likely due to the inclusion of the Fisher Information term in the KIC criterion which will be unreasonably small for deficient models.

Judith Schenk, Colorado School of Mines, Golden, CO
Judith Schenk has worked on numerous modeling projects including development of the regional San Luis Valley, Colorado and the Great Sand Dunes National Park groundwater models for which she also wrote software to integrate data from different sources into the groundwater models. She has completed modeling projects for the Environmental Protection Agency, the Office of Surface Mining, the National Park Service, the Justice Department, and the Fish and Wildlife Service in addition to other modeling projects for municipalities and private firms. She has developed several MODFLOW modules including a mine adit module which was used to assess the effects of plugging the Reynolds adit at the Summitville mine site in Colorado. Her current research involves investigation of AICc and KIC multi-model discrimination criteria used in model selection and multi-model inference.

Eileen P. Poeter, International Ground-Water Modeling Center, Department Geology and Geological Engineering, Colorado School of Mines, Golden, CO
Eileen Poeter, Ph.D is an Emeritus Professor of Geological Engineering at the Colorado School of Mines and Director of the International Ground Water Modeling Center. Before entering academia, she worked for Golder Associates then formed Poeter Engineering in 1984. Poeter earned a B.S. in Geology from Lehigh University in 1975, and an M.S. and Ph.D. in 1978 and 1980 in Engineering Science from Washington State University. She has taught semester and short courses for 26 years. Her research focuses on parameter estimation and multi-model evaluation. She is the author of UCODE_2005, MMA, and Sim-Adjust, and was the 2006 Darcy lecturer.