XI Conference – European Congress of Methodology

Comparing nested multilevel models with the Vuong test

24 Jul 2025, 15:00

15m

Faculty of Social Sciences and Communication. (The Pyramid)/13 - Room (Faculty of Social Sciences and Communication. (The Pyramid))

Oral Presentation Statistical analyses Session 15 : "Multilevel models and Individual differences"

Speaker

Mr José María Santa Olalla Tovar (Department of Methodology of Behavioral Sciences. National Distance Education University (Spain))

Abstract

Vuong test is an extension of Likelihood Ratio Test that allows to compare non nested models. Besides, there is no difference between Likelihood Ratio Test and Vuong test for nested models.
Voung test has been only available for classical models for years. The dependence of Level 1 units blocks application of Vuong test for multilevel models. Moreover, the lack of independence blocks Likelihood Ratio Test implementation too. However, Likelihood Ratio Test is usually applied through anova function in R.
Recently Vuong test has been implemented for SEM and Multilevel models in the nonnest2 R package. The nonnest2 package version of Vuong test, in the case of multilevel models, overcome the problem of lack of independence of level 1 units. For this reason, Vuong test as implemented in nonnest2 might be more efficient than anova function when comparing nested multilevel models if the sample is clearly dependent.
The objective of this work is to contrast the efficiency of Voung test with that of Likely Ratio Test for nested model comparison. We will try to answer the following question: Under which conditions does Vuong test through nonnest2 perform better that Likelihood Ratio Test through anova R function.
To achieve that aim, a simulation study will be performed for nested multilevel models’ comparison. Different sample sizes of level 1 and level 2, inter group and residual variance and effect size of the predictors will be simulated as conditions of the study.
The null model, a model with a predictor and a model with two independent variables, one of them non significative will be estimated.
The simulation study is being developed now, and though results are not available yet, we will dispose them in July 2025. We have developed a preliminary pilot study that suggest that Likelihood Ratio Test for dependent data is less efficient as the sample size increases. Once the simulation study will be done Voung test as implemented in nonnest2 might achieve high proficiency while Likelihood Ratio Test implemented through anova function fails.

Presentation materials

EAM2025-VUONG TEST 5.pdf

EAM2025-VUONG TEST 5.pptx