Theory building and testing in psychological research by Dr Eiko Fried

About the speaker

Dr Eiko Fried is an Assistant Professor in clinical psychology at Leiden University working in the fields of clinical psychology, psychiatry, epidemiology and methodology. His main focus is on studying individual symptoms of mental disorders and their causal relations. Broader, his interests include measurement (how to best assess whether someone is ill), modelling (what statistical models are most appropriate to model psychopathology), ontology (what are mental disorders) and nosology (how do we best classify them).

About the talk

The last decade has brought reforms to improve methodological practices, with the goal to increase the reliability and replicability of effects. However, explanations of effects remain scarce, and a growing chorus of scholars argues that the replicability crisis has distracted from a crisis of theory. In the same decade, the empirical literature using factor and network models has grown rapidly. In this talk, I discuss three ways in which this literature falls short of theory building and testing. First, statistical and theoretical models are conflated, leading to invalid inferences such as the existence of psychological constructs based on factor models, or recommendations for clinical interventions based on network models. I demonstrate this inferential gap in a simulation study on statistical equivalence: excellent model fit does little to corroborate a theory, regardless of quality or quantity of data. Second, researchers fail to explicate theories about psychological constructs, but use implicit causal beliefs to guide inferences. These latent theories have led to problematic best practices in psychological research where inferences are drawn based on one specific causal model that is assumed, but not explicated. Third, explicated theories are often weak theories: narrative and imprecise descriptions vulnerable to hidden assumptions and unknowns. They fail to make clear predictions, and it remains unclear whether statistical effects corroborate such theories or not. Weak theories are immune to refutation or revision. I argue that these three challenges to theory building and testing are common and harmful, and impede theory formation, failure, and reform. A renewed focus on theoretical psychology and formal models offers a way forward.

Related preprint: https://psyarxiv.com/zg84s/