RIOT Science Club

Theory building and testing in psychological research by Dr Eiko Fried
2020-10-29 14:00 - 15:00 (Europe/London) | MS Teams
About the speaker
Dr Eiko Fried is an Assistant Professor in clinical psychology at Leiden University working in the fields of clinical psychology, psychiatry, epidemiology and methodology. His main focus is on studying individual symptoms of mental disorders and their causal relations. Broader, his interests include measurement (how to best assess whether someone is ill), modelling (what statistical models are most appropriate to model psychopathology), ontology (what are mental disorders) and nosology (how do we best classify them).
About the talk
The last decade has brought reforms to improve methodological practices, with the goal to increase the reliability and replicability of effects. However, explanations of effects remain scarce, and a growing chorus of scholars argues that the replicability crisis has distracted from a crisis of theory. In the same decade, the empirical literature using factor and network models has grown rapidly. In this talk, I discuss three ways in which this literature falls short of theory building and testing. First, statistical and theoretical models are conflated, leading to invalid inferences such as the existence of psychological constructs based on factor models, or recommendations for clinical interventions based on network models. I demonstrate this inferential gap in a simulation study on statistical equivalence: excellent model fit does little to corroborate a theory, regardless of quality or quantity of data. Second, researchers fail to explicate theories about psychological constructs, but use implicit causal beliefs to guide inferences. These latent theories have led to problematic best practices in psychological research where inferences are drawn based on one specific causal model that is assumed, but not explicated. Third, explicated theories are often weak theories: narrative and imprecise descriptions vulnerable to hidden assumptions and unknowns. They fail to make clear predictions, and it remains unclear whether statistical effects corroborate such theories or not. Weak theories are immune to refutation or revision. I argue that these three challenges to theory building and testing are common and harmful, and impede theory formation, failure, and reform. A renewed focus on theoretical psychology and formal models offers a way forward.
Related preprint: https://psyarxiv.com/zg84s/

A walk in the garden of forking paths by Dr Julia Rohrer
2020-10-22 12:00 - 13:00 (Europe/London) | MS Teams
About the speaker
I am a personality psychologist by training and my work covers a broad range of topics, including the effects of birth order, age patterns in personality, and the correlates and determinants of subjective well-being. My methodological interests include causal inference on the basis of observational data and data analytic flexibility. I am an active advocate for increased research transparency and have frequently given talks on the topic.
I recently finished my doctoral degree as a fellow of the International Max Planck Research School on the Life Course and am now a lecturer (Akademische Assistentin) at the Department of Psychology, University of Leipzig.
Together with Anne Scheel, Malte Elson, and Ruben Arslan, I blog at The 100% CI. If you want to get in touch with me, I’m fairly active on Twitter.
To learn more, go to https://juliarohrer.com/.
About the talk
Data analysis requires researchers to make many decisions — and sometimes, they may not know which choices are most appropriate. In this talk, I will give an overview of ways to tackle researcher degrees of freedom in a transparent manner (such as robustness checks, multiverse and specification curve analyses), highlight their commonalities, and discuss some crucial concerns.

Easing into open science: A guide for graduate students and their advisors by Dr Priya Silverstein
2020-10-15 12:00 - 13:00 UTC | MS Teams
About the speaker
Dr Priya Silverstein is a postdoctoral researcher in the CoGDeV Lab at the University of Surrey, studying the relationship between Lego construction ability, spatial thinking, and numeracy achievement in children. She is passionate about methods for studying development, and how we can do better, more transparent, reproducible, and diverse science.
About the talk
This talk provides a roadmap for engaging in open science practices, suggesting eight open science practices that novices could begin adopting today. The topics that will be covered include journal clubs, project workflow, preprints, reproducible code, data sharing, transparent writing, preregistration, and registered reports. To address concerns about not knowing how to engage in open science practices, we provide a difficulty rating of each behaviour (easy, medium, difficult), present them in order of suggested adoption, and follow the format of what, why, how, and worries. We emphasise that engaging in open science behaviours need not be an all or nothing approach and that you can engage with any number of the behaviours outlined. We also discuss the specific benefits of engaging with these practices now, at a time when a lot of our research has been disrupted.

Five selfish reasons to work reproducibly by Dr Florian Markowetz
2020-10-08 14:00 - 15:00 UTC | MS Teams
About the speaker
Florian Markowetz is a Senior Group Leader at the Cancer Research UK Cambridge Institute. He is a Royal Society Wolfson Research Merit Award holder and received a CRUK Future Leader in Cancer Research prize. He holds degrees in Mathematics (Dipl. math.) and Philosophy (M.A.) from the University of Heidelberg and a Dr. rer. nat. (PhD equivalent) in Computational Biology from Free University Berlin, for which he was awarded an Otto-Hahn Medal by the Max Planck Society. His group at the CRUK Cambridge Institute combines computational work on cancer evolution and image analysis of the tumour tissue with experimental work on understanding key cancer mechanisms like the estrogen receptor.
About the talk
And so, my fellow scientists: ask not what you can do for reproducibility; ask what reproducibility can do for you! In my talk, I will present five reasons why working reproducibly pays off in the long run and is in the self-interest of every ambitious, career-oriented scientist.

Why p-values can't tell you what you need to know and what to do about it by Prof David Colquhoun
2020-10-01 12:00 - 13:00 UTC | MS Teams
About the speaker
Professor David Colquhoun is a pharmacologist notable for his insights into the biophysics of drug–receptor interactions of single ion channels. He was elected a Fellow of the Royal Society (FRS) in 1985, was awarded the Humboldt Prize in 1990 and made Honorary Fellow of UCL in 2004. Colquhoun joined the Pharmacology Department at UCL in 1964, and, apart from 9 years, has remained there ever since. He can be found on twitter (@david_colquhoun) and via his website DC’s Improbable Science, where he writes critically about pseudoscience and alternative medicine.
About the talk
It is a truth universally acknowledged that the literature has too many false positives. One reason, among many, for this is misunderstanding of p-values. There are three main reasons why a p-value can’t tell you much about whether a hypothesis is true or not. It transposes the conditional, It has the wrong denominator. It takes into account values that have not been observed. These will be explained, and suggestions made about possible solutions.

Pitfalls of functional MRI by Dr Marion Criaud
2020-09-24 13:00 - 14:00 UTC | MS Teams
About the speaker
Dr Marion Criaud is a postdoctoral research associate in the Child and Adolescent Psychiatry department at King’s College London. Her current work focuses on developing a new non-medical treatment for children with ADHD called fMRI NeuroFeedback. She has a keen interest in identifying best practice in fMRI and understanding the limitations when applied in psychiatry.
About the talk
Functional MRI is not highly reproducible, and a lot of factors are influencing this. Some we can control, and some we can’t. fMRI data are extremely sensitive; we cannot always control the quality. But there are things we can do to improve reliability in fMRI. We are going to go over the fragility of fMRI data and what we can do to ensure the data is managed in the best way possible.

Synthetic data: A primer by Dr Dan Quintana
2020-07-30 12:00 - 13:00 UTC | MS Teams
About the speaker
Daniel S. Quintana is a researcher in biological psychiatry at the University of Oslo. His research interests include social behaviour, the oxytocin system, heart rate variability, and meta-analysis. He currently leads a project investigating the role of the oxytocin system in the development of severe mental illness and metabolic disorder, which is supported by an Excellence Project for Young Researchers grant from the Novo Nordisk Foundation. Dan is also known for the Everything Hertz podcast, which he co-hosts with James Heathers. Widely praised for his training materials to support the adoption of open research practices, Dan is a strong advocate for disseminating scholarly outputs with the aid of social media, which was the subject of his talk for the Rotterdam RIOTS Club in May.
About the talk
Open research data provide considerable scientific, societal, and economic benefits. However, disclosure risks can sometimes limit the sharing of open data, especially in datasets that include sensitive details or information from individuals with rare disorders. This talk introduces the concept of synthetic datasets, which is an emerging method originally developed to permit the sharing of confidential census data. Synthetic datasets mimic real datasets by preserving their statistical properties and the relationships between variables. Importantly, this method also reduces disclosure risk to essentially nil as no record in the synthetic dataset represents a real individual. This practical guide with accompanying R script enables biobehavioural researchers to create synthetic datasets and assess their utility via the synthpop R package. By sharing synthetic datasets that mimic original datasets that could not otherwise be made open, researchers can ensure the reproducibility of their results and facilitate data exploration while maintaining participant privacy.

Collaborative Assessment for Trustworthy Science: the repliCATS project by Prof Fiona Fidler
2020-07-23 11:00 - 12:00 UTC | MS Teams
About the speaker
Fiona has a degree in Psychology, with a second major in Sociology, and was awarded her PhD in History and Philosophy of Science at the University of Melbourne in 2006. She is now a Professor at the University of Melbourne, based between the School of BioSciences and the School of Historical and Philosophical Studies. Fiona is also an Australian Research Council Future Fellow, leading a range of meta-research projects across ecology and conservation science, as well as projects in psychology, and other social science fields. Her meta-research work is driven by an underlying interest in how scientists and other experts reason, make and justify decisions, and change their minds. She is co-lead (with Simine Vazire) of MetaMelb, a recently launched meta-science/meta-research group at the University of Melbourne, and she leads the repliCATS project, for Collaborative Assessments for Trustworthy Science.
About the talk
The repliCATS project evaluates published scientific research. As the acronym—Collaborative Assessments for Trustworthy Science—suggests, repliCATS is a group activity, centred around assessing the trustworthiness of research claims. Reviewers first make private individual assessments about a research claim—judging its comprehensibility, the prior plausibility of underlying effect, and its likely replicability. Reviewers then share their judgements and reasoning with group members, providing both new information and the opportunity for feedback and calibration. The group interrogates differences in opinion and explores counterfactuals. After discussion, there is a final opportunity for privately updating individual judgements. Importantly the repliCATS process is not consensus-driven – reviewers can disagree, and their ratings and probability judgements are mathematically aggregated into a final assessment. At the moment, the repliCATS platform exists primarily to predict replicability. Launched in January 2019 as part of the DARPA SCORE program, over 18 months repliCATS elicited group assessments and captured associated reasoning and discussion, for 3,000 published social scientific research claims in 8 disciplines (Business, Criminology, Economics, Education, Political Science, Psychology, Public Administration, and Sociology). The repliCATS team are now working to extend the platform beyond merely predicting replicability, to deliver a more comprehensive peer review protocol. Suspected advantages of a repliCATS process over traditional peer review include: inbuilt training and calibration; feedback that is intrinsically rewarding; an inherently interactive process, but one which does not implicitly rely on ‘consensus by fatigue’; and a process that actively encourages interrogation. This talk will present some preliminary findings, and discuss the future of the platform.

Beyond authorship: Introducing the Contributor Role Taxonomy (CRediT) by Dr Liz Allen
2020-07-16 12:00 - 13:00 UTC | MS Teams
About the speaker

Liz Allen is Director of Strategic Initiatives at F1000Research. Prior to joining F1000Research in 2015, she spent over a decade leading the Evaluation Team at Wellcome. Liz is a Visiting Senior Research Fellow in the Policy Institute at King’s College London, with a particular interest in science policy and research-related indicators. Liz is a Crossref Board Director, co-founder of CRediT (Contributor Roles Taxonomy), and serves on the Advisory Board for the Software Sustainability Institute.
About the talk
Original research articles with one author – particularly in the life sciences – are increasingly rare, and the concept of ‘authorship’ in science has become outdated and often unhelpful. CRediT was developed through a collaboration of research funders, publishers, researchers and institutions with the aim was to provide greater granularity, visibility and transparency to the myriad of contributions that researchers make to published scholarly research output today.
This talk explains the origins, rationale and current use of the Contributor Role Taxonomy (CRediT) – leaving room for discussion of the challenges and opportunities for the future of this initiative.

Why preclinical research should embrace Open Science by Dr Emily Sena
2020-07-09 12:00 - 13:00 UTC | MS Teams
About the speaker
Emily is a Stroke Association Kirby Laing Foundation Senior Non-Clinical Lecturer at the University of Edinburgh. She is specialised in the validity of preclinical research. Her interests are in the use of meta-research approaches (research on research) to drive improvements in the validity, transparency and reproducibility of primary research using animal models of human diseases. Her work has informed laboratory practice guidelines, editorial policy and clinical trials design. Emily is the inaugural Editor-in-Chief of BMJ Open Science and convenor of CAMARADES.
About the talk
Laboratory experiments of the life sciences are often conducted with the aim of improving human health. The translation of findings observed in preclinical animal studies to humans in a clinical setting has proven difficult in many areas. Discrepancies between the results of preclinical animal studies and human clinical trials, and limited reproducibility between laboratories, has in part been attributed to compromised internal and external validity of animal experiments, and the presence of publication bias. Meta-research approaches, including, systematic review and meta-analysis of preclinical studies have proven to be useful tools in quantitatively estimating the impact of study quality and informing the design of clinical trials. The use and implementation of open science approaches may improve the quality and transparency of preclinical research but it is important the effect of these tools are assessed.
If preclinical studies are used to inform future research decisions in the life sciences their design, conduct and reporting must be rigorous and their results disseminated in an unbiased and timely manner. Improving our approach to preclinical practice and evidence based clinical trial design may improve translation from bench to bedside.

How computational modelling can force theory building in psychological science by Dr Olivia Guest
2020-07-02 12:00 - 13:00 UTC | MS Teams
About the speaker

Olivia Guest is a computational cognitive modeller at RISE in Cyprus and University College London (UCL) in the UK. She emigrated to the UK to pursue a BSc in Computer Science at the University of York (2009), with an eye towards research at the intersection of AI and Psychology. She followed her undergraduate work with an MSc in Cognitive and Decision Sciences at UCL (2010) and a PhD in Psychology at Birkbeck (2014) where she focused on modelling semantic cognition using deep and shallow neural networks. Since then she has continued research in the area of categorisation, and conceptual representation broadly, using various modelling techniques — such as deep and shallow neural networks — with postdoctoral positions at the University of Oxford (2014–2016) and UCL (2016–2019). She was exposed to open communities in her teen years, e.g. open-source and open licensing, and believes modellers can have a doubly important role to play in guiding and enacting useful changes in open cognitive science: firstly, from experience with the open-source community and secondly, from experience navigating interdisciplinary settings. She is an editor-in-chief at ReScience C and a topic editor at the Journal of Open Source Software. She is committed to equity, diversity, and inclusion in (open) science, e.g., promoting access to technical skills training. In addition, Christina Bergmann and Olivia try to maintain a list of underrepresented cognitive computational scientists at compcog.science.
You can find more information about Olivia by clicking here.
About the talk
Psychology endeavours to develop theories of human capacities and behaviours based on a variety of methodologies and dependent measures. We argue that one of the most divisive factors in our field is whether researchers choose to employ computational modelling of theories (over and above data) during the scientific inference process. Modelling is undervalued, yet holds promise for advancing psychological science. The inherent demands of computational modelling guide us towards better science by forcing us to conceptually analyze, specify, and formalise intuitions which otherwise remain unexamined — what we dub “open theory”. Constraining our inference process through modelling enables us to build explanatory and predictive theories. Herein, we present scientific inference in psychology as a path function, where each step shapes the next. Computational modelling can constrain these steps, thus advancing scientific inference over and above stewardship of experimental practice (e.g. preregistration). If psychology continues to eschew computational modelling, we predict more replicability “crises” and persistent failure at coherent theory-building. This is because without formal modelling we lack open and transparent theorising. We also explain how to formalise, specify, and implement a computational model, emphasizing that the advantages of modelling can be achieved by anyone with benefit to all.
You can read more about those issues here.

Reproducibility in neuroimaging: Problems and solutions by Prof Russell Poldrack
2020-06-25 18:00 - 19:00 UTC | MS Teams
About the speaker
Russell A. Poldrack is the Albert Ray Lang Professor in the Department of Psychology and Professor (by courtesy) of Computer Science at Stanford University, and Director of the Stanford Center for Reproducible Neuroscience. His research uses neuroimaging to understand the brain systems underlying decision making and executive function. His lab is also engaged in the development of neuroinformatics tools to help improve the reproducibility and transparency of neuroscience, including the Openneuro.org and Neurovault.org data sharing projects and the Cognitive Atlas ontology.
About the talk
There is increasing concern regarding the reproducibility and generalisability of neuroimaging research. Prof Poldrack will outline the problem and discuss a number of important causes, including low statistical power, analytic flexibility, and the lack of robust software engineering practices. He will also outline a set of community efforts that are trying to address these challenges.

How reliable is fMRI, and does it actually matter? by Dr Matt Wall
2020-06-18 12:00 - 13:00 UTC | MS Teams
About the speaker
Dr Matt Wall completed his PhD in Cambridge, then did post-doctoral positions at Royal Holloway and UCL before selling out and moving into the private sector. He currently works for Invicro, a global imaging company that provides research services for the pharmaceutical and biotechnology industries. His research interests have meandered around somewhat, taking in cognitive psychology, visual psychophysics, pain, psychopharmacology, psychedelics, addiction, neuroendocrinology, and methods development, but all focussed around the use of fMRI. His current collaborative projects involve giving teenagers cannabis and showing women pornography, as well as a number of commercial studies, and ongoing methodological work.
About the talk
fMRI is an established workhorse method in modern cognitive neuroscience but relatively little attention has been paid to characterising its basic features, in particular, the closely related concepts of reliability, replicability, and reproducibility. I’ll discuss some recent work we’ve done examining test-retest reliability metrics on a range of different cognitive and sensory task paradigms. We’ve also examined the effects of different de-noising procedures on these reliability metrics in the same set of data, as well as in resting-state fMRI data. There has been a great deal of discussion in recent years on improving power, reliability, and replicability in neuroscience, but which of these is the most important, and which are we actually able to change? Does reliability actually matter all that much, and if it does, how do we get people to care about it?

Leadership & research integrity by Nadia Soliman
2020-06-04 12:00 - 13:00 UTC | MS Teams
About the speaker

With a BSc (Hons) Pharmacology (University of Liverpool), Nadia has returned to academia having served in both the regular and reserve of the British Army for over ten years. Since returning she has gained an MSc in Drug Discovery Skills (KCL) and MRes in Cellular and Molecular Biosciences and is in her second year of a PhD at Imperial College London. Her PhD is focused on developing automation technologies to improve the feasibility, efficiency and accuracy of preclinical systematic reviews while addressing neurobiological questions of interest. She has an interest in sharing her military experiences and knowledge of leadership development to engender a more positive research culture.
About the talk

Nadia will share she has learned about leadership and leadership development from the Army. She will then go on to discuss what we value in academia and whether our behaviours are congruent with upholding our values and demonstrate that good leadership is critical to conducting science that is reproducible, interpretable, open and transparent. Lastly, she will ask us to reflect on our roles and responsibilities and our agency to make and influence change.

The new heuristics: Jumping through hoops instead of improving our science by Prof Daniel Lakens
2020-05-28 12:00 - 13:00 UTC | MS Teams
About the talk
Psychological science has been at the forefront of improving research practices. Yet, psychology is also a strongly norm driven field, and we risk replacing old norms with new norms, without increasing true understanding or the ability to justify our actions. I will unsuccessfully try to prevent you from just adopting the New Heuristics of scientific reform.

About the speaker

Professor Daniel Lakens is based at Human-Technology Interaction group at Eindhoven University of Technology. His research focuses on how to design and interpret studies, applied (meta)-statistics, and reward structures in science, as well as having research interests in conceptual thought and meaning. Daniel is noted for his teaching and creation of useful resources. He received the 2017 Leamer-Rosenthal prize for Open Social Science as a Leader in Education, and his course on research methods for young scholars (here) is widely praised and highly subscribed, along with his blog on methods and statistics and practical primers on effect sizes, sequential analysis, and equivalence tests. Recently, Daniel has developed an interest in the importance of (preferably pre-registered) replications and ways to improve how we interpret and design studies. Daniel believes that we can try a little harder to make science as open and robust as possible, and give the tax payer as much value for money as we can, and that science should be a much more collaborative enterprise (see his TEDx talk on this topic here).

Fast-tracked COVID-19 research: our experience by Prof Mitul Mehta & Dr Ndaba Mazibuko
2020-05-21 12:00 - 13:00 UTC | MS Teams
About the speakers
Professor Mitul Mehta
Professor Mitul Mehta graduated from Cambridge University in Natural Sciences, where he also completed his PhD. An MRC Fellowship to train in positron emission tomography at Imperial College was followed by a Wellcome Trust Award to continue his research using magnetic resonance imaging at King’s. His investigations on the dopamine system and human cognitive function earned him the British Association for Psychopharmacology (BAP) young investigator award in 2005. He has subsequently applied his research to assess novel drugs to improve brain function, and studies the role of acquisition and analysis methods in pharmacological imaging. His group have developed pipelines suitable for assaying novel compounds and he has worked with numerous companies in this work. He is currently Head of the Neuropharmacology group in the Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, Director of the newly formed Centre for Innovative Brain Therapeutics, Deputy lead for Neuroimaging in the NIHR-BRC for Mental Health and Meetings Secretary for the BAP.
Dr Ndaba Mazibuko
Dr Ndaba Mazibuko is a Clinical Research Fellow in Clinical Neuroscience (Institute of Psychiatry, Psychology and Neuroscience – King’s College London). His work includes clinical trial design and management with respect to pharmacological and neuroimaging studies including a role as a Medical Principal Investigator on a recent academic / industry clinical trial collaboration. He provides clinical oversight for the broad portfolio of the Department of Neuroimaging studies. His work has honed in on drug repurposing translation. He has recently joined the Centre for Innovative Therapeutics (C-FIT) as Repurposing Lead. Prior to clinical academia, he worked as a Neurology and Acute Stroke Registrar at King’s College Hospital and has several years’ experience as a clinician in varied disciplines within the NHS in addition to clinical experience in Australia (where he trained) and also in Africa.
About the talk
During the current pandemic there has been an opportunity to develop research studies rapidly. This is necessarily opportunistic and for studies in patients the government has set up systems of prioritisation. Our experience of setting up two studies highlights some positives in terms of collegiate behaviour and collaboration potential. We also have acute experiences of the negative effects of fast-tracking in the UK. The desire for rapid answers, at least for treatments, has, we believe, created a paradoxical scenario whereby studies that can report rapidly are being delayed or even being denied access to patients. We will also talk about the events that have led to card-carrying brain researchers to spend so much time thinking about respiratory disease.

The generalizability crisis by Dr Tal Yarkoni
2020-05-14 15:00 - 16:00 UTC | MS Teams
About the speaker
Tal Yarkoni a Research Associate Professor in the Department of Psychology at the University of Texas at Austin. His academic work focuses on developing new tools and methods for the analysis of psychology and neuroimaging data. Tal mostly builds open-source software tools and writes methods-y papers about reproducibility, data standards, and best practices. He also has a background in substantive areas of psychology, including personality, executive control, and psycholinguistics.
About the talk
Most theories and hypotheses in psychology are verbal in nature, yet their evaluation overwhelmingly relies on inferential statistical procedures. The validity of the move from qualitative to quantitative analysis depends on the verbal and statistical expressions of a hypothesis being closely aligned—that is, that the two must refer to roughly the same set of hypothetical observations. Here I argue that most inferential statistical tests in psychology fail to meet this basic condition. I demonstrate how foundational assumptions of the “random effects” model used pervasively in psychology impose far stronger constraints on the generalizability of results than most researchers appreciate. Ignoring these constraints dramatically inflates false positive rates and routinely leads researchers to draw sweeping verbal generalizations that lack any meaningful connection to the statistical quantities they are putatively based on. I argue that the routine failure to consider the generalizability of one’s conclusions from a statistical perspective lies at the root of many of psychology’s ongoing problems (e.g., the replication crisis), and conclude with a discussion of several potential avenues for improvement.

Moral overreach and ethical failure in setting psychological research standards by Prof Roger Giner-Sorolla
2020-05-07 12:00 - 13:00 UTC | MS Teams
About the speaker
Professor Roger Giner-Sorolla completed his undergraduate degree at Cornell University and was awarded his PhD in Social Psychology from New York University in 1996. After a postdoctoral fellowship at the University of Virginia and two-year-long contracts, he joined the University of Kent in 2001. He was promoted to Professor in 2013 and has also been Editor-in-Chief of the Journal of Experimental Social Psychology since 2016. Roger is an advocate of transparency in scientific reporting and has written several articles and editorials in support of improved reporting guidelines and pre-registration. He has taught Master’s statistics and methodology since 2001 at Kent. He is a Fellow of the Association for Psychological Science and a member of the Society for Experimental Social Psychology.

About the talk

Recent discussions about research methods and reporting reform in psychology have often taken on a moralized tone. I will argue that any appeal to morality needs to be well-informed about the potential contradictions in open and robust standards, showing a number of examples in which simplistic application leads reformers astray. I will also analyse the apparent moral failure to translate decades-old APA policy about ethical results reporting into workable, enforced journal policy, and point to some recent apparent successes. Instead of easy heuristics or unrealistic standards I advocate a focus on actionable changes that maximise the credibility of research.

Laboratory Efficiency Assessment Framework (LEAF) by Dr Martin Farley
2020-04-30 12:00 - 13:00 UTC | MS Teams
About the speaker
Raised between France and the US, Martin started down the path of becoming a researcher in his university years, taking him to Canada, the US, and the Netherlands. This path was interrupted though when his passion for sustainability led him to become the UK’s (and Europe’s) first full-time sustainable laboratory specialist at the University of Edinburgh. After he moved south to London where he founded Green Lab Associates, while establishing the sustainable labs program at King’s College London (where he was recognized with the Green Gown – Sustainability Professional award). Today he consults for a variety of institutions, whilst regularly speaking and teaching on the topic.
About the talk
In this talk, Martin will go over The Laboratory Efficiency Assessment Framework (LEAF) which contains criteria surrounding improving the sustainability and efficiency of laboratories, as well as calculators which allow users to estimate impacts. LEAF includes content to improve research integrity and is supported by the UK Reproducibility Network. This has been done in recognition that research which is not reproducible is unsustainable (waste of resources and effort) and that good practice steps can improve reproducibility. Universities which participated in LEAF’s pilot include Cambridge, UCL, King’s College London, Imperial, Edinburgh, Glasgow, Swansea, The Crick Institute, Manchester, and more.

Octopus by Dr Alex Freeman
2020-04-23 12:00 - 13:00 UTC | MS Teams
About the speaker
Before joining the Winton Centre in 2016, Alex Freeman had a 16-year career at the BBC, working on series such as Walking with Beasts, Life in the Undergrowth, Bang Goes the Theory, Climate Change by Numbers and as series producer of Trust Me, I’m a Doctor. Her work won a number of awards, from a BAFTA to a AAAS Kavli gold award for science journalism. In addition to developing and making television series, Alex worked with associated content across a whole range of other media – designing websites, games, formal learning resources and social media content – to bring science to the widest possible audience. Now back at the Winton Centre she has a particular interest in helping professionals such as doctors, journalists or legal professionals communicate numbers and uncertainty better, and in whether narrative can be used as a tool to inform but not persuade. She is an advocate of open research practices and the reform of the science publishing system.
About the talk
Alex is going to talk about Octopus – her idea to create a new, dedicated primary research record, designed to incentivise good research practices and improve the scientific process. Instead of the traditional ‘paper’ being both the way to announce or ’sell’ conclusions and the primary record of exact, replicable methods and data, Alex will show how separating the latter into Octopus and breaking down the unit of publication from a paper into smaller publications could change the incentive structure, speed up research and hence revolutionise science. She will also, hopefully, demonstrate it as a working platform for the first time (hot-off-the-press)!

Registered Reports & F1000 by Emma Henderson & Demitra Ellina
2020-04-16 12:00 - 13:30 UTC | MS Teams
About the speakers
Emma Henderson
Emma is studying for her PhD in Psychology at Kingston University. She is interested in the use of heuristics in every day judgements and decisions. She is also interested in psychological methods, meta-science and philosophy of science. Emma is passionate about improving the reliability of scientific research through using and promoting open science practices including pre-registration and open data. She is an Ambassador for the Center for Open Science, a community committee member for Open Thesis.
Demitra Ellina
Demitra Ellina is the Editorial Community Manager at F1000 which she joined in 2017 after working in journal development at Springer Nature. She is a strong advocate of Open Research and engages with the research community to raise awareness of the F1000 publishing platforms.
About the talk
Emma will give a presentation entitled Ten reasons to write a Registered Report (now). Registered Reports (RR), a type of empirical article that is published based on the scientific merit of a preregistered protocol. Specifically, before results are known, and in principle acceptance is given to a prospective study and/or analysis plan following successful peer-review. Providing that the authors closely follow the protocol, the final submitted manuscript is published: regardless of the results. Emma will expand on this description, provide examples, and give an overview of the impact RRs have had since their introduction in 2013 by the journal Cortex.
The talk will be a ~30-minutes pre-recording followed by a live Q & A.
Demitra will give a live 10-minute talk on the wonderful work she does at F1000Research followed by a Q & A. She helps researchers understand Open Science practices and reproducibility at F1000Research, which is the first publisher to integrate the Registered Report format using an open post-publication peer review model (the first RR we published is linked here). Alongside their open data policy, this format helps enhance credibility while reducing researcher bias and supports reproducibility.

Fetishisation of RCTs by Professor David Papineau
2020-04-09 12:00 - 13:00 UTC | Zoom, United Kingdom
About the speaker
Professor David Papineau is a British academic philosopher who has worked in metaphysics, epistemology, and the philosophies of science, mind, and mathematics. He is one of the originators of the teleosemantic theory of mental representation, a solution to the problem of intentionality which derives the intentional content of our beliefs from their biological purpose. In 1990, David joined the Department of Philosophy at King’s College London as Professor of Philosophy of Science.
About the talk
Many medical practitioners view Randomized Controlled Trials as the only good way to establish causal conclusions. Not only is this a methodological mistake, but it fosters the view that the end of greater medical knowledge justifies the means of patient deception. This talk will give examples.

Model selection using information criteria by Prof Daniel Stahl
2020-04-02 12:00 - 13:00 UTC | MS Teams
About the speaker
Prof Daniel Stahl is lead of the precision medicine and statistical learning research group in the Department of Biostatistics and Health Informatics. His main research interest is applying statistical and machine learning methods to identify predictors, mediators and moderators of treatment success and the development of robust risk prediction models.
About the talk
Prof Stahl’s talk will discuss model fitting problems, focusing on ways to select your analytical model using information criteria.

Jasp and Jamovi by Dr Samuel Westwood
2020-03-30 12:00 - 13:00 UTC | MS Teams
About the speaker
Samuel Westwood was awarded his PhD in Neuroscience from Aston University, Birmingham in 2018. He joined King’s College London in 2017 as a post-doctoral research associate in the Child & Adolescent Psychiatry Department and is investigating the effectiveness of combining cognitive training and non-invasive brain stimulation in improving inattention and self-control in boys with ADHD. He founded and co-organised the RIOT Science Club and the King’s Open Research Group Initiative. Sam is also the local lead for the UK Reproducibility Network.
About the talk
Dr Samuel Westwood gives a talk on Jasp and Jamovi, two graphic user interfaces (GUI) for R, which make statistical analyses and data visualisation simple, quick, and open to all. Sam shows us the advantages of these GUIs compared to SPSS and R, while providing useful examples on how to perform several statistical analyses and create graphical data representations.

Big data and mental health by Dr Johnny Downs
2020-03-12 12:00 - 13:00 UTC | Institute of Psychiatry, Psychology & Neuroscience (IoPPN), 16 De Crespigny Park, London, London, SE5 8AF, United Kingdom
About the speaker
The research of Dr Johnny Downs focuses on combining epidemiological approaches with health informatics to improve outcomes for childhood neurodevelopmental and mental health disorders. His interests lie within several main areas – implementation of patient portals to enhance patient care, natural language processing to refine the data extracted from free-text electronic records, and data-linkages which integrate non-health data into clinical record analyses.
About the talk
Dr Downs will talk about the uses of big data in mental health research.

Meta-analysis in R by Dr Isabel Yorke
2020-02-27 12:00 - 13:00 UTC | Institute of Psychiatry, Psychology & Neuroscience (IoPPN), 16 De Crespigny Park, London, London, SE5 8AF, United Kingdom
About the speaker
Dr Isabel Yorke completed her MSc at the SGDP, and was awarded her PhD investigating environmental factors that contribute to additional mental health problems in people with ASD.
About the talk
Dr Yorke will give a basic introduction to key meta-analytic techniques, using the R package Metafor. She will cover organising input data, calculating/converting between effect sizes, applying models to pool effect sizes, producing plots (e.g. forest and funnel plots) and (time permitting) meta-regression, subset analysis, sensitivity analysis and interpreting results from these functions. We will use examples from meta-analyses of correlations and of group differences.
Disclaimer from Dr Yorke: I will be showing you various meta-analytic methods that I am familiar with. I am by no means a meta-analysis or R expert! There are surely alternative, possibly even better, ways of doing the things I will show you – but these are ways that I have found worked for my purposes. Please feel free to ask questions or make suggestions – we can discuss limitations and merits of different methods and I’m sure I will learn from the session too.

Open science for early career researchers by Alexandra Lautarescu
2020-02-25 12:00 - 13:00 UTC | St Thomas’ Hospital, Westminster Bridge Road, London, SE1 7EH, United Kingdom
About the speaker
Alexandra Lautarescu is a PhD student based in the Department of Perinatal Imaging and Health at St Thomas’ Hospital. She earned a BSc(Hons) in Psychology from University of Exeter in 2011, followed by an MPhil in Medical Sciences (Psychiatry) from the University of Cambridge. Her current research focuses on maternal mental health during pregnancy and its relationship with fetal and neonatal brain development, as part of the developing Human Connectome Project. She is a co-founder of the Postdocalypse Podcast and a Graduate Teaching Assistant on the BSc Psychology programme at King’s College London.
About the talk
Alexandra Lautarescu gives a talk on how early career researchers can implement open science practices in their workflow, ensuring that reproducibility and transparency are embedded in the research process.

The unconscious mind by Prof David Shanks
2020-02-20 12:00 - 13:00 UTC | Institute of Psychiatry, Psychology & Neuroscience (IoPPN), 16 De Crespigny Park, London, London, SE5 8AF, United Kingdom
About the speaker
David Shanks is Professor of Psychology and Deputy Dean of the Faculty of Brain Sciences at University College London (UCL), and a Fellow of the Academy of Social Sciences. Previously he was Head of Psychology at UCL from 2003-17. His research falls across a number of topics in experimental cognitive psychology, including learning, memory, judgment and decision-making, and he is the author (with Newell and Lagnado) of Straight choices: The psychology of decision making (2nd ed., 2015). A number of his early replication studies in the field of ‘social priming’ led to an enduring interest in and contribution to debates about research transparency.
About the talk
Professor Shanks talks about the important role in the reproducibility crisis played by the unchallenged, pervasive and probably false conception of the unconscious mind which dominates much thinking in psychology and neuroscience.

Is there a reproducibility crisis in science? by Dr Samuel Westwood
2020-02-19 12:00 - 13:00 UTC | St Thomas’ Hospital, Westminster Bridge Road, London, SE1 7EH, United Kingdom
About the speaker
Samuel Westwood was awarded his PhD in Neuroscience from Aston University, Birmingham in 2018. He joined King’s College London in 2017 as a post-doctoral research associate in the Child & Adolescent Psychiatry Department and is investigating the effectiveness of combining cognitive training and non-invasive brain stimulation in improving inattention and self-control in boys with ADHD. He founded and co-organised the RIOT Science Club and the King’s Open Research Group Initiative. Sam is also the local lead for the UK Reproducibility Network.
About the talk
Dr Samuel Westwood will give a talk on whether there is a reproducibility crisis in science, focusing on its causes: perverse incentives and questionable research practices.

The 7 plagues of neuroimaging studies by Prof Henning Tiemeier
2020-02-18 12:00 - 13:00 UTC | St Thomas’ Hospital, Westminster Bridge Road, London, SE1 7EH, United Kingdom
About the speaker
Henning Tiemeier is a Professor of Social and Behavioural Science and the Sumner and Esther Feldberg Chair of Maternal and Child Health at the Harvard T.H. Chan School of Public Health. Prof Tiemeier received both his medical and sociological degree from the University of Bonn, Germany, and his PhD from the Erasmus University in Rotterdam, Netherlands. Henning has published extensively on the aetiology of child developmental problems with a particular focus on prenatal exposures. Most of his research was performed in population-based cohort studies and his work often takes a neurodevelopmental approach. He is a principal investigator of the Generation R Study, a large pre-birth cohort in Rotterdam, that enrolled nearly 10,000 mothers and their children.
About the talk
Prof Tiemeier talks about some of the issues in neuroimaging research.
