Fast-tracked COVID-19 research: our experience by Prof Mitul Mehta & Dr Ndaba Mazibuko
2020-05-21 12:00 - 13:00 UTC | MS Teams
About the speakers
Professor Mitul Mehta
Professor Mitul Mehta graduated from Cambridge University in Natural Sciences, where he also completed his PhD. An MRC Fellowship to train in positron emission tomography at Imperial College was followed by a Wellcome Trust Award to continue his research using magnetic resonance imaging at King’s. His investigations on the dopamine system and human cognitive function earned him the British Association for Psychopharmacology (BAP) young investigator award in 2005. He has subsequently applied his research to assess novel drugs to improve brain function, and studies the role of acquisition and analysis methods in pharmacological imaging. His group have developed pipelines suitable for assaying novel compounds and he has worked with numerous companies in this work. He is currently Head of the Neuropharmacology group in the Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, Director of the newly formed Centre for Innovative Brain Therapeutics, Deputy lead for Neuroimaging in the NIHR-BRC for Mental Health and Meetings Secretary for the BAP.
Dr Ndaba Mazibuko
Dr Ndaba Mazibuko is a Clinical Research Fellow in Clinical Neuroscience (Institute of Psychiatry, Psychology and Neuroscience – King’s College London). His work includes clinical trial design and management with respect to pharmacological and neuroimaging studies including a role as a Medical Principal Investigator on a recent academic / industry clinical trial collaboration. He provides clinical oversight for the broad portfolio of the Department of Neuroimaging studies. His work has honed in on drug repurposing translation. He has recently joined the Centre for Innovative Therapeutics (C-FIT) as Repurposing Lead. Prior to clinical academia, he worked as a Neurology and Acute Stroke Registrar at King’s College Hospital and has several years’ experience as a clinician in varied disciplines within the NHS in addition to clinical experience in Australia (where he trained) and also in Africa.
About the talk
During the current pandemic there has been an opportunity to develop research studies rapidly. This is necessarily opportunistic and for studies in patients the government has set up systems of prioritisation. Our experience of setting up two studies highlights some positives in terms of collegiate behaviour and collaboration potential. We also have acute experiences of the negative effects of fast-tracking in the UK. The desire for rapid answers, at least for treatments, has, we believe, created a paradoxical scenario whereby studies that can report rapidly are being delayed or even being denied access to patients. We will also talk about the events that have led to card-carrying brain researchers to spend so much time thinking about respiratory disease.

The new heuristics: Jumping through hoops instead of improving our science by Prof Daniel Lakens
2020-05-28 12:00 - 13:00 UTC | MS Teams
About the talk
Psychological science has been at the forefront of improving research practices. Yet, psychology is also a strongly norm driven field, and we risk replacing old norms with new norms, without increasing true understanding or the ability to justify our actions. I will unsuccessfully try to prevent you from just adopting the New Heuristics of scientific reform.

About the speaker

Professor Daniel Lakens is based at Human-Technology Interaction group at Eindhoven University of Technology. His research focuses on how to design and interpret studies, applied (meta)-statistics, and reward structures in science, as well as having research interests in conceptual thought and meaning. Daniel is noted for his teaching and creation of useful resources. He received the 2017 Leamer-Rosenthal prize for Open Social Science as a Leader in Education, and his course on research methods for young scholars (here) is widely praised and highly subscribed, along with his blog on methods and statistics and practical primers on effect sizes, sequential analysis, and equivalence tests. Recently, Daniel has developed an interest in the importance of (preferably pre-registered) replications and ways to improve how we interpret and design studies. Daniel believes that we can try a little harder to make science as open and robust as possible, and give the tax payer as much value for money as we can, and that science should be a much more collaborative enterprise (see his TEDx talk on this topic here).

Leadership & research integrity by Nadia Soliman
2020-06-04 12:00 - 13:00 UTC | MS Teams
About the speaker

With a BSc (Hons) Pharmacology (University of Liverpool), Nadia has returned to academia having served in both the regular and reserve of the British Army for over ten years. Since returning she has gained an MSc in Drug Discovery Skills (KCL) and MRes in Cellular and Molecular Biosciences and is in her second year of a PhD at Imperial College London. Her PhD is focused on developing automation technologies to improve the feasibility, efficiency and accuracy of preclinical systematic reviews while addressing neurobiological questions of interest. She has an interest in sharing her military experiences and knowledge of leadership development to engender a more positive research culture.
About the talk

Nadia will share she has learned about leadership and leadership development from the Army. She will then go on to discuss what we value in academia and whether our behaviours are congruent with upholding our values and demonstrate that good leadership is critical to conducting science that is reproducible, interpretable, open and transparent. Lastly, she will ask us to reflect on our roles and responsibilities and our agency to make and influence change.

How reliable is fMRI, and does it actually matter? by Dr Matt Wall
2020-06-18 12:00 - 13:00 UTC | MS Teams
About the speaker
Dr Matt Wall completed his PhD in Cambridge, then did post-doctoral positions at Royal Holloway and UCL before selling out and moving into the private sector. He currently works for Invicro, a global imaging company that provides research services for the pharmaceutical and biotechnology industries. His research interests have meandered around somewhat, taking in cognitive psychology, visual psychophysics, pain, psychopharmacology, psychedelics, addiction, neuroendocrinology, and methods development, but all focussed around the use of fMRI. His current collaborative projects involve giving teenagers cannabis and showing women pornography, as well as a number of commercial studies, and ongoing methodological work.
About the talk
fMRI is an established workhorse method in modern cognitive neuroscience but relatively little attention has been paid to characterising its basic features, in particular, the closely related concepts of reliability, replicability, and reproducibility. I’ll discuss some recent work we’ve done examining test-retest reliability metrics on a range of different cognitive and sensory task paradigms. We’ve also examined the effects of different de-noising procedures on these reliability metrics in the same set of data, as well as in resting-state fMRI data. There has been a great deal of discussion in recent years on improving power, reliability, and replicability in neuroscience, but which of these is the most important, and which are we actually able to change? Does reliability actually matter all that much, and if it does, how do we get people to care about it?

Reproducibility in neuroimaging: Problems and solutions by Prof Russell Poldrack
2020-06-25 18:00 - 19:00 UTC | MS Teams
About the speaker
Russell A. Poldrack is the Albert Ray Lang Professor in the Department of Psychology and Professor (by courtesy) of Computer Science at Stanford University, and Director of the Stanford Center for Reproducible Neuroscience. His research uses neuroimaging to understand the brain systems underlying decision making and executive function. His lab is also engaged in the development of neuroinformatics tools to help improve the reproducibility and transparency of neuroscience, including the Openneuro.org and Neurovault.org data sharing projects and the Cognitive Atlas ontology.
About the talk
There is increasing concern regarding the reproducibility and generalisability of neuroimaging research. Prof Poldrack will outline the problem and discuss a number of important causes, including low statistical power, analytic flexibility, and the lack of robust software engineering practices. He will also outline a set of community efforts that are trying to address these challenges.

How computational modelling can force theory building in psychological science by Dr Olivia Guest
2020-07-02 12:00 - 13:00 UTC | MS Teams
About the speaker

Olivia Guest is a computational cognitive modeller at RISE in Cyprus and University College London (UCL) in the UK. She emigrated to the UK to pursue a BSc in Computer Science at the University of York (2009), with an eye towards research at the intersection of AI and Psychology. She followed her undergraduate work with an MSc in Cognitive and Decision Sciences at UCL (2010) and a PhD in Psychology at Birkbeck (2014) where she focused on modelling semantic cognition using deep and shallow neural networks. Since then she has continued research in the area of categorisation, and conceptual representation broadly, using various modelling techniques — such as deep and shallow neural networks — with postdoctoral positions at the University of Oxford (2014–2016) and UCL (2016–2019). She was exposed to open communities in her teen years, e.g. open-source and open licensing, and believes modellers can have a doubly important role to play in guiding and enacting useful changes in open cognitive science: firstly, from experience with the open-source community and secondly, from experience navigating interdisciplinary settings. She is an editor-in-chief at ReScience C and a topic editor at the Journal of Open Source Software. She is committed to equity, diversity, and inclusion in (open) science, e.g., promoting access to technical skills training. In addition, Christina Bergmann and Olivia try to maintain a list of underrepresented cognitive computational scientists at compcog.science.
You can find more information about Olivia by clicking here.
About the talk
Psychology endeavours to develop theories of human capacities and behaviours based on a variety of methodologies and dependent measures. We argue that one of the most divisive factors in our field is whether researchers choose to employ computational modelling of theories (over and above data) during the scientific inference process. Modelling is undervalued, yet holds promise for advancing psychological science. The inherent demands of computational modelling guide us towards better science by forcing us to conceptually analyze, specify, and formalise intuitions which otherwise remain unexamined — what we dub “open theory”. Constraining our inference process through modelling enables us to build explanatory and predictive theories. Herein, we present scientific inference in psychology as a path function, where each step shapes the next. Computational modelling can constrain these steps, thus advancing scientific inference over and above stewardship of experimental practice (e.g. preregistration). If psychology continues to eschew computational modelling, we predict more replicability “crises” and persistent failure at coherent theory-building. This is because without formal modelling we lack open and transparent theorising. We also explain how to formalise, specify, and implement a computational model, emphasizing that the advantages of modelling can be achieved by anyone with benefit to all.
You can read more about those issues here.

Why preclinical research should embrace Open Science by Dr Emily Sena
2020-07-09 12:00 - 13:00 UTC | MS Teams
About the speaker
Emily is a Stroke Association Kirby Laing Foundation Senior Non-Clinical Lecturer at the University of Edinburgh. She is specialised in the validity of preclinical research. Her interests are in the use of meta-research approaches (research on research) to drive improvements in the validity, transparency and reproducibility of primary research using animal models of human diseases. Her work has informed laboratory practice guidelines, editorial policy and clinical trials design. Emily is the inaugural Editor-in-Chief of BMJ Open Science and convenor of CAMARADES.
About the talk
Laboratory experiments of the life sciences are often conducted with the aim of improving human health. The translation of findings observed in preclinical animal studies to humans in a clinical setting has proven difficult in many areas. Discrepancies between the results of preclinical animal studies and human clinical trials, and limited reproducibility between laboratories, has in part been attributed to compromised internal and external validity of animal experiments, and the presence of publication bias. Meta-research approaches, including, systematic review and meta-analysis of preclinical studies have proven to be useful tools in quantitatively estimating the impact of study quality and informing the design of clinical trials. The use and implementation of open science approaches may improve the quality and transparency of preclinical research but it is important the effect of these tools are assessed.
If preclinical studies are used to inform future research decisions in the life sciences their design, conduct and reporting must be rigorous and their results disseminated in an unbiased and timely manner. Improving our approach to preclinical practice and evidence based clinical trial design may improve translation from bench to bedside.

Beyond authorship: Introducing the Contributor Role Taxonomy (CRediT) by Dr Liz Allen
2020-07-16 12:00 - 13:00 UTC | MS Teams
About the speaker

Liz Allen is Director of Strategic Initiatives at F1000Research. Prior to joining F1000Research in 2015, she spent over a decade leading the Evaluation Team at Wellcome. Liz is a Visiting Senior Research Fellow in the Policy Institute at King’s College London, with a particular interest in science policy and research-related indicators. Liz is a Crossref Board Director, co-founder of CRediT (Contributor Roles Taxonomy), and serves on the Advisory Board for the Software Sustainability Institute.
About the talk
Original research articles with one author – particularly in the life sciences – are increasingly rare, and the concept of ‘authorship’ in science has become outdated and often unhelpful. CRediT was developed through a collaboration of research funders, publishers, researchers and institutions with the aim was to provide greater granularity, visibility and transparency to the myriad of contributions that researchers make to published scholarly research output today.
This talk explains the origins, rationale and current use of the Contributor Role Taxonomy (CRediT) – leaving room for discussion of the challenges and opportunities for the future of this initiative.

Collaborative Assessment for Trustworthy Science: the repliCATS project by Prof Fiona Fidler
2020-07-23 11:00 - 12:00 UTC | MS Teams
About the speaker
Fiona has a degree in Psychology, with a second major in Sociology, and was awarded her PhD in History and Philosophy of Science at the University of Melbourne in 2006. She is now a Professor at the University of Melbourne, based between the School of BioSciences and the School of Historical and Philosophical Studies. Fiona is also an Australian Research Council Future Fellow, leading a range of meta-research projects across ecology and conservation science, as well as projects in psychology, and other social science fields. Her meta-research work is driven by an underlying interest in how scientists and other experts reason, make and justify decisions, and change their minds. She is co-lead (with Simine Vazire) of MetaMelb, a recently launched meta-science/meta-research group at the University of Melbourne, and she leads the repliCATS project, for Collaborative Assessments for Trustworthy Science.
About the talk
The repliCATS project evaluates published scientific research. As the acronym—Collaborative Assessments for Trustworthy Science—suggests, repliCATS is a group activity, centred around assessing the trustworthiness of research claims. Reviewers first make private individual assessments about a research claim—judging its comprehensibility, the prior plausibility of underlying effect, and its likely replicability. Reviewers then share their judgements and reasoning with group members, providing both new information and the opportunity for feedback and calibration. The group interrogates differences in opinion and explores counterfactuals. After discussion, there is a final opportunity for privately updating individual judgements. Importantly the repliCATS process is not consensus-driven – reviewers can disagree, and their ratings and probability judgements are mathematically aggregated into a final assessment. At the moment, the repliCATS platform exists primarily to predict replicability. Launched in January 2019 as part of the DARPA SCORE program, over 18 months repliCATS elicited group assessments and captured associated reasoning and discussion, for 3,000 published social scientific research claims in 8 disciplines (Business, Criminology, Economics, Education, Political Science, Psychology, Public Administration, and Sociology). The repliCATS team are now working to extend the platform beyond merely predicting replicability, to deliver a more comprehensive peer review protocol. Suspected advantages of a repliCATS process over traditional peer review include: inbuilt training and calibration; feedback that is intrinsically rewarding; an inherently interactive process, but one which does not implicitly rely on ‘consensus by fatigue’; and a process that actively encourages interrogation. This talk will present some preliminary findings, and discuss the future of the platform.

Synthetic data: A primer by Dr Dan Quintana
2020-07-30 12:00 - 13:00 UTC | MS Teams
About the speaker
Daniel S. Quintana is a researcher in biological psychiatry at the University of Oslo. His research interests include social behaviour, the oxytocin system, heart rate variability, and meta-analysis. He currently leads a project investigating the role of the oxytocin system in the development of severe mental illness and metabolic disorder, which is supported by an Excellence Project for Young Researchers grant from the Novo Nordisk Foundation. Dan is also known for the Everything Hertz podcast, which he co-hosts with James Heathers. Widely praised for his training materials to support the adoption of open research practices, Dan is a strong advocate for disseminating scholarly outputs with the aid of social media, which was the subject of his talk for the Rotterdam RIOTS Club in May.
About the talk
Open research data provide considerable scientific, societal, and economic benefits. However, disclosure risks can sometimes limit the sharing of open data, especially in datasets that include sensitive details or information from individuals with rare disorders. This talk introduces the concept of synthetic datasets, which is an emerging method originally developed to permit the sharing of confidential census data. Synthetic datasets mimic real datasets by preserving their statistical properties and the relationships between variables. Importantly, this method also reduces disclosure risk to essentially nil as no record in the synthetic dataset represents a real individual. This practical guide with accompanying R script enables biobehavioural researchers to create synthetic datasets and assess their utility via the synthpop R package. By sharing synthetic datasets that mimic original datasets that could not otherwise be made open, researchers can ensure the reproducibility of their results and facilitate data exploration while maintaining participant privacy.

Pitfalls of functional MRI by Dr Marion Criaud
2020-09-24 13:00 - 14:00 UTC | MS Teams
About the speaker
Dr Marion Criaud is a postdoctoral research associate in the Child and Adolescent Psychiatry department at King’s College London. Her current work focuses on developing a new non-medical treatment for children with ADHD called fMRI NeuroFeedback. She has a keen interest in identifying best practice in fMRI and understanding the limitations when applied in psychiatry.
About the talk
Functional MRI is not highly reproducible, and a lot of factors are influencing this. Some we can control, and some we can’t. fMRI data are extremely sensitive; we cannot always control the quality. But there are things we can do to improve reliability in fMRI. We are going to go over the fragility of fMRI data and what we can do to ensure the data is managed in the best way possible.

Why p-values can't tell you what you need to know and what to do about it by Prof David Colquhoun
2020-10-01 12:00 - 13:00 UTC | MS Teams
About the speaker
Professor David Colquhoun is a pharmacologist notable for his insights into the biophysics of drug–receptor interactions of single ion channels. He was elected a Fellow of the Royal Society (FRS) in 1985, was awarded the Humboldt Prize in 1990 and made Honorary Fellow of UCL in 2004. Colquhoun joined the Pharmacology Department at UCL in 1964, and, apart from 9 years, has remained there ever since. He can be found on twitter (@david_colquhoun) and via his website DC’s Improbable Science, where he writes critically about pseudoscience and alternative medicine.
About the talk
It is a truth universally acknowledged that the literature has too many false positives. One reason, among many, for this is misunderstanding of p-values. There are three main reasons why a p-value can’t tell you much about whether a hypothesis is true or not. It transposes the conditional, It has the wrong denominator. It takes into account values that have not been observed. These will be explained, and suggestions made about possible solutions.

Five selfish reasons to work reproducibly by Dr Florian Markowetz
2020-10-08 14:00 - 15:00 UTC | MS Teams
About the speaker
Florian Markowetz is a Senior Group Leader at the Cancer Research UK Cambridge Institute. He is a Royal Society Wolfson Research Merit Award holder and received a CRUK Future Leader in Cancer Research prize. He holds degrees in Mathematics (Dipl. math.) and Philosophy (M.A.) from the University of Heidelberg and a Dr. rer. nat. (PhD equivalent) in Computational Biology from Free University Berlin, for which he was awarded an Otto-Hahn Medal by the Max Planck Society. His group at the CRUK Cambridge Institute combines computational work on cancer evolution and image analysis of the tumour tissue with experimental work on understanding key cancer mechanisms like the estrogen receptor.
About the talk
And so, my fellow scientists: ask not what you can do for reproducibility; ask what reproducibility can do for you! In my talk, I will present five reasons why working reproducibly pays off in the long run and is in the self-interest of every ambitious, career-oriented scientist.

Easing into open science: A guide for graduate students and their advisors by Dr Priya Silverstein
2020-10-15 12:00 - 13:00 UTC | MS Teams
About the speaker
Dr Priya Silverstein is a postdoctoral researcher in the CoGDeV Lab at the University of Surrey, studying the relationship between Lego construction ability, spatial thinking, and numeracy achievement in children. She is passionate about methods for studying development, and how we can do better, more transparent, reproducible, and diverse science.
About the talk
This talk provides a roadmap for engaging in open science practices, suggesting eight open science practices that novices could begin adopting today. The topics that will be covered include journal clubs, project workflow, preprints, reproducible code, data sharing, transparent writing, preregistration, and registered reports. To address concerns about not knowing how to engage in open science practices, we provide a difficulty rating of each behaviour (easy, medium, difficult), present them in order of suggested adoption, and follow the format of what, why, how, and worries. We emphasise that engaging in open science behaviours need not be an all or nothing approach and that you can engage with any number of the behaviours outlined. We also discuss the specific benefits of engaging with these practices now, at a time when a lot of our research has been disrupted.

A walk in the garden of forking paths by Dr Julia Rohrer
2020-10-22 12:00 - 13:00 (Europe/London) | MS Teams
About the speaker
I am a personality psychologist by training and my work covers a broad range of topics, including the effects of birth order, age patterns in personality, and the correlates and determinants of subjective well-being. My methodological interests include causal inference on the basis of observational data and data analytic flexibility. I am an active advocate for increased research transparency and have frequently given talks on the topic.
I recently finished my doctoral degree as a fellow of the International Max Planck Research School on the Life Course and am now a lecturer (Akademische Assistentin) at the Department of Psychology, University of Leipzig.
Together with Anne Scheel, Malte Elson, and Ruben Arslan, I blog at The 100% CI. If you want to get in touch with me, I’m fairly active on Twitter.
To learn more, go to https://juliarohrer.com/.
About the talk
Data analysis requires researchers to make many decisions — and sometimes, they may not know which choices are most appropriate. In this talk, I will give an overview of ways to tackle researcher degrees of freedom in a transparent manner (such as robustness checks, multiverse and specification curve analyses), highlight their commonalities, and discuss some crucial concerns.

Theory building and testing in psychological research by Dr Eiko Fried
2020-10-29 14:00 - 15:00 (Europe/London) | MS Teams
About the speaker
Dr Eiko Fried is an Assistant Professor in clinical psychology at Leiden University working in the fields of clinical psychology, psychiatry, epidemiology and methodology. His main focus is on studying individual symptoms of mental disorders and their causal relations. Broader, his interests include measurement (how to best assess whether someone is ill), modelling (what statistical models are most appropriate to model psychopathology), ontology (what are mental disorders) and nosology (how do we best classify them).
About the talk
The last decade has brought reforms to improve methodological practices, with the goal to increase the reliability and replicability of effects. However, explanations of effects remain scarce, and a growing chorus of scholars argues that the replicability crisis has distracted from a crisis of theory. In the same decade, the empirical literature using factor and network models has grown rapidly. In this talk, I discuss three ways in which this literature falls short of theory building and testing. First, statistical and theoretical models are conflated, leading to invalid inferences such as the existence of psychological constructs based on factor models, or recommendations for clinical interventions based on network models. I demonstrate this inferential gap in a simulation study on statistical equivalence: excellent model fit does little to corroborate a theory, regardless of quality or quantity of data. Second, researchers fail to explicate theories about psychological constructs, but use implicit causal beliefs to guide inferences. These latent theories have led to problematic best practices in psychological research where inferences are drawn based on one specific causal model that is assumed, but not explicated. Third, explicated theories are often weak theories: narrative and imprecise descriptions vulnerable to hidden assumptions and unknowns. They fail to make clear predictions, and it remains unclear whether statistical effects corroborate such theories or not. Weak theories are immune to refutation or revision. I argue that these three challenges to theory building and testing are common and harmful, and impede theory formation, failure, and reform. A renewed focus on theoretical psychology and formal models offers a way forward.
Related preprint: https://psyarxiv.com/zg84s/

Visualising variability and uncertainty in R by Jack Taylor
2020-11-05 12:00 - 13:00 (Europe/London) | MS Teams
About the speaker
Jack Taylor is a PhD student at the University of Glasgow. He is in interested in how we represent words, particularly how we represent and access concepts associated with words, like emotion and imageability. Jack is also interested in the extent to which we predict the visual features of words, given a semantic context. Jack also has a keen interest in supporting reproducible research practices, including data visualisation among others, and you can find some useful links below. Jack can also be found on Twitter @JackEdTaylor.
Useful links:

LexOPS is an R package I’ve written for generating word stimuli, to use in Psychology experiments.
Hack Your Data Beautiful (HYDB) was an introductory workshop for R, sponsored by the Scottish Graduate School of Social Science.
Shiny Tutorials introduce Shiny to R users with a focus on Psychology research.
GitHub is where I share most of my code.
Twitter is where I am occasionally spotted.

About the talk
A picture is worth a thousand words, and good data visualisation is worth a thousand summary statistics. I’ll argue that good data visualisation is a key component of open and transparent science. I’ll highlight some example ways of visualising data transparently, focusing on presenting individual observations, visualising uncertainty, and embracing variability. Because it’s well-known, I’ll show some implementations in the ggplot2 package of R, highlighting some really useful functions and extensions for presenting informative features of data and statistical models.

Open science with OSF by Dr Courtney Soderberg
2020-11-12 14:00 - 15:00 UTC | MS Teams
About the speaker
Courtney Soderberg is the Statistician and Data Scientist at the Center for Open Science. Before working at COS, she received her PhD in Social Psychology from the University of California, Davis.
About the talk
Open research practices around data, materials, code, and articles are becoming more common in many fields, but researchers often aren’t trained in how to best make their research open, or what they need to consider when doing this. Using OSF as an example tool, I will walk through efficient ways for researchers to make their work more open and reproducible. I’ll also discuss what can be done when there are ethical or legal reasons why some data/materials cannot be openly shared.

Measurement schmeasurement: Questionable measurement practices and how to avoid them by Dr Jessica Flake
2020-11-19 15:00 - 16:00 (Europe/London) | MS Teams
About the speaker
For details about the speaker, please see https://www.jessicakayflake.com.
About the talk
Questionable measurement practices are decisions researchers make that raise doubts about the validity of the measures, and ultimately the validity of study conclusions. They make it impossible to evaluate a wide range of potential validity threats to the study’s conclusions. I will demonstrate that psychology is plagued by a measurement schmeasurement attitude: QMPs are common, offer a stunning source of researcher degrees of freedom, pose a serious threat to cumulative psychological science, but are largely ignored. I will review a list of questions researchers and consumers of scientific research can consider to identify and avoid QMPs. Transparent answers to these measurement questions promote rigorous research, allow for thorough evaluations of a study’s inferences, and are necessary for meaningful replication studies.

How do we ask better questions and get better answers? by Prof Stephen Cole
2020-11-20 14:00 - 15:00 UTC | MS Teams
About the speaker
Stephen R. Cole is a Professor in Epidemiology at UNC Gillings School of Global Public Health. Stephen R. Cole works to build accurate and impactful knowledge, particularly population-health (epidemiologic) knowledge. Professor Cole is interested in study designs and analyses that accurately estimate parameters of central interest to population-health scientists, such as risk. These study designs include randomized experiments, pseudoexperiments (i.e., observational studies) and thought-experiments (e.g., simulation studies).
About the talk
What is a good question? Provisionally, good questions have proper syntax, are semantically meaningful, answerable, and important. Good epidemiologic questions require a measurable outcome, including the context; treatments or actions, if causal; and translate unambiguously to a parameter of interest. What is a good answer? Again provisionally, good answers are as robust and as accurate as possible. We present results from a simulation experiment of a minimal nontrivial example, motivated by epidemiology, which illustrates some features of good answers using estimators from nonparametric, parametric, and semiparametric models. In summary, an estimator from a semiparametric model performed best among candidate estimators that were not omnisciently correct.

Drifting into the world of open science and reproducibility by Dr Gavin Buckingham
2020-11-26 14:30 - 15:30 (Europe/London) | MS Teams
About the speaker
Gavin Buckingham was awarded his PhD in Psychology from the University of Aberdeen in 2008. Following the completion of his PhD, he moved to Canada to take up a position as a postdoctoral fellow in the Brain and Mind Institute at Western University in Canada working with Prof Mel Goodale. His work here mostly focused on how we perceive weight and interact with objects in the world around us. In January 2013, he was appointed as a lecturer in the Psychology Department at Heriot-Watt University in Edinburgh. Three years later he joined the Department Sport and Health Sciences at the University of Exeter, where he is currently a senior lecturer and leads the Object Interaction Lab.
About the talk
I am going to talk about my perspectives on open science and reproducibility from the (slightly unusual) perspective of not being a significant part of the open science ecosystem. I have not developed any reproducibility initiatives. I have not devoted my career to improving science. I do not force my trainees to do things ‘my way’. But, since taking up a permanent position, I have been making small tangible steps toward improving my scientific practices. In this talk, I’ll give my reflections on how these many incremental changes to my research practices have improved the quality of my work which, hopefully, will be of interest to other scientists hoping to move in the right direction, but unsure where best to start.

Reproducible Research Oxford by Dr Malika Ihle
2020-12-03 12:00 - 13:00 UTC | MS Teams
About the talk
How can an open research working group help drive a research culture change at the institutional level? Reproducible Research Oxford a.k.a. RROx (“rocks”) has been working to achieve this by (i) building cross-disciplinary communities, (ii) providing training and promoting practices that are beneficial to research communities, and sustainable for researchers, and (iii) liaising with stakeholders to inform the design of policy.
About the speaker

Malika Ihle is the Reproducible Research Oxford Coordinator, supporting the Steering Group in developing a comprehensive approach to open scholarship and reproducible research that extends across all disciplines, using both bottom-up and top-down strategies.
Malika holds a Master degree in Biology from Université de Bourgogne and Université du Québec à Montréal, and a PhD in Behavioral Ecology from the Max Planck Institute for Ornithology. During her post-docs at the University of Sheffield and University of Florida, she organised a post-conference symposium on open science, she co-authored an invited idea in the journal Behavioral Ecology titled “Striving for transparent and credible research: practical guidelines for behavioural ecologist”, and she developed and taught a graduate course guiding students through the scientific pipeline, from preregistration to a reproducible and open workflow.
Malika is an Executive Committee member of the Society for Open Reliable Transparent Ecology and Evolutionary biology (SORTEE) and the UK Reproducibility Network (UKRN) local network lead at the University of Oxford.

Conflicts and complexities on the path to open data by Prof Louise Connell
2020-12-10 12:00 - 13:00 UTC | MS Teams
About the talk
Even with the best intentions and motivations, sharing your data in an open repository is not always as easy as it should be. Barriers can appear in a variety of forms, including uncertainty regarding copyright and data ownership, diverging interpretations of GDPR and ethical issues, and sheer lack of time and resources to curate your data into a shareable format. I discuss some ways my collaborators and I have navigated these barriers and share some tips and tricks for making the process as painless as possible.
About the speaker
My research interests surround cognitive science and embodied cognition. Specifically, I’m interested in how mental representations and conceptual knowledge are grounded, how we access and use these representations in language, and how we can create computational models to better understand human cognitive behaviour. I use a range of interdisciplinary methods from experimental psychology and cognitive modelling to corpus linguistics and machine learning. Some recent work has examined how sensorimotor experience (i.e., what we sense via different perceptual modalities, what we do via different action effectors) underpins word meaning, how distributional statistics from language capture important information about conceptual and social knowledge, how information about space and time is mentally represented, and how people combine concepts into new entities.

Open knowledge: A silent revolution in science by Alexandra Elbakyan
2021-01-28 12:00 - 13:00 UTC | MS Teams
About the speaker
A full autobiography is available here.
About the talk
On the 5th September 2011, aged 23, Alexandra Elbakyan created a PHP script that downloaded articles for free to make them open access to all. That script came to be known as Sci-Hub, “the first pirate website in the world to provide mass and public access to tens of millions of research papers” (more information here). Now, in its 10th year, Sci-Hub has become one of – if not the – most popular means to access academic literature hidden behind a paywall. Sci-Hub’s creator will join us to talk about the origin of Sci-Hub, where it is now, and what the future holds. The talk will be composed of a 25 minute pre-recorded talk followed by a live Q&A.

The IGDORE project: Reinventing academia with open (science) mind by Dr Rebecca Willén
2021-02-04 12:00 - 13:00 UTC | Zoom, United Kingdom
About the speaker

Dr Willén is an independent researcher in psychology. Her main professional interests are research ethics, research methodology and investigative psychology (forensic psychology). She is the founder of IGDORE, an independent virtual research institute, and a strong proponent of open and replicable science. In 2016, she finished her PhD at University of Gothenburg, Sweden, with a thesis on police interviewing techniques and verbal deception detection. The thesis included retroactive disclosure statements (RDS; p’s 27-28 in the thesis) which she followed up on in 2018 with additional statements published on her website. Her decision to publish RDS has been covered in an episode of Everything Hertz, mentioned in a piece in Undark Magazine, and she has co-authored a paper (in press at Perspectives on Psychological Science) on similar statements. Curate Science has turned her RDS into a reporting standard badge called ‘Basic 7 (retroactive)’. After 5 years abroad (Indonesia, Canada, Netherlands), she returned to Sweden in December 2019. She now resides in Gothenburg with her two children and three rescue dogs. She’s a first generation university student, liberal leftist, and a proponent of viewpoint diversity.
You can find out more about Dr Willén here:
CV – ORCID – Publons – Google Scholar – Twitter – GitHub – Blog
About the talk

Institute for Globally Distributed Open Research and Education (IGDORE) is an independent research institute dedicated to improving the quality of science, science education, and quality of life for scientists, students and their families. Founded in 2016 with one single researcher, the founder herself, the institute today has over 110 researchers from 35 countries. Dr Willén’s talk will provide a brief background to why the IGDORE project was launched, an introduction to its goals, visions, and organisation, and how IGDORE hopes to further advance the movement for open and replicable science.

Chiral materials and finding your voice as an early career researcher by Dr Jess Wade
2021-02-10 12:30 - 13:30 UTC | MS Teams
About the speaker
Bio available here and here.
About the talk
Dr Jess Wade is a Research Fellow at Imperial College London investigating spin selective charge transport through chiral systems in the Department of Materials. As well as speaking on this work and her career thus far, Jess will talk on work outside the lab. This work has included campaigning for greater diversity in science, such as involving more women in physics and STEM subjects more broadly, as well as tackling systemic gender and racial biases Wikipedia. Please join us for what will be a fascinating talk on the barriers in science and how to navigate them as an early career researcher.
To join, please follow this link.

Learning the hard way: Reproducible workflows by Dr James Bartlett
2021-02-16 13:00 - 14:00 UTC | MS Teams
About the speaker
Dr James Bartlett completed his PhD at Coventry University on the neurocognitive mechanisms associated with addictive behaviour in daily and non-daily smokers. James is a lecturer in psychology at Arden University and leads undergraduate and postgraduate modules on research methods. He specialises in quantitative research methods and his research focuses on addictive behaviour. All his research and resources are available here.
About the talk
The lifecycle of a research project spans months and often years, with plenty of disruptions along the way. From preregistration to publication, there is plenty of room for error as you forget what settings you used for your analyses or make a mistake cleaning your data. In this seminar, I will outline how establishing a reproducible workflow helps yourself and the scientific community. Changing your approach can be daunting, so I will be outlining the mistakes I made along the way including a spreadsheet mix-up and a cringy preregistration. By the end of the seminar, you will see how you can incrementally create your own reproducible workflow.

Introduction to preregistration and registered reports by Dr Kaitlyn M. Werner
2021-02-19 13:00 - 14:00 UTC | Zoom, United Kingdom
About the speaker

I am a SSHRC Banting postdoctoral research fellow in the Affective Science and Health Laboratory (PI: Dr Brett Ford) at the University of Toronto. My research takes an interdisciplinary approach to understand the factors that facilitate successful self-regulation and goal pursuit. I also have a strong interest in measurement (e.g., construct and ecological validity), advanced quantitative methods (e.g., multilevel SEM, Bayesian statistics), research design (e.g., experimental, longitudinal, experience sampling), and meta-science (e.g., transparent and reproducible research practices). Outside of the lab, I am also an Ambassador for the Center for Open Science.​
To learn more about Dr Werner, visit her website.
About the talk
This session will provide an overview of preregistration and registered reports.

Rmarkdown and Github: A walkthrough for sharing code and results by Dr Oliver Pain
2021-02-25 14:00 - 15:00 UTC | MS Teams
About the speaker
Dr Oliver Pain is a postdoctoral research associate at the Social Genetic and Developmental Psychiatry (SGDP) Centre, King’s College London. His work uses statistical genetics to understand and predict outcomes related to mental health.
About the talk
This workshop will provide an introduction to Rmarkdown websites and how to host them online using GitHub pages. Oliver will take you through a few exercises to create a Rmarkdown website online.
Important note: for the practical, Oliver suggests using Rstudio and GitHub Desktop, but he will also provide instructions using R and Git on the command line. You will also need a GitHub account and to download the workshop materials in advance.

Psychology in a crisis by Dr Stuart Ritchie
2021-03-04 12:00 - 13:00 UTC | MS Teams
About the speaker
Dr Ritchie is a psychologist with research interests in a variety of areas, including intelligence, individual differences in cognitive abilities, behavioural genetics, and more recently in the replication crisis. In 2014, he completed his PhD at the university of Edinburgh, staying on to become a postdoctoral research fellow investigating difference in intelligence, working on the world renowned Lothian Birth Cohort dataset. In 2018, he became a lecturer in the Social, Genetic and Developmental Psychiatry Centre at King’s College London. In his recent book, Science Fictions, Stuart examines fraud, bias, error, and hype in scientific research, and the incentives that undermine scientific integrity. More recently, Stuart focused his energies on cross-examining scientific claims relevant to the coronavirus pandemic, and in pursuit of this has set up a Myth Busting website.
About the talk
At the beginning of the COVID-19 pandemic, psychologists and other behavioural scientists sprung into action, writing high-profile papers that described the application of their research findings to the crisis. Since changing people’s behaviours is a big part of controlling a pandemic, they saw many opportunities to use their research to help out. As we know, though, the behavioural sciences have been going through their own crisis over the past years – a crisis of confidence in the quality, replicability, and generalisability of their findings. How appropriate was it to try to apply these findings to a completely unprecedented scenario – and one where lives were at stake? What are the pitfalls in doing so? In this talk, I’ll discuss what’s happened with psychology and the pandemic, and propose a new scheme—partly stolen from NASA—to assess how ready any given piece of research is for translation to a crisis.
