The quest for better science: an interview with John Ioannidis

The Laboratoire d'informatique de l'X and the Centre interdisciplinaire Hi! PARIS jointly organised a visit to the École polytechnique on 10 November by John Ioannidis, Professor of Medicine and of Epidemiology and Population Health at the Stanford University School of Medicine. In a lecture entitled "Meta-research and the quest for better science", the researcher highlighted elements of the research world that he believes can be improved. For example, while 95% of research papers published since 1990 claim to have statistically significant results, most of them have low statistical credibility, often due to small data sets and non-reproducible methods, thus reducing the reliability of these results.

John Ioannidis also questions the relationship of scientists with the current publication system. He seeks to raise awareness of the refusal to refute certain research papers, the false positives and negatives linked to small samples, and the need to take into account the biases inherent in the various fields of research. His work focuses in particular on the development of tools for measuring the reliability of publications and on the reinvention of the criteria for evaluating researchers. Indeed, the main current criterion is the number of citations and research papers published. He therefore proposes others that would be added to it, such as the reproducibility of results and the sharing of data.

> Discover the conference in video

In order to elaborate and understand better his message, discover below three questions asked to John Ioannidis:

How did meta-research become the focus of your work?
Meta-research is inseparable from research, it is an integral part of the scientific method: it is about anticipating the pitfalls that the study may face and what biases may affect its results. I believe that the methods of a research study is more important than its conclusions, and I consider the scientists who criticise my studies to be my benefactors! Seeking to make all research papers more reliable, starting with my own, has been one of my first goals as a researcher.

Your most cited research paper is "Why Most Published Research Findings Are False", how did you come to this conclusion?
This work is the result of several years of research and data analysis on the various biases and factors that erode the quality of research. It is a framework that takes into account the processes of discovery and scientific validation of studies, with an empirical basis. Some fields of research have publications that are intended to be presented as narratives, but this kind of process can lead to simplifications, particularly by removing the uncertainties that do not serve the narrative.

Applying two different protocols to the same dataset can lead to entirely different results. Can we make the data say what we want? How can trust in scientific studies be restored?
While it is true that different analyses can give different results, this highlights the importance of showing all the options tested in a research paper, not just the ones that were convincing. Scientists should not be blamed, but we should be cautious about taking assertive statements as strict truth. Trust in science is at two different levels: between scientists on the one hand and between scientists and the general public on the other. For the first level, I believe that transparency and the sharing of knowledge and know-how enable the best possible understanding of the study and thus give confidence in the work done. For the second level, being able to say "we don't know" seems essential to me. We should be able to separate the scientific environment from the political and cultural environment, to leave science as impartial as possible.